[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:23:39.216455  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:44785
I20260709 12:23:39.218127  6451 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:23:39.219208  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:39.237905  6460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:39.237818  6457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:39.240219  6451 server_base.cc:1061] running on GCE node
W20260709 12:23:39.237809  6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:40.458894  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:40.459160  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:40.459483  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599820459455 us; error 0 us; skew 500 ppm
I20260709 12:23:40.460397  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:40.467422  6451 webserver.cc:533] Webserver started at http://127.6.76.254:38713/ using document root <none> and password file <none>
I20260709 12:23:40.468417  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:40.468688  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:40.469235  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:40.475526  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "0972b48461b74aefb3fd784a8b5872d8"
format_stamp: "Formatted at 2026-07-09 12:23:40 on dist-test-slave-n447"
I20260709 12:23:40.484395  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:23:40.490577  6466 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:40.492004  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 12:23:40.492476  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "0972b48461b74aefb3fd784a8b5872d8"
format_stamp: "Formatted at 2026-07-09 12:23:40 on dist-test-slave-n447"
I20260709 12:23:40.492921  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:40.550390  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:40.552134  6451 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:23:40.552675  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:40.631693  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:44785
I20260709 12:23:40.631762  6517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:44785 every 8 connection(s)
I20260709 12:23:40.640589  6518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:40.664538  6518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: Bootstrap starting.
I20260709 12:23:40.671005  6518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:40.673152  6518 log.cc:826] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: Log is configured to *not* fsync() on all Append() calls
I20260709 12:23:40.678375  6518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: No bootstrap required, opened a new log
I20260709 12:23:40.700301  6518 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER }
I20260709 12:23:40.700912  6518 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:40.701116  6518 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0972b48461b74aefb3fd784a8b5872d8, State: Initialized, Role: FOLLOWER
I20260709 12:23:40.702171  6518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER }
I20260709 12:23:40.702726  6518 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:40.702993  6518 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:40.703428  6518 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:40.708011  6518 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER }
I20260709 12:23:40.708912  6518 leader_election.cc:304] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0972b48461b74aefb3fd784a8b5872d8; no voters: 
I20260709 12:23:40.710794  6518 leader_election.cc:290] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:40.711187  6521 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:40.714349  6521 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 1 LEADER]: Becoming Leader. State: Replica: 0972b48461b74aefb3fd784a8b5872d8, State: Running, Role: LEADER
I20260709 12:23:40.715494  6521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER }
I20260709 12:23:40.716678  6518 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:40.727394  6523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0972b48461b74aefb3fd784a8b5872d8. Latest consensus state: current_term: 1 leader_uuid: "0972b48461b74aefb3fd784a8b5872d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER } }
I20260709 12:23:40.727385  6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0972b48461b74aefb3fd784a8b5872d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0972b48461b74aefb3fd784a8b5872d8" member_type: VOTER } }
I20260709 12:23:40.728291  6523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:40.728431  6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:40.736351  6530 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:40.749774  6530 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:40.752301  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:40.768635  6530 catalog_manager.cc:1383] Generated new cluster ID: 33cebfb66d284579800e0b341c717811
I20260709 12:23:40.768993  6530 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:40.797634  6530 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:40.799414  6530 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:40.824237  6530 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: Generated new TSK 0
I20260709 12:23:40.825395  6530 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:40.884202  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:40.894467  6541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:40.895707  6542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:40.900269  6544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:40.900322  6451 server_base.cc:1061] running on GCE node
I20260709 12:23:40.901595  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:40.901925  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:40.902149  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599820902130 us; error 0 us; skew 500 ppm
I20260709 12:23:40.903024  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:40.907684  6451 webserver.cc:533] Webserver started at http://127.6.76.193:32937/ using document root <none> and password file <none>
I20260709 12:23:40.908517  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:40.908850  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:40.909235  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:40.911013  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "918c6d087dde45a9bfde0b9f18936f2c"
format_stamp: "Formatted at 2026-07-09 12:23:40 on dist-test-slave-n447"
I20260709 12:23:40.919526  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:23:40.925431  6549 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:40.926487  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:23:40.926894  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "918c6d087dde45a9bfde0b9f18936f2c"
format_stamp: "Formatted at 2026-07-09 12:23:40 on dist-test-slave-n447"
I20260709 12:23:40.927327  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:40.984051  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:40.985580  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:40.988373  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:40.992775  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:40.993145  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:40.993462  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:40.993678  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.101068  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:40721
I20260709 12:23:41.101153  6611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:40721 every 8 connection(s)
I20260709 12:23:41.109364  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:41.120620  6616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:41.124260  6617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:41.135848  6619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:41.138520  6612 heartbeater.cc:344] Connected to a master server at 127.6.76.254:44785
I20260709 12:23:41.139007  6612 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:41.140102  6612 heartbeater.cc:507] Master 127.6.76.254:44785 requested a full tablet report, sending...
I20260709 12:23:41.140343  6451 server_base.cc:1061] running on GCE node
I20260709 12:23:41.141448  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:41.141798  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:41.142014  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599821141995 us; error 0 us; skew 500 ppm
I20260709 12:23:41.142899  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:41.144289  6483 ts_manager.cc:194] Registered new tserver with Master: 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721)
I20260709 12:23:41.146333  6451 webserver.cc:533] Webserver started at http://127.6.76.194:43583/ using document root <none> and password file <none>
I20260709 12:23:41.146867  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:41.147056  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:41.147372  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:41.147627  6483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38664
I20260709 12:23:41.148623  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "e613440a882e4b6c9149f57ebac09338"
format_stamp: "Formatted at 2026-07-09 12:23:41 on dist-test-slave-n447"
I20260709 12:23:41.156211  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:23:41.161571  6624 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.162596  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:41.162945  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "e613440a882e4b6c9149f57ebac09338"
format_stamp: "Formatted at 2026-07-09 12:23:41 on dist-test-slave-n447"
I20260709 12:23:41.163275  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:41.182186  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:41.183701  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:41.185410  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:41.188778  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:41.188997  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.189342  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:41.189518  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.299499  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:39553
I20260709 12:23:41.299638  6687 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:39553 every 8 connection(s)
I20260709 12:23:41.310287  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:41.321461  6692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:41.323648  6691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:41.324409  6688 heartbeater.cc:344] Connected to a master server at 127.6.76.254:44785
I20260709 12:23:41.324944  6688 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:41.326162  6688 heartbeater.cc:507] Master 127.6.76.254:44785 requested a full tablet report, sending...
I20260709 12:23:41.326890  6451 server_base.cc:1061] running on GCE node
W20260709 12:23:41.327452  6694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:41.328665  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:41.328946  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:41.328967  6483 ts_manager.cc:194] Registered new tserver with Master: e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:41.329214  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599821329195 us; error 0 us; skew 500 ppm
I20260709 12:23:41.330109  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:41.330778  6483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38674
I20260709 12:23:41.333418  6451 webserver.cc:533] Webserver started at http://127.6.76.195:41045/ using document root <none> and password file <none>
I20260709 12:23:41.334108  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:41.334360  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:41.334704  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:41.336210  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
format_stamp: "Formatted at 2026-07-09 12:23:41 on dist-test-slave-n447"
I20260709 12:23:41.342463  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:23:41.347174  6699 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.348242  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:41.348665  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
format_stamp: "Formatted at 2026-07-09 12:23:41 on dist-test-slave-n447"
I20260709 12:23:41.349080  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:41.375159  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:41.376940  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:41.378985  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:41.382403  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:41.382721  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.383019  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:41.383255  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:41.479045  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:38173
I20260709 12:23:41.479110  6761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:38173 every 8 connection(s)
I20260709 12:23:41.497879  6762 heartbeater.cc:344] Connected to a master server at 127.6.76.254:44785
I20260709 12:23:41.498301  6762 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:41.499296  6762 heartbeater.cc:507] Master 127.6.76.254:44785 requested a full tablet report, sending...
I20260709 12:23:41.501606  6483 ts_manager.cc:194] Registered new tserver with Master: 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:41.502266  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017166452s
I20260709 12:23:41.503052  6483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38676
I20260709 12:23:41.559998  6483 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38692:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:41.564221  6483 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:41.657068  6727 tablet_service.cc:1511] Processing CreateTablet for tablet cce9a98aa6ab41b8a92a778e96738f54 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:41.658931  6727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce9a98aa6ab41b8a92a778e96738f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.666184  6726 tablet_service.cc:1511] Processing CreateTablet for tablet ed747d61cc9e4bfeae5c25705d917fab (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:41.667831  6726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed747d61cc9e4bfeae5c25705d917fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.687332  6780 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:41.692880  6577 tablet_service.cc:1511] Processing CreateTablet for tablet cce9a98aa6ab41b8a92a778e96738f54 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:41.693548  6575 tablet_service.cc:1511] Processing CreateTablet for tablet f218385ed61b4334becc562630e707e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:41.696553  6577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce9a98aa6ab41b8a92a778e96738f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.702523  6653 tablet_service.cc:1511] Processing CreateTablet for tablet cce9a98aa6ab41b8a92a778e96738f54 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:41.704634  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce9a98aa6ab41b8a92a778e96738f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.707312  6575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f218385ed61b4334becc562630e707e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.707566  6780 tablet_bootstrap.cc:654] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:41.710448  6651 tablet_service.cc:1511] Processing CreateTablet for tablet f218385ed61b4334becc562630e707e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:41.702952  6652 tablet_service.cc:1511] Processing CreateTablet for tablet ed747d61cc9e4bfeae5c25705d917fab (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:41.721321  6727 tablet_service.cc:1511] Processing CreateTablet for tablet f218385ed61b4334becc562630e707e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:41.722783  6652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed747d61cc9e4bfeae5c25705d917fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.723645  6727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f218385ed61b4334becc562630e707e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.699628  6576 tablet_service.cc:1511] Processing CreateTablet for tablet ed747d61cc9e4bfeae5c25705d917fab (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:41.728871  6576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed747d61cc9e4bfeae5c25705d917fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.736258  6651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f218385ed61b4334becc562630e707e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.782644  6780 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:41.784899  6780 ts_tablet_manager.cc:1403] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.100s	user 0.004s	sys 0.020s
I20260709 12:23:41.788940  6780 raft_consensus.cc:359] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:41.789927  6780 raft_consensus.cc:385] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:41.790210  6780 raft_consensus.cc:740] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:41.791061  6780 consensus_queue.cc:260] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:41.796422  6762 heartbeater.cc:499] Master 127.6.76.254:44785 was elected leader, sending a full tablet report...
I20260709 12:23:41.797044  6780 ts_tablet_manager.cc:1434] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 12:23:41.798246  6780 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:41.808000  6780 tablet_bootstrap.cc:654] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:41.818562  6780 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:41.819089  6780 ts_tablet_manager.cc:1403] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.009s
I20260709 12:23:41.822428  6780 raft_consensus.cc:359] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:41.823288  6780 raft_consensus.cc:385] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:41.823642  6780 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:41.825152  6780 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:41.827668  6780 ts_tablet_manager.cc:1434] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:23:41.828976  6780 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:41.838223  6780 tablet_bootstrap.cc:654] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:41.840721  6784 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:41.846122  6780 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:41.847287  6780 ts_tablet_manager.cc:1403] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:23:41.848796  6784 tablet_bootstrap.cc:654] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:41.850301  6780 raft_consensus.cc:359] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:41.851161  6780 raft_consensus.cc:385] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:41.851572  6780 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:41.852594  6780 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:41.855520  6780 ts_tablet_manager.cc:1434] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:41.855865  6782 raft_consensus.cc:493] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.856467  6782 raft_consensus.cc:515] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:41.863137  6782 leader_election.cc:290] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721)
I20260709 12:23:41.869302  6784 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:41.869928  6784 ts_tablet_manager.cc:1403] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.000s
I20260709 12:23:41.873138  6784 raft_consensus.cc:359] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:41.874127  6784 raft_consensus.cc:385] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:41.874527  6784 raft_consensus.cc:740] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:41.875437  6784 consensus_queue.cc:260] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:41.887084  6784 ts_tablet_manager.cc:1434] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:23:41.888927  6789 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:41.896813  6789 tablet_bootstrap.cc:654] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:41.899362  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:41.900321  6587 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:41.908905  6727 tablet_service.cc:1511] Processing CreateTablet for tablet b20067e4eb864af3aabc75928bc82133 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:41.910439  6727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20067e4eb864af3aabc75928bc82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.913354  6726 tablet_service.cc:1511] Processing CreateTablet for tablet 89ccc56afe934355bd89d66dd67d43e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:41.915829  6701 leader_election.cc:304] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:41.915926  6653 tablet_service.cc:1511] Processing CreateTablet for tablet b20067e4eb864af3aabc75928bc82133 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:41.917042  6782 raft_consensus.cc:2804] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:41.917359  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20067e4eb864af3aabc75928bc82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.917448  6782 raft_consensus.cc:493] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.917853  6782 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:41.924642  6612 heartbeater.cc:499] Master 127.6.76.254:44785 was elected leader, sending a full tablet report...
I20260709 12:23:41.935541  6650 tablet_service.cc:1511] Processing CreateTablet for tablet 89ccc56afe934355bd89d66dd67d43e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:41.937213  6650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ccc56afe934355bd89d66dd67d43e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:41.937623  6613 tablet.cc:2406] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:41.941469  6649 tablet_service.cc:1511] Processing CreateTablet for tablet 6a399992a1a84bdfa163873fe1dc52a5 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:41.944742  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 5dbe0fcace364cc78073bb8a812cea6f (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:41.949617  6726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ccc56afe934355bd89d66dd67d43e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.943080  6649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a399992a1a84bdfa163873fe1dc52a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.984510  6572 tablet_service.cc:1511] Processing CreateTablet for tablet 703b9373822148949130c2160adcb8be (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:41.985983  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703b9373822148949130c2160adcb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.987298  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dbe0fcace364cc78073bb8a812cea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.014559  6577 tablet_service.cc:1511] Processing CreateTablet for tablet b20067e4eb864af3aabc75928bc82133 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:42.016182  6577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20067e4eb864af3aabc75928bc82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.025331  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 89ccc56afe934355bd89d66dd67d43e2 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:42.020545  6653 tablet_service.cc:1511] Processing CreateTablet for tablet cb239e3ddb3942398c4ce34c95a0674b (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:42.056059  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 5dbe0fcace364cc78073bb8a812cea6f (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:42.057785  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 6a399992a1a84bdfa163873fe1dc52a5 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:42.060453  6648 tablet_service.cc:1511] Processing CreateTablet for tablet 703b9373822148949130c2160adcb8be (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:42.062942  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703b9373822148949130c2160adcb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.062086  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ccc56afe934355bd89d66dd67d43e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.071552  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb239e3ddb3942398c4ce34c95a0674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.072319  6570 tablet_service.cc:1511] Processing CreateTablet for tablet cb239e3ddb3942398c4ce34c95a0674b (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:42.083765  6789 tablet_bootstrap.cc:492] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:42.086182  6789 ts_tablet_manager.cc:1403] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.198s	user 0.072s	sys 0.105s
I20260709 12:23:42.088836  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dbe0fcace364cc78073bb8a812cea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.097013  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a399992a1a84bdfa163873fe1dc52a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:42.097059  6532 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:42.103081  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb239e3ddb3942398c4ce34c95a0674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.152456  6724 tablet_service.cc:1511] Processing CreateTablet for tablet 703b9373822148949130c2160adcb8be (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:42.154094  6724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703b9373822148949130c2160adcb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.186630  6784 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:42.204566  6784 tablet_bootstrap.cc:654] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.211484  6782 raft_consensus.cc:515] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:42.216238  6723 tablet_service.cc:1511] Processing CreateTablet for tablet 5dbe0fcace364cc78073bb8a812cea6f (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:42.217954  6723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dbe0fcace364cc78073bb8a812cea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.230943  6722 tablet_service.cc:1511] Processing CreateTablet for tablet cb239e3ddb3942398c4ce34c95a0674b (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:42.232389  6722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb239e3ddb3942398c4ce34c95a0674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.246788  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.254815  6791 raft_consensus.cc:493] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.255827  6791 raft_consensus.cc:515] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.257016  6780 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.268584  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
W20260709 12:23:42.273962  6468 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.193:40721, user_credentials={real_user=slave}} blocked reactor thread for 295987us
I20260709 12:23:42.247845  6789 raft_consensus.cc:359] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:42.294641  6725 tablet_service.cc:1511] Processing CreateTablet for tablet 6a399992a1a84bdfa163873fe1dc52a5 (DEFAULT_TABLE table=test-workload [id=68868e9ccbac40f1a60755c377c1b0e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
W20260709 12:23:42.296062  6469 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.194:39553, user_credentials={real_user=slave}} blocked reactor thread for 213945us
I20260709 12:23:42.297219  6725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a399992a1a84bdfa163873fe1dc52a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.286540  6789 raft_consensus.cc:385] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.314296  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:42.304767  6789 raft_consensus.cc:740] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.316401  6587 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.312584  6782 leader_election.cc:290] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721)
I20260709 12:23:42.322073  6780 tablet_bootstrap.cc:654] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.333843  6688 heartbeater.cc:499] Master 127.6.76.254:44785 was elected leader, sending a full tablet report...
I20260709 12:23:42.334594  6587 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:42.347579  6701 leader_election.cc:304] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:42.350117  6782 raft_consensus.cc:2804] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:42.356303  6784 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:42.356906  6784 ts_tablet_manager.cc:1403] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.171s	user 0.020s	sys 0.000s
I20260709 12:23:42.364673  6784 raft_consensus.cc:359] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.365659  6784 raft_consensus.cc:385] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.366071  6784 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.377131  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:42.379494  6791 leader_election.cc:290] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.380355  6792 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.385901  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.349284  6789 consensus_queue.cc:260] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:42.367026  6784 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.380971  6792 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.394667  6782 raft_consensus.cc:697] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:42.396950  6663 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
W20260709 12:23:42.420092  6702 leader_election.cc:343] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:42.424567  6792 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.427729  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:42.429066  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
W20260709 12:23:42.429368  6701 leader_election.cc:343] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:42.430694  6702 leader_election.cc:343] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:42.431457  6702 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3; no voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338
I20260709 12:23:42.434201  6780 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:42.412786  6587 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:42.438700  6780 ts_tablet_manager.cc:1403] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.182s	user 0.021s	sys 0.007s
I20260709 12:23:42.437865  6792 raft_consensus.cc:2749] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:42.438409  6782 consensus_queue.cc:237] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:42.443895  6784 ts_tablet_manager.cc:1434] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.014s
I20260709 12:23:42.473889  6789 ts_tablet_manager.cc:1434] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.386s	user 0.018s	sys 0.002s
I20260709 12:23:42.475281  6789 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:42.477147  6784 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:42.478394  6701 leader_election.cc:304] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:42.470795  6780 raft_consensus.cc:359] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.479154  6780 raft_consensus.cc:385] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.479915  6780 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.480068  6792 raft_consensus.cc:2804] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:42.480544  6792 raft_consensus.cc:493] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.480926  6792 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.480866  6780 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.483999  6780 ts_tablet_manager.cc:1434] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:23:42.485067  6780 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.491564  6789 tablet_bootstrap.cc:654] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.494520  6780 tablet_bootstrap.cc:654] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.496059  6792 raft_consensus.cc:515] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.508100  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:42.509080  6587 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.513404  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:42.514912  6789 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:42.515529  6789 ts_tablet_manager.cc:1403] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.007s
W20260709 12:23:42.515609  6702 leader_election.cc:343] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:42.504876  6792 leader_election.cc:290] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.521345  6784 tablet_bootstrap.cc:654] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.521974  6789 raft_consensus.cc:359] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.526674  6482 catalog_manager.cc:5697] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.526324  6789 raft_consensus.cc:385] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.530597  6587 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:42.529376  6789 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.532430  6789 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.534082  6701 leader_election.cc:304] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:42.535264  6789 ts_tablet_manager.cc:1434] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:23:42.535316  6792 raft_consensus.cc:2804] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:42.535876  6792 raft_consensus.cc:697] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:42.535296  6780 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:42.536459  6789 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:42.536756  6792 consensus_queue.cc:237] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.537709  6780 ts_tablet_manager.cc:1403] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.009s
I20260709 12:23:42.542012  6780 raft_consensus.cc:359] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.546664  6789 tablet_bootstrap.cc:654] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.548169  6780 raft_consensus.cc:385] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.553383  6780 raft_consensus.cc:740] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.561650  6784 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:42.562361  6784 ts_tablet_manager.cc:1403] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.003s
I20260709 12:23:42.566262  6780 consensus_queue.cc:260] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.569630  6780 ts_tablet_manager.cc:1434] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 12:23:42.571082  6780 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.585003  6784 raft_consensus.cc:359] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.585947  6784 raft_consensus.cc:385] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.586387  6784 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.587498  6784 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.589903  6780 tablet_bootstrap.cc:654] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.597575  6789 tablet_bootstrap.cc:492] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:42.597889  6784 ts_tablet_manager.cc:1434] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 12:23:42.598090  6789 ts_tablet_manager.cc:1403] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.028s
I20260709 12:23:42.599833  6784 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:42.601814  6789 raft_consensus.cc:359] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.602567  6789 raft_consensus.cc:385] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.602850  6789 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.608549  6780 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:42.609227  6780 ts_tablet_manager.cc:1403] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.003s
I20260709 12:23:42.609099  6789 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.611486  6789 ts_tablet_manager.cc:1434] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 12:23:42.612496  6789 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:42.614880  6780 raft_consensus.cc:359] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.615722  6780 raft_consensus.cc:385] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.616082  6780 raft_consensus.cc:740] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.616933  6780 consensus_queue.cc:260] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.611078  6784 tablet_bootstrap.cc:654] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.632983  6780 ts_tablet_manager.cc:1434] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.005s
I20260709 12:23:42.634193  6780 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.637012  6789 tablet_bootstrap.cc:654] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.659634  6784 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:42.660897  6784 ts_tablet_manager.cc:1403] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.018s
I20260709 12:23:42.661574  6780 tablet_bootstrap.cc:654] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.667940  6784 raft_consensus.cc:359] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.668896  6784 raft_consensus.cc:385] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.669251  6784 raft_consensus.cc:740] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.670313  6784 consensus_queue.cc:260] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.665728  6481 catalog_manager.cc:5697] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.683914  6784 ts_tablet_manager.cc:1434] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.003s
I20260709 12:23:42.695871  6784 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:42.706648  6784 tablet_bootstrap.cc:654] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.708029  6780 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:42.708619  6780 ts_tablet_manager.cc:1403] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.006s
I20260709 12:23:42.709759  6789 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:42.710410  6789 ts_tablet_manager.cc:1403] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.098s	user 0.054s	sys 0.041s
I20260709 12:23:42.714594  6789 raft_consensus.cc:359] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.714440  6780 raft_consensus.cc:359] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.715761  6789 raft_consensus.cc:385] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.715847  6780 raft_consensus.cc:385] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.716267  6789 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.716410  6780 raft_consensus.cc:740] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.717453  6789 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.720746  6789 ts_tablet_manager.cc:1434] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:23:42.725188  6792 raft_consensus.cc:493] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.727066  6784 tablet_bootstrap.cc:492] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:42.723889  6780 consensus_queue.cc:260] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.728207  6784 ts_tablet_manager.cc:1403] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.004s
I20260709 12:23:42.727955  6792 raft_consensus.cc:515] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.732250  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:42.732667  6780 ts_tablet_manager.cc:1434] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.003s
I20260709 12:23:42.733083  6587 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:42.733114  6784 raft_consensus.cc:359] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.734138  6780 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.734396  6784 raft_consensus.cc:385] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.734774  6784 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.735486  6701 leader_election.cc:304] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:42.735742  6784 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.736909  6782 raft_consensus.cc:2804] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:42.738317  6782 raft_consensus.cc:493] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.738780  6782 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.741927  6784 ts_tablet_manager.cc:1434] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:23:42.743396  6784 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:42.745144  6792 leader_election.cc:290] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.746763  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.747726  6663 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:42.753906  6782 raft_consensus.cc:515] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.754477  6796 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.755779  6780 tablet_bootstrap.cc:654] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.755499  6796 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.722047  6789 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:42.763314  6796 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:42.771715  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:42.774575  6663 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.771893  6782 leader_election.cc:290] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.795753  6784 tablet_bootstrap.cc:654] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.803921  6780 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:42.806433  6780 ts_tablet_manager.cc:1403] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.016s
I20260709 12:23:42.811249  6789 tablet_bootstrap.cc:654] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.825367  6796 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.831349  6796 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.833746  6796 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:42.832083  6780 raft_consensus.cc:359] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.837692  6780 raft_consensus.cc:385] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.838747  6780 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.842046  6780 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.838780  6663 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:42.847378  6780 ts_tablet_manager.cc:1434] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.040s	user 0.018s	sys 0.000s
I20260709 12:23:42.848753  6780 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: Bootstrap starting.
I20260709 12:23:42.849509  6702 leader_election.cc:304] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:42.851328  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:42.852799  6587 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.851326  6782 raft_consensus.cc:2804] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:42.857601  6782 raft_consensus.cc:697] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:42.889679  6792 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.889398  6782 consensus_queue.cc:237] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.890298  6792 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.902370  6780 tablet_bootstrap.cc:654] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:42.905966  6789 tablet_bootstrap.cc:492] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:42.906708  6789 ts_tablet_manager.cc:1403] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.185s	user 0.026s	sys 0.014s
I20260709 12:23:42.909598  6663 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.910419  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:42.912587  6587 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:42.911437  6586 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:42.915525  6792 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:42.915809  6701 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:42.916932  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.918215  6792 raft_consensus.cc:2804] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:42.919407  6702 leader_election.cc:343] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:42.919760  6792 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.921883  6792 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.930739  6736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:42.932344  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:42.935988  6737 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
I20260709 12:23:42.942502  6797 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.947134  6784 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:42.944590  6552 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:42.947911  6784 ts_tablet_manager.cc:1403] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.020s
I20260709 12:23:42.939935  6782 raft_consensus.cc:493] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.950634  6796 raft_consensus.cc:2804] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:42.949460  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.955858  6796 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.958784  6796 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:42.959540  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:42.953189  6782 raft_consensus.cc:515] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.945545  6797 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.961653  6662 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
W20260709 12:23:42.971869  6552 leader_election.cc:343] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:42.972550  6684 debug-util.cc:398] Leaking SignalData structure 0x7b080012a060 after lost signal to thread 6454
I20260709 12:23:42.974802  6789 raft_consensus.cc:359] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.975855  6789 raft_consensus.cc:385] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:42.975917  6684 debug-util.cc:398] Leaking SignalData structure 0x7b080012a740 after lost signal to thread 6517
I20260709 12:23:42.976308  6789 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:42.971195  6794 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.978099  6794 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.978006  6784 raft_consensus.cc:359] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.984215  6784 raft_consensus.cc:385] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:42.977295  6789 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:42.987782  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:42.995982  6784 raft_consensus.cc:740] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.000618  6782 leader_election.cc:290] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.010991  6784 consensus_queue.cc:260] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.010568  6794 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.020733  6780 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: No bootstrap required, opened a new log
I20260709 12:23:43.021497  6780 ts_tablet_manager.cc:1403] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent bootstrapping tablet: real 0.173s	user 0.033s	sys 0.006s
I20260709 12:23:43.011313  6782 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:43.015920  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:43.023718  6586 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:43.045403  6780 raft_consensus.cc:359] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.013945  6797 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
W20260709 12:23:43.056071  6820 log.cc:927] Time spent T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:23:43.048532  6794 raft_consensus.cc:493] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.068123  6794 raft_consensus.cc:515] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.084436  6585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:43.085707  6794 leader_election.cc:290] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.047078  6780 raft_consensus.cc:385] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.087600  6780 raft_consensus.cc:740] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Initialized, Role: FOLLOWER
W20260709 12:23:43.088515  6701 leader_election.cc:343] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.088585  6780 consensus_queue.cc:260] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.091557  6796 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.086799  6792 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.097136  6736 raft_consensus.cc:2393] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 918c6d087dde45a9bfde0b9f18936f2c in current term 1: Already voted for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in this term.
W20260709 12:23:43.104671  6532 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:43.108011  6780 ts_tablet_manager.cc:1434] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: Time spent starting tablet: real 0.086s	user 0.011s	sys 0.001s
I20260709 12:23:43.110073  6552 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c; no voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.125479  6796 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.126701  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:43.127345  6796 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.141878  6789 ts_tablet_manager.cc:1434] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.235s	user 0.009s	sys 0.006s
I20260709 12:23:43.144083  6789 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:43.147162  6796 raft_consensus.cc:2749] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.153614  6784 ts_tablet_manager.cc:1434] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.204s	user 0.011s	sys 0.005s
W20260709 12:23:43.156255  6702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:39553, user_credentials={real_user=slave}} blocked reactor thread for 167866us
I20260709 12:23:43.163520  6792 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.176049  6823 raft_consensus.cc:493] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:43.173352  6552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:38173, user_credentials={real_user=slave}} blocked reactor thread for 63494.5us
I20260709 12:23:43.181267  6784 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:43.176695  6823 raft_consensus.cc:515] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.191303  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.197190  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.203099  6662 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 0.
I20260709 12:23:43.212260  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:43.209827  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.212600  6823 leader_election.cc:290] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.217346  6662 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.220620  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
W20260709 12:23:43.221103  6702 leader_election.cc:343] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.225936  6702 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:43.231510  6823 raft_consensus.cc:2804] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:43.235126  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
W20260709 12:23:43.235889  6702 leader_election.cc:343] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:43.236744  6702 leader_election.cc:343] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.235484  6823 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.237758  6823 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:43.241897  6702 leader_election.cc:343] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.234575  6587 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:43.259263  6736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.259905  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
W20260709 12:23:43.265208  6626 leader_election.cc:343] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.267772  6792 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.268802  6789 tablet_bootstrap.cc:654] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.271390  6823 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.275410  6736 raft_consensus.cc:2393] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e613440a882e4b6c9149f57ebac09338 in current term 1: Already voted for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in this term.
I20260709 12:23:43.278716  6823 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.281384  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:43.283802  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:43.284723  6784 tablet_bootstrap.cc:654] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.286204  6663 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.326921  6789 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:43.327836  6789 ts_tablet_manager.cc:1403] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.184s	user 0.029s	sys 0.002s
I20260709 12:23:43.332980  6784 tablet_bootstrap.cc:492] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:43.335377  6784 ts_tablet_manager.cc:1403] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.163s	user 0.016s	sys 0.018s
I20260709 12:23:43.334017  6663 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:43.338918  6702 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:43.343508  6792 raft_consensus.cc:2804] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:43.347577  6792 raft_consensus.cc:697] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:43.351114  6789 raft_consensus.cc:359] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.367709  6792 consensus_queue.cc:237] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.368875  6822 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.369530  6789 raft_consensus.cc:385] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.371631  6789 raft_consensus.cc:740] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.370781  6784 raft_consensus.cc:359] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.374528  6784 raft_consensus.cc:385] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.375761  6784 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.379894  6784 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.376586  6789 consensus_queue.cc:260] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.386302  6784 ts_tablet_manager.cc:1434] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.006s
I20260709 12:23:43.386353  6789 ts_tablet_manager.cc:1434] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.058s	user 0.011s	sys 0.003s
I20260709 12:23:43.388794  6789 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:43.393441  6784 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
W20260709 12:23:43.416690  6701 leader_election.cc:343] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.417240  6701 leader_election.cc:304] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3; no voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.418538  6782 raft_consensus.cc:2749] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.426474  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:43.427197  6737 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.428401  6481 catalog_manager.cc:5697] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.433943  6627 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e613440a882e4b6c9149f57ebac09338; no voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:43.436806  6828 raft_consensus.cc:2749] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.432880  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:43.443531  6661 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.425272  6701 leader_election.cc:304] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.447956  6823 raft_consensus.cc:2804] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:43.448513  6823 raft_consensus.cc:493] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.449016  6823 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.438939  6784 tablet_bootstrap.cc:654] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.448740  6782 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:43.448907  6701 leader_election.cc:343] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.455093  6701 leader_election.cc:304] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3; no voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.457639  6789 tablet_bootstrap.cc:654] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:43.457540  6701 leader_election.cc:343] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.458428  6782 raft_consensus.cc:2749] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.461886  6737 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:43.463311  6552 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:43.465124  6829 raft_consensus.cc:2804] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:43.465826  6829 raft_consensus.cc:697] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:43.464284  6701 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.467954  6829 consensus_queue.cc:237] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.468689  6794 raft_consensus.cc:2804] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:43.468917  6792 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.469354  6794 raft_consensus.cc:697] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:43.470031  6782 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.470664  6661 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:43.470345  6794 consensus_queue.cc:237] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.490538  6827 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.491458  6827 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.501864  6823 raft_consensus.cc:515] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.514326  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.516244  6661 raft_consensus.cc:2393] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 918c6d087dde45a9bfde0b9f18936f2c in current term 1: Already voted for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in this term.
I20260709 12:23:43.518407  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.505268  6827 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.519011  6823 leader_election.cc:290] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:43.521562  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:43.529932  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:43.535125  6584 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.534916  6552 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c; no voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338
W20260709 12:23:43.541610  6702 leader_election.cc:343] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:43.542309  6827 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.565583  6789 tablet_bootstrap.cc:492] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:43.566120  6789 ts_tablet_manager.cc:1403] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.178s	user 0.015s	sys 0.010s
I20260709 12:23:43.573280  6784 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:43.574156  6784 ts_tablet_manager.cc:1403] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.187s	user 0.035s	sys 0.023s
I20260709 12:23:43.569118  6789 raft_consensus.cc:359] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.575801  6789 raft_consensus.cc:385] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.576156  6789 raft_consensus.cc:740] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.577564  6784 raft_consensus.cc:359] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.578513  6784 raft_consensus.cc:385] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.578893  6784 raft_consensus.cc:740] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.580276  6784 consensus_queue.cc:260] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.583133  6784 ts_tablet_manager.cc:1434] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:43.584666  6784 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: Bootstrap starting.
I20260709 12:23:43.587886  6827 raft_consensus.cc:2749] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.588640  6584 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 1.
I20260709 12:23:43.591104  6789 consensus_queue.cc:260] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.596873  6701 leader_election.cc:304] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: e613440a882e4b6c9149f57ebac09338
I20260709 12:23:43.598083  6823 raft_consensus.cc:2804] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:43.598690  6823 raft_consensus.cc:697] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:43.599785  6823 consensus_queue.cc:237] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.600538  6784 tablet_bootstrap.cc:654] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.604233  6789 ts_tablet_manager.cc:1434] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.001s
I20260709 12:23:43.621990  6483 catalog_manager.cc:5697] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 1 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.624267  6789 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:43.641992  6789 tablet_bootstrap.cc:654] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.650590  6784 tablet_bootstrap.cc:492] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: No bootstrap required, opened a new log
I20260709 12:23:43.651139  6784 ts_tablet_manager.cc:1403] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: Time spent bootstrapping tablet: real 0.067s	user 0.010s	sys 0.012s
I20260709 12:23:43.658933  6480 catalog_manager.cc:5697] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.663192  6784 raft_consensus.cc:359] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.664250  6784 raft_consensus.cc:385] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.664616  6480 catalog_manager.cc:5697] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.666636  6789 tablet_bootstrap.cc:492] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:43.664619  6784 raft_consensus.cc:740] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.667280  6480 catalog_manager.cc:5697] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.668351  6784 consensus_queue.cc:260] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.669440  6789 ts_tablet_manager.cc:1403] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.012s
I20260709 12:23:43.674217  6784 ts_tablet_manager.cc:1434] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:23:43.676890  6789 raft_consensus.cc:359] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.677676  6789 raft_consensus.cc:385] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.677971  6789 raft_consensus.cc:740] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.678838  6789 consensus_queue.cc:260] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.682199  6789 ts_tablet_manager.cc:1434] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:23:43.683214  6789 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: Bootstrap starting.
I20260709 12:23:43.689877  6789 tablet_bootstrap.cc:654] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:43.705878  6789 tablet_bootstrap.cc:492] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: No bootstrap required, opened a new log
I20260709 12:23:43.706522  6789 ts_tablet_manager.cc:1403] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: Time spent bootstrapping tablet: real 0.024s	user 0.002s	sys 0.014s
I20260709 12:23:43.709940  6789 raft_consensus.cc:359] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.710796  6789 raft_consensus.cc:385] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:43.711474  6789 raft_consensus.cc:740] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Initialized, Role: FOLLOWER
I20260709 12:23:43.712435  6789 consensus_queue.cc:260] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.715107  6789 ts_tablet_manager.cc:1434] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:23:43.730930  6827 raft_consensus.cc:493] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.731647  6827 raft_consensus.cc:515] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.734395  6827 leader_election.cc:290] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.737790  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.738739  6661 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
I20260709 12:23:43.739141  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.739933  6737 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
I20260709 12:23:43.740545  6552 leader_election.cc:304] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:43.742779  6827 raft_consensus.cc:2804] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:43.743299  6827 raft_consensus.cc:493] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.743695  6827 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.755512  6827 raft_consensus.cc:515] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.762441  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:43.763273  6661 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.783651  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:43.784760  6737 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.795329  6661 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:43.797206  6552 leader_election.cc:304] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:43.798385  6796 raft_consensus.cc:2804] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:43.798998  6796 raft_consensus.cc:697] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:43.800227  6796 consensus_queue.cc:237] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.793326  6827 leader_election.cc:290] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.807574  6737 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:43.824848  6480 catalog_manager.cc:5697] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 1 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.846541  6797 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.847208  6797 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.849752  6797 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.852036  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.854478  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:43.856801  6586 raft_consensus.cc:2393] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e613440a882e4b6c9149f57ebac09338 in current term 1: Already voted for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in this term.
I20260709 12:23:43.858532  6626 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e613440a882e4b6c9149f57ebac09338; no voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:43.860184  6797 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.867838  6796 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.876510  6797 raft_consensus.cc:2749] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.882745  6823 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.891383  6797 raft_consensus.cc:493] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.892019  6797 raft_consensus.cc:515] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.913352  6796 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.916292  6822 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:43.916543  6827 raft_consensus.cc:493] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.917199  6827 raft_consensus.cc:515] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:43.917979  6822 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:43.920632  6827 leader_election.cc:290] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.923213  6797 leader_election.cc:290] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:43.925532  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c" is_pre_election: true
I20260709 12:23:43.926651  6584 raft_consensus.cc:2393] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e613440a882e4b6c9149f57ebac09338 in current term 1: Already voted for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in this term.
I20260709 12:23:43.931325  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.936527  6627 leader_election.cc:304] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e613440a882e4b6c9149f57ebac09338; no voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:43.937036  6736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" is_pre_election: true
I20260709 12:23:43.938035  6828 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.937932  6736 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
I20260709 12:23:43.943755  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338" is_pre_election: true
I20260709 12:23:43.944773  6661 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 0.
I20260709 12:23:43.966265  6552 leader_election.cc:304] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:43.968313  6827 raft_consensus.cc:2804] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:43.968755  6827 raft_consensus.cc:493] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.969094  6827 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:43.978330  6828 raft_consensus.cc:2749] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:43.984961  6822 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:44.020784  6827 raft_consensus.cc:515] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.070948  6827 leader_election.cc:290] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.077864  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.079772  6737 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:44.084193  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:44.085271  6661 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:44.095067  6737 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:44.119503  6823 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:44.124941  6532 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:44.126214  6552 leader_election.cc:304] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:44.121089  6661 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 1.
I20260709 12:23:44.129585  6823 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:44.130645  6827 raft_consensus.cc:2804] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:44.131631  6827 raft_consensus.cc:697] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.133107  6827 consensus_queue.cc:237] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.159610  6483 catalog_manager.cc:5697] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 1 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.250244  6737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.251047  6737 raft_consensus.cc:606] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:44.265766  6737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.266494  6737 raft_consensus.cc:606] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:44.276324  6737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.276980  6737 raft_consensus.cc:606] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:23:44.283521  6827 consensus_queue.cc:1048] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:44.285365  6737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.286537  6737 raft_consensus.cc:606] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:44.297324  6737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.298477  6737 raft_consensus.cc:606] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:44.334436  6584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "703b9373822148949130c2160adcb8be"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:44978
I20260709 12:23:44.335276  6584 raft_consensus.cc:606] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:44.335072  6829 consensus_queue.cc:1048] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:44.337270  6829 raft_consensus.cc:993] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.337891  6829 raft_consensus.cc:1081] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.350371  6736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:44.351405  6736 raft_consensus.cc:606] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:44.357299  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "703b9373822148949130c2160adcb8be"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50276
I20260709 12:23:44.358073  6661 raft_consensus.cc:493] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.358506  6661 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.361663  6584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:44978
I20260709 12:23:44.362501  6584 raft_consensus.cc:606] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Received request to transfer leadership to TS 4f9a79c23b5e41e9aa4ce456a79560e3
I20260709 12:23:44.369751  6661 raft_consensus.cc:515] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.373512  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:44.374292  6584 raft_consensus.cc:3055] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.374694  6584 raft_consensus.cc:740] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.375721  6584 consensus_queue.cc:260] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.377297  6584 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.377240  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703b9373822148949130c2160adcb8be" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.378419  6737 raft_consensus.cc:3060] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.386463  6584 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.387174  6661 leader_election.cc:290] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.388533  6626 leader_election.cc:304] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.389515  6737 raft_consensus.cc:2468] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.389938  6828 raft_consensus.cc:2804] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.390647  6828 raft_consensus.cc:697] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:44.392948  6828 consensus_queue.cc:237] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.404699  6483 catalog_manager.cc:5697] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 1 to 2, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.443691  6823 raft_consensus.cc:993] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.444285  6794 raft_consensus.cc:1081] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.446931  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:23:44.447685  6661 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.448170  6661 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.453403  6661 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.455400  6661 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.456427  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:44.457090  6584 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.466496  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.466847  6584 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.467258  6737 raft_consensus.cc:3055] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.467646  6737 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.468536  6737 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.468917  6626 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.470003  6737 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.470249  6828 raft_consensus.cc:2804] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.470789  6828 raft_consensus.cc:697] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:44.471891  6828 consensus_queue.cc:237] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.478309  6737 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.486564  6483 catalog_manager.cc:5697] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.505848  6823 raft_consensus.cc:993] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.507195  6782 raft_consensus.cc:1081] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.509745  6584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44960
I20260709 12:23:44.510452  6584 raft_consensus.cc:493] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.510855  6584 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.520408  6584 raft_consensus.cc:515] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.523481  6584 leader_election.cc:290] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.525065  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.525942  6737 raft_consensus.cc:3055] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.526326  6737 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.525065  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:44.527781  6661 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.527587  6737 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.529110  6737 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.538450  6661 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.540920  6552 leader_election.cc:304] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.542316  6829 raft_consensus.cc:2804] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.542971  6829 raft_consensus.cc:697] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.544341  6829 consensus_queue.cc:237] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.548151  6737 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.548918  6796 consensus_queue.cc:1048] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:44.556072  6822 raft_consensus.cc:993] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.556682  6794 raft_consensus.cc:1081] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.562531  6663 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:23:44.566256  6822 raft_consensus.cc:993] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.563994  6663 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.566751  6663 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.566838  6822 raft_consensus.cc:1081] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:44.569753  6662 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:23:44.570416  6662 raft_consensus.cc:493] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.570817  6662 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.575424  6827 raft_consensus.cc:993] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:44.576470  6796 raft_consensus.cc:1081] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Signalling peer 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:44.578310  6737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:39748
I20260709 12:23:44.582162  6737 raft_consensus.cc:493] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.583273  6737 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.590276  6822 raft_consensus.cc:993] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.590816  6822 raft_consensus.cc:1081] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.593755  6584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44960
I20260709 12:23:44.594548  6584 raft_consensus.cc:493] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.595042  6584 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.597929  6662 raft_consensus.cc:515] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.614228  6584 raft_consensus.cc:515] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.613250  6737 raft_consensus.cc:515] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.616212  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:44.617149  6586 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.625285  6827 consensus_queue.cc:1048] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:44.625681  6663 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.637241  6662 leader_election.cc:290] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.638761  6663 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.641076  6585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:44.642297  6584 leader_election.cc:290] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:44.642195  6585 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.643030  6736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.644109  6736 raft_consensus.cc:3055] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.645233  6736 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.644677  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.647620  6734 raft_consensus.cc:3055] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.648017  6734 raft_consensus.cc:740] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.646620  6735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.651091  6735 raft_consensus.cc:3055] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.651484  6735 raft_consensus.cc:740] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.648945  6734 consensus_queue.cc:260] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.652383  6735 consensus_queue.cc:260] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.650616  6823 raft_consensus.cc:993] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.655215  6794 raft_consensus.cc:1081] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:23:44.649412  6736 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.654114  6734 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.654106  6735 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.657182  6736 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.661993  6584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44960
I20260709 12:23:44.662838  6584 raft_consensus.cc:493] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.663447  6584 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.671296  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:44.672111  6737 leader_election.cc:290] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:44.673259  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb239e3ddb3942398c4ce34c95a0674b" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:44.673907  6587 raft_consensus.cc:3055] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.674281  6587 raft_consensus.cc:740] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.675186  6587 consensus_queue.cc:260] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.677947  6587 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.680042  6584 raft_consensus.cc:515] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:44.683797  6584 leader_election.cc:290] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Requested vote from peers 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:44.685945  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:44.680466  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:44.687767  6737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:44.683722  6480 catalog_manager.cc:5697] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 2 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.688783  6737 raft_consensus.cc:3055] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.689194  6737 raft_consensus.cc:740] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.687819  6663 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.688351  6662 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.686340  6661 raft_consensus.cc:3060] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.695873  6737 consensus_queue.cc:260] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:44.697587  6737 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.679149  6586 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.709375  6585 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.710374  6626 leader_election.cc:304] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.710768  6663 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.711467  6828 raft_consensus.cc:2804] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.711719  6736 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.711995  6828 raft_consensus.cc:697] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:44.712868  6552 leader_election.cc:304] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.713279  6626 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:44.713099  6828 consensus_queue.cc:237] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.713814  6829 raft_consensus.cc:2804] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.714118  6797 raft_consensus.cc:2804] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.714359  6829 raft_consensus.cc:697] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.715502  6829 consensus_queue.cc:237] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:44.720083  6552 leader_election.cc:304] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:44.721307  6827 raft_consensus.cc:2804] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.721926  6827 raft_consensus.cc:697] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:44.722923  6827 consensus_queue.cc:237] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.720335  6587 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 2.
I20260709 12:23:44.729830  6701 leader_election.cc:304] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:44.731045  6794 raft_consensus.cc:2804] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.731775  6794 raft_consensus.cc:697] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:44.732901  6794 consensus_queue.cc:237] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.747675  6662 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.744382  6735 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.752632  6797 raft_consensus.cc:697] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:44.751983  6734 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 2.
I20260709 12:23:44.759037  6480 catalog_manager.cc:5697] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 1 to 2, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:44.763270  6797 consensus_queue.cc:237] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:44.766757  6483 catalog_manager.cc:5697] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 2 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:44.768898  6483 catalog_manager.cc:5697] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 2 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.771487  6661 raft_consensus.cc:2468] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 2.
I20260709 12:23:44.786445  6737 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 2.
I20260709 12:23:44.790045  6587 raft_consensus.cc:1275] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.792001  6797 consensus_queue.cc:1048] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.808455  6737 raft_consensus.cc:1275] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.810655  6828 consensus_queue.cc:1048] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.821573  6483 catalog_manager.cc:5697] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.824460  6483 catalog_manager.cc:5697] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 1 to 2, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.913980  6737 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.916056  6828 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.926775  6587 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.937358  6828 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.154866  6661 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.156664  6827 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.166111  6587 raft_consensus.cc:1275] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.168596  6794 consensus_queue.cc:1048] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.177213  6737 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.180047  6829 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.201864  6737 raft_consensus.cc:1275] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.203920  6828 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.205814  6661 raft_consensus.cc:1275] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.212307  6794 consensus_queue.cc:1048] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.232901  6661 raft_consensus.cc:1275] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.237682  6829 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:23:45.268539  6662 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.269670  6735 raft_consensus.cc:1275] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.274713  6829 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.275185  6827 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.279611  6587 raft_consensus.cc:1275] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.282435  6828 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.294054  6736 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.296015  6827 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.322965  6732 raft_consensus.cc:1275] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.327893  6875 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.334178  6587 raft_consensus.cc:1275] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.336439  6828 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.414212  6451 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:23:47.415740  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.440297  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:23:47.441308  6661 raft_consensus.cc:606] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Received request to transfer leadership to TS 4f9a79c23b5e41e9aa4ce456a79560e3
I20260709 12:23:47.462553  6587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:44992
I20260709 12:23:47.463397  6587 raft_consensus.cc:606] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Received request to transfer leadership to TS 4f9a79c23b5e41e9aa4ce456a79560e3
I20260709 12:23:47.465408  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:47.466306  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:47.466264  6896 raft_consensus.cc:993] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:47.467199  6873 raft_consensus.cc:1081] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Signalling peer 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:47.468964  6735 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:39748
I20260709 12:23:47.469475  6735 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.469815  6735 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.482486  6735 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.485569  6735 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 2 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:47.487274  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:47.487906  6661 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.490412  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:47.491282  6587 raft_consensus.cc:3055] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.491998  6587 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:47.492975  6587 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.494534  6587 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.494935  6661 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 2.
I20260709 12:23:47.499887  6702 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:47.500911  6892 raft_consensus.cc:2804] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.503659  6892 raft_consensus.cc:697] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:47.505947  6892 consensus_queue.cc:237] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.515581  6587 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 2.
I20260709 12:23:47.518824  6483 catalog_manager.cc:5697] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 1 to 2, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.588335  6903 raft_consensus.cc:993] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: : Instructing follower 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:47.588940  6875 raft_consensus.cc:1081] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Signalling peer 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:23:47.592765  6734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:39754
I20260709 12:23:47.593393  6734 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.593811  6734 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.603190  6734 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.607383  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:47.607479  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:23:47.608199  6587 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.608633  6734 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 3 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:23:47.608199  6661 raft_consensus.cc:3055] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:47.612449  6661 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:47.613605  6661 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.615024  6661 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.617108  6587 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 3.
I20260709 12:23:47.618559  6701 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:23:47.619628  6892 raft_consensus.cc:2804] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:47.620165  6892 raft_consensus.cc:697] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:47.621081  6892 consensus_queue.cc:237] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:47.623124  6661 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 3.
I20260709 12:23:47.633021  6483 catalog_manager.cc:5697] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 2 to 3, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.903290  6661 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.906365  6892 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.941953  6587 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.943686  6908 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:48.138338  6587 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:48.140319  6908 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:48.156669  6661 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:48.158313  6897 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:48.468204  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.477949  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:48.478569  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:49.480852  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:49.488029  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:49.488685  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.490010  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.496434  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:50.497030  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.498255  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.503867  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:51.504576  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.505751  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.510818  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:52.511611  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:53.600291  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.606104  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:53.606570  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:23:54.287485  6758 debug-util.cc:398] Leaking SignalData structure 0x7b080016d060 after lost signal to thread 6454
W20260709 12:23:54.288491  6758 debug-util.cc:398] Leaking SignalData structure 0x7b080016d740 after lost signal to thread 6517
W20260709 12:23:54.289654  6758 debug-util.cc:398] Leaking SignalData structure 0x7b08001ece80 after lost signal to thread 6611
W20260709 12:23:54.290630  6758 debug-util.cc:398] Leaking SignalData structure 0x7b08001ec8e0 after lost signal to thread 6687
W20260709 12:23:54.291478  6758 debug-util.cc:398] Leaking SignalData structure 0x7b08001f2720 after lost signal to thread 6761
I20260709 12:23:54.608021  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.613561  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:54.614076  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:55.615793  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.623198  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:55.623883  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.625154  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.630494  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:56.631013  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:57.671813  6451 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:23:57.679502  6587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:44978
I20260709 12:23:57.680724  6587 raft_consensus.cc:606] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:57.689072  6734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:57.689749  6734 raft_consensus.cc:606] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:57.699194  6587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:44978
I20260709 12:23:57.700196  6587 raft_consensus.cc:606] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:57.708774  6587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:44978
I20260709 12:23:57.709668  6587 raft_consensus.cc:606] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:57.721735  6734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
mode: GRACEFUL
new_leader_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:23:57.722524  6734 raft_consensus.cc:606] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Received request to transfer leadership to TS e613440a882e4b6c9149f57ebac09338
I20260709 12:23:57.750417  6921 raft_consensus.cc:993] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.751009  6926 raft_consensus.cc:1081] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.753284  6662 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50276
I20260709 12:23:57.753880  6662 raft_consensus.cc:493] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.754256  6662 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.761767  6927 raft_consensus.cc:993] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.762544  6922 raft_consensus.cc:1081] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.762550  6662 raft_consensus.cc:515] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.765479  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:23:57.766137  6661 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.766518  6661 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.766762  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:57.767439  6662 leader_election.cc:290] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:57.767570  6734 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.766661  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:57.769455  6587 raft_consensus.cc:3055] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.769831  6587 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:57.770715  6587 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.772075  6587 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.774030  6734 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.775552  6627 leader_election.cc:304] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:57.776114  6661 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.776487  6917 raft_consensus.cc:2804] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.778719  6917 raft_consensus.cc:697] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:57.780494  6661 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:57.779979  6917 consensus_queue.cc:237] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.781141  6587 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.780694  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:57.782826  6586 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.782966  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:57.783792  6734 raft_consensus.cc:3055] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.784163  6734 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:57.785198  6734 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.786801  6734 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.791451  6586 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.792862  6626 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:57.793653  6917 raft_consensus.cc:2804] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.794377  6734 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.794437  6917 raft_consensus.cc:697] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:57.795809  6917 consensus_queue.cc:237] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.796103  6483 catalog_manager.cc:5697] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 2 to 3, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 3 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.806627  6921 raft_consensus.cc:993] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.807274  6926 raft_consensus.cc:1081] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:57.809149  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50276
I20260709 12:23:57.809832  6661 raft_consensus.cc:493] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.810163  6661 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.817822  6661 raft_consensus.cc:515] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.820025  6483 catalog_manager.cc:5697] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 2 to 3, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 3 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.821260  6661 leader_election.cc:290] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:57.821447  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:57.821992  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:57.822407  6586 raft_consensus.cc:3055] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.822641  6734 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.822877  6586 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:57.823915  6586 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.825167  6586 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.829353  6734 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.830720  6627 leader_election.cc:304] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:57.831703  6917 raft_consensus.cc:2804] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.832617  6917 raft_consensus.cc:697] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:57.832685  6586 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:57.834158  6917 consensus_queue.cc:237] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:57.844815  6483 catalog_manager.cc:5697] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 2 to 3, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 3 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.093003  6927 raft_consensus.cc:993] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:58.093564  6927 raft_consensus.cc:1081] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:58.095510  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:23:58.096176  6661 raft_consensus.cc:493] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.096477  6661 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:58.102535  6661 raft_consensus.cc:515] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:58.105130  6661 leader_election.cc:290] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 4 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:23:58.106438  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:58.106536  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a399992a1a84bdfa163873fe1dc52a5" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:58.107365  6734 raft_consensus.cc:3055] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:58.107684  6734 raft_consensus.cc:740] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:23:58.107613  6586 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:58.108714  6734 consensus_queue.cc:260] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:58.109813  6734 raft_consensus.cc:3060] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:58.113739  6586 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 4.
I20260709 12:23:58.115183  6626 leader_election.cc:304] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918c6d087dde45a9bfde0b9f18936f2c, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:58.115187  6734 raft_consensus.cc:2468] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 4.
I20260709 12:23:58.116168  6917 raft_consensus.cc:2804] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:58.116703  6917 raft_consensus.cc:697] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 4 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:58.117623  6917 consensus_queue.cc:237] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:23:58.126777  6483 catalog_manager.cc:5697] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 3 to 4, leader changed from 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 4 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.182796  6586 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.184638  6917 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.200331  6734 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.202545  6928 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:58.233686  6921 raft_consensus.cc:993] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: : Instructing follower e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:58.234206  6921 raft_consensus.cc:1081] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Signalling peer e613440a882e4b6c9149f57ebac09338 to start an election
I20260709 12:23:58.236407  6661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
 from {username='slave'} at 127.0.0.1:50276
I20260709 12:23:58.236976  6661 raft_consensus.cc:493] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.237353  6661 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.242666  6661 raft_consensus.cc:515] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:58.244778  6661 leader_election.cc:290] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Requested vote from peers 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173), 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721)
I20260709 12:23:58.245877  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:23:58.246256  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce9a98aa6ab41b8a92a778e96738f54" candidate_uuid: "e613440a882e4b6c9149f57ebac09338" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:23:58.246574  6734 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.247032  6586 raft_consensus.cc:3055] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.247442  6586 raft_consensus.cc:740] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:23:58.248176  6586 consensus_queue.cc:260] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:58.249300  6586 raft_consensus.cc:3060] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.252734  6734 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:58.254174  6627 leader_election.cc:304] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:23:58.255543  6928 raft_consensus.cc:2804] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:58.256129  6928 raft_consensus.cc:697] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Leader. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:23:58.256991  6928 consensus_queue.cc:237] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } }
I20260709 12:23:58.257323  6586 raft_consensus.cc:2468] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e613440a882e4b6c9149f57ebac09338 in term 3.
I20260709 12:23:58.266022  6481 catalog_manager.cc:5697] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 reported cstate change: term changed from 2 to 3, leader changed from 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193) to e613440a882e4b6c9149f57ebac09338 (127.6.76.194). New cstate: current_term: 3 leader_uuid: "e613440a882e4b6c9149f57ebac09338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.321429  6586 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.323324  6917 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:58.324942  6734 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.326447  6917 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.376853  6586 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.378762  6940 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.393203  6734 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.395155  6917 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:58.506436  6735 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:58.508018  6917 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:58.519915  6587 raft_consensus.cc:1275] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:58.521667  6928 consensus_queue.cc:1048] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:58.685053  6735 raft_consensus.cc:1275] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.686764  6928 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.698501  6587 raft_consensus.cc:1275] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer e613440a882e4b6c9149f57ebac09338: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.700435  6928 consensus_queue.cc:1048] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:00.782590  6451 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:24:00.783690  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:00.791862  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:24:00.793157  6661 raft_consensus.cc:606] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Received request to transfer leadership to TS 4f9a79c23b5e41e9aa4ce456a79560e3
I20260709 12:24:00.797248  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:24:00.797927  6661 raft_consensus.cc:606] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:24:00.801227  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:24:00.801867  6661 raft_consensus.cc:606] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:24:00.805394  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:24:00.806010  6661 raft_consensus.cc:606] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Received request to transfer leadership to TS 4f9a79c23b5e41e9aa4ce456a79560e3
I20260709 12:24:00.809082  6661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "e613440a882e4b6c9149f57ebac09338"
mode: GRACEFUL
new_leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:50288
I20260709 12:24:00.809651  6661 raft_consensus.cc:606] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Received request to transfer leadership to TS 918c6d087dde45a9bfde0b9f18936f2c
I20260709 12:24:00.811070  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:00.811897  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:00.860013  6953 raft_consensus.cc:993] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:00.860772  6917 raft_consensus.cc:1081] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:00.862933  6586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44966
I20260709 12:24:00.863693  6586 raft_consensus.cc:493] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:00.864074  6586 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.869938  6953 raft_consensus.cc:993] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:00.870632  6917 raft_consensus.cc:1081] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:00.872123  6586 raft_consensus.cc:515] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.873036  6587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f218385ed61b4334becc562630e707e2"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44966
I20260709 12:24:00.873929  6587 raft_consensus.cc:493] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:00.874331  6587 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.875564  6586 leader_election.cc:290] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 4 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:24:00.877833  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:24:00.877966  6735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20067e4eb864af3aabc75928bc82133" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:24:00.878983  6735 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.879613  6661 raft_consensus.cc:3055] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:00.880018  6661 raft_consensus.cc:740] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:24:00.881263  6661 consensus_queue.cc:260] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.883162  6661 raft_consensus.cc:3060] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.884035  6587 raft_consensus.cc:515] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.886008  6587 leader_election.cc:290] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 4 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:24:00.887458  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:24:00.888111  6734 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.889082  6735 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 4.
I20260709 12:24:00.890802  6552 leader_election.cc:304] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:24:00.891396  6661 raft_consensus.cc:2468] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 4.
I20260709 12:24:00.892530  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f218385ed61b4334becc562630e707e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:24:00.893249  6662 raft_consensus.cc:3055] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:00.893599  6662 raft_consensus.cc:740] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:24:00.894886  6662 consensus_queue.cc:260] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.896485  6662 raft_consensus.cc:3060] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.904004  6734 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 4.
I20260709 12:24:00.905224  6953 raft_consensus.cc:993] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: : Instructing follower 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:24:00.906167  6917 raft_consensus.cc:1081] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Signalling peer 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:24:00.909627  6955 raft_consensus.cc:2804] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:00.910125  6552 leader_election.cc:304] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:24:00.910799  6662 raft_consensus.cc:2468] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 4.
I20260709 12:24:00.915900  6958 raft_consensus.cc:2804] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:00.916960  6958 raft_consensus.cc:697] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:24:00.918341  6955 raft_consensus.cc:697] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:24:00.919534  6734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:39754
I20260709 12:24:00.919725  6958 consensus_queue.cc:237] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.920390  6734 raft_consensus.cc:493] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:00.920867  6734 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.920707  6955 consensus_queue.cc:237] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.923583  6917 raft_consensus.cc:993] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: : Instructing follower 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:24:00.924292  6953 raft_consensus.cc:1081] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Signalling peer 4f9a79c23b5e41e9aa4ce456a79560e3 to start an election
I20260709 12:24:00.926380  6735 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab"
dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
 from {username='slave'} at 127.0.0.1:39754
I20260709 12:24:00.927052  6735 raft_consensus.cc:493] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:00.927472  6735 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.929612  6734 raft_consensus.cc:515] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.934764  6587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:24:00.936653  6587 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.937491  6735 raft_consensus.cc:515] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.941789  6735 leader_election.cc:290] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 4 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:24:00.941777  6586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
I20260709 12:24:00.943081  6586 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.943547  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed747d61cc9e4bfeae5c25705d917fab" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:24:00.944231  6662 raft_consensus.cc:3055] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:00.944598  6662 raft_consensus.cc:740] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:24:00.945564  6662 consensus_queue.cc:260] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.946812  6662 raft_consensus.cc:3060] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.947039  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dbe0fcace364cc78073bb8a812cea6f" candidate_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:24:00.947687  6661 raft_consensus.cc:3055] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:00.947978  6661 raft_consensus.cc:740] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:24:00.948731  6661 consensus_queue.cc:260] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.949872  6661 raft_consensus.cc:3060] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.957053  6662 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 4.
I20260709 12:24:00.958786  6702 leader_election.cc:304] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, e613440a882e4b6c9149f57ebac09338; no voters: 
I20260709 12:24:00.960073  6927 raft_consensus.cc:2804] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:00.959965  6734 leader_election.cc:290] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 3 election: Requested vote from peers 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721), e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553)
I20260709 12:24:00.962824  6586 raft_consensus.cc:2468] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 4.
I20260709 12:24:00.968981  6927 raft_consensus.cc:697] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:24:00.968981  6587 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 3.
I20260709 12:24:00.970602  6927 consensus_queue.cc:237] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.971709  6701 leader_election.cc:304] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:24:00.972929  6961 raft_consensus.cc:2804] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:00.969228  6661 raft_consensus.cc:2468] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9a79c23b5e41e9aa4ce456a79560e3 in term 3.
I20260709 12:24:00.980520  6961 raft_consensus.cc:697] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Becoming Leader. State: Replica: 4f9a79c23b5e41e9aa4ce456a79560e3, State: Running, Role: LEADER
I20260709 12:24:00.981899  6961 consensus_queue.cc:237] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:00.986665  6482 catalog_manager.cc:5697] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 3 to 4, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 4 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.986801  6481 catalog_manager.cc:5697] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 3 to 4, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 4 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:01.011632  6953 raft_consensus.cc:993] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: : Instructing follower 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:01.012271  6917 raft_consensus.cc:1081] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Signalling peer 918c6d087dde45a9bfde0b9f18936f2c to start an election
I20260709 12:24:01.013355  6481 catalog_manager.cc:5697] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 3 to 4, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 4 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:01.014362  6587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2"
dest_uuid: "918c6d087dde45a9bfde0b9f18936f2c"
 from {username='slave'} at 127.0.0.1:44966
I20260709 12:24:01.013757  6482 catalog_manager.cc:5697] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 reported cstate change: term changed from 2 to 3, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:01.015007  6587 raft_consensus.cc:493] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:01.015491  6587 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.022392  6587 raft_consensus.cc:515] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:01.024982  6587 leader_election.cc:290] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 3 election: Requested vote from peers e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553), 4f9a79c23b5e41e9aa4ce456a79560e3 (127.6.76.195:38173)
I20260709 12:24:01.026219  6734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3"
I20260709 12:24:01.026237  6661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ccc56afe934355bd89d66dd67d43e2" candidate_uuid: "918c6d087dde45a9bfde0b9f18936f2c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e613440a882e4b6c9149f57ebac09338"
I20260709 12:24:01.026998  6734 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.027014  6661 raft_consensus.cc:3055] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:01.027657  6661 raft_consensus.cc:740] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e613440a882e4b6c9149f57ebac09338, State: Running, Role: LEADER
I20260709 12:24:01.028383  6661 consensus_queue.cc:260] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:01.029628  6661 raft_consensus.cc:3060] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.032950  6734 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 3.
I20260709 12:24:01.034255  6552 leader_election.cc:304] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9a79c23b5e41e9aa4ce456a79560e3, 918c6d087dde45a9bfde0b9f18936f2c; no voters: 
I20260709 12:24:01.035395  6955 raft_consensus.cc:2804] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:01.035493  6661 raft_consensus.cc:2468] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918c6d087dde45a9bfde0b9f18936f2c in term 3.
I20260709 12:24:01.036227  6955 raft_consensus.cc:697] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 LEADER]: Becoming Leader. State: Replica: 918c6d087dde45a9bfde0b9f18936f2c, State: Running, Role: LEADER
I20260709 12:24:01.037303  6955 consensus_queue.cc:237] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } }
I20260709 12:24:01.047814  6482 catalog_manager.cc:5697] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c reported cstate change: term changed from 2 to 3, leader changed from e613440a882e4b6c9149f57ebac09338 (127.6.76.194) to 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193). New cstate: current_term: 3 leader_uuid: "918c6d087dde45a9bfde0b9f18936f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:01.380699  6734 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.382917  6955 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.400393  6661 raft_consensus.cc:1275] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.403281  6955 consensus_queue.cc:1048] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.426538  6587 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.428052  6961 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:01.433714  6661 raft_consensus.cc:1275] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:01.436368  6955 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:01.451054  6663 raft_consensus.cc:1275] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.452351  6586 raft_consensus.cc:1275] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:01.452955  6661 raft_consensus.cc:1275] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Refusing update from remote peer 4f9a79c23b5e41e9aa4ce456a79560e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:01.452800  6961 consensus_queue.cc:1048] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:01.455883  6961 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:01.458115  6962 consensus_queue.cc:1048] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918c6d087dde45a9bfde0b9f18936f2c" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:01.465276  6734 raft_consensus.cc:1275] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:01.467723  6955 consensus_queue.cc:1048] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:01.492967  6660 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.496201  6736 raft_consensus.cc:1275] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Refusing update from remote peer 918c6d087dde45a9bfde0b9f18936f2c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:01.499017  6957 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9a79c23b5e41e9aa4ce456a79560e3" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.504458  6957 consensus_queue.cc:1048] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e613440a882e4b6c9149f57ebac09338" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:01.813773  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:01.819263  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:01.819818  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:02.821200  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:02.827301  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:02.827814  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:03.828992  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:03.835714  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:03.836633  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:04.838828  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:04.844983  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:04.845554  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:05.846920  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:05.853930  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:05.854367  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:06.949124  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.959579  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:06.960136  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:07.770809  6608 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 6454
W20260709 12:24:07.771790  6608 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 6517
W20260709 12:24:07.772882  6608 debug-util.cc:398] Leaking SignalData structure 0x7b080014ebc0 after lost signal to thread 6611
W20260709 12:24:07.773774  6608 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0460 after lost signal to thread 6687
W20260709 12:24:07.774549  6608 debug-util.cc:398] Leaking SignalData structure 0x7b0800114a60 after lost signal to thread 6761
I20260709 12:24:07.961426  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:07.967172  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:07.967890  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:08.969049  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:08.974424  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:08.974934  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.977066  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.985810  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:09.986588  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:11.040901  6451 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:24:11.041750  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
W20260709 12:24:11.046027  6626 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:24:11.055737  6626 consensus_peers.cc:597] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 -> Peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Couldn't send request to peer 918c6d087dde45a9bfde0b9f18936f2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.093866  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:11.094794  6451 tablet_replica.cc:333] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.095794  6451 raft_consensus.cc:2243] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.096921  6451 raft_consensus.cc:2272] T 703b9373822148949130c2160adcb8be P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.099781  6451 tablet_replica.cc:333] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.100477  6451 raft_consensus.cc:2243] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.101104  6451 raft_consensus.cc:2272] T cb239e3ddb3942398c4ce34c95a0674b P 918c6d087dde45a9bfde0b9f18936f2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.103858  6451 tablet_replica.cc:333] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.104875  6451 raft_consensus.cc:2243] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.105535  6451 raft_consensus.cc:2272] T 5dbe0fcace364cc78073bb8a812cea6f P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:11.106603  6701 consensus_peers.cc:597] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 -> Peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Couldn't send request to peer 918c6d087dde45a9bfde0b9f18936f2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.108063  6451 tablet_replica.cc:333] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.108660  6451 raft_consensus.cc:2243] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.109539  6451 raft_consensus.cc:2272] T 89ccc56afe934355bd89d66dd67d43e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.111617  6451 tablet_replica.cc:333] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.112162  6451 raft_consensus.cc:2243] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.112984  6451 raft_consensus.cc:2272] T b20067e4eb864af3aabc75928bc82133 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.114996  6451 tablet_replica.cc:333] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.115635  6451 raft_consensus.cc:2243] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.116273  6451 raft_consensus.cc:2272] T 6a399992a1a84bdfa163873fe1dc52a5 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.118151  6451 tablet_replica.cc:333] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.118793  6451 raft_consensus.cc:2243] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.119406  6451 raft_consensus.cc:2272] T ed747d61cc9e4bfeae5c25705d917fab P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.121240  6451 tablet_replica.cc:333] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.121877  6451 raft_consensus.cc:2243] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.122896  6451 raft_consensus.cc:2272] T f218385ed61b4334becc562630e707e2 P 918c6d087dde45a9bfde0b9f18936f2c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.125124  6451 tablet_replica.cc:333] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c: stopping tablet replica
I20260709 12:24:11.125687  6451 raft_consensus.cc:2243] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.126199  6451 raft_consensus.cc:2272] T cce9a98aa6ab41b8a92a778e96738f54 P 918c6d087dde45a9bfde0b9f18936f2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.250974  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
I20260709 12:24:11.273219  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
I20260709 12:24:11.314586  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:11.315721  6451 tablet_replica.cc:333] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.316686  6451 raft_consensus.cc:2243] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.317410  6451 raft_consensus.cc:2272] T cb239e3ddb3942398c4ce34c95a0674b P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.319914  6451 tablet_replica.cc:333] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.320570  6451 raft_consensus.cc:2243] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.321223  6451 raft_consensus.cc:2272] T 89ccc56afe934355bd89d66dd67d43e2 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.323593  6451 tablet_replica.cc:333] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.324234  6451 raft_consensus.cc:2243] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.324800  6451 raft_consensus.cc:2272] T 5dbe0fcace364cc78073bb8a812cea6f P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.326933  6451 tablet_replica.cc:333] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.327692  6451 raft_consensus.cc:2243] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.328694  6451 raft_consensus.cc:2272] T 703b9373822148949130c2160adcb8be P e613440a882e4b6c9149f57ebac09338 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.331341  6451 tablet_replica.cc:333] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.331995  6451 raft_consensus.cc:2243] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.332556  6451 raft_consensus.cc:2272] T ed747d61cc9e4bfeae5c25705d917fab P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.334937  6451 tablet_replica.cc:333] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.335698  6451 raft_consensus.cc:2243] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.336848  6451 raft_consensus.cc:2272] T 6a399992a1a84bdfa163873fe1dc52a5 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.339017  6451 tablet_replica.cc:333] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.339663  6451 raft_consensus.cc:2243] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.340222  6451 raft_consensus.cc:2272] T b20067e4eb864af3aabc75928bc82133 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.342510  6451 tablet_replica.cc:333] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.343147  6451 raft_consensus.cc:2243] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.343775  6451 raft_consensus.cc:2272] T f218385ed61b4334becc562630e707e2 P e613440a882e4b6c9149f57ebac09338 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.345788  6451 tablet_replica.cc:333] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338: stopping tablet replica
I20260709 12:24:11.346422  6451 raft_consensus.cc:2243] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.347474  6451 raft_consensus.cc:2272] T cce9a98aa6ab41b8a92a778e96738f54 P e613440a882e4b6c9149f57ebac09338 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:11.386654  6702 consensus_peers.cc:597] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 -> Peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Couldn't send request to peer e613440a882e4b6c9149f57ebac09338. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:39553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.420938  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
I20260709 12:24:11.445122  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
W20260709 12:24:11.450464  6701 consensus_peers.cc:597] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 -> Peer 918c6d087dde45a9bfde0b9f18936f2c (127.6.76.193:40721): Couldn't send request to peer 918c6d087dde45a9bfde0b9f18936f2c. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:40721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:11.478775  6702 consensus_peers.cc:597] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 -> Peer e613440a882e4b6c9149f57ebac09338 (127.6.76.194:39553): Couldn't send request to peer e613440a882e4b6c9149f57ebac09338. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:39553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.490674  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:11.491616  6451 tablet_replica.cc:333] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.492473  6451 raft_consensus.cc:2243] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.493315  6451 raft_consensus.cc:2272] T 703b9373822148949130c2160adcb8be P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.495712  6451 tablet_replica.cc:333] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.496388  6451 raft_consensus.cc:2243] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.496961  6451 raft_consensus.cc:2272] T 89ccc56afe934355bd89d66dd67d43e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.498931  6451 tablet_replica.cc:333] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.499636  6451 raft_consensus.cc:2243] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.500555  6451 raft_consensus.cc:2272] T 5dbe0fcace364cc78073bb8a812cea6f P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.502493  6451 tablet_replica.cc:333] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.503058  6451 raft_consensus.cc:2243] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.503969  6451 raft_consensus.cc:2272] T cb239e3ddb3942398c4ce34c95a0674b P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.505878  6451 tablet_replica.cc:333] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.506470  6451 raft_consensus.cc:2243] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.507032  6451 raft_consensus.cc:2272] T b20067e4eb864af3aabc75928bc82133 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.509598  6451 tablet_replica.cc:333] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.510226  6451 raft_consensus.cc:2243] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.510794  6451 raft_consensus.cc:2272] T f218385ed61b4334becc562630e707e2 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.512849  6451 tablet_replica.cc:333] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.513872  6451 raft_consensus.cc:2243] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.514700  6451 raft_consensus.cc:2272] T 6a399992a1a84bdfa163873fe1dc52a5 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.516981  6451 tablet_replica.cc:333] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.517668  6451 raft_consensus.cc:2243] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.518838  6451 raft_consensus.cc:2272] T ed747d61cc9e4bfeae5c25705d917fab P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.521260  6451 tablet_replica.cc:333] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3: stopping tablet replica
I20260709 12:24:11.521924  6451 raft_consensus.cc:2243] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:11.522514  6451 raft_consensus.cc:2272] T cce9a98aa6ab41b8a92a778e96738f54 P 4f9a79c23b5e41e9aa4ce456a79560e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.574699  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:24:11.598125  6451 master.cc:562] Master@127.6.76.254:44785 shutting down...
I20260709 12:24:11.782886  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:11.784096  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:11.784619  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0972b48461b74aefb3fd784a8b5872d8: stopping tablet replica
I20260709 12:24:11.808375  6451 master.cc:584] Master@127.6.76.254:44785 shutdown complete.
4f9a79c23b5e41e9aa4ce456a79560e3  1
918c6d087dde45a9bfde0b9f18936f2c  4
e613440a882e4b6c9149f57ebac09338  4
4f9a79c23b5e41e9aa4ce456a79560e3  3
918c6d087dde45a9bfde0b9f18936f2c  3
e613440a882e4b6c9149f57ebac09338  3
4f9a79c23b5e41e9aa4ce456a79560e3  1
918c6d087dde45a9bfde0b9f18936f2c  0
e613440a882e4b6c9149f57ebac09338  8
4f9a79c23b5e41e9aa4ce456a79560e3  3
918c6d087dde45a9bfde0b9f18936f2c  3
e613440a882e4b6c9149f57ebac09338  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (32652 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:24:11.866247  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:39151
I20260709 12:24:11.867913  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:11.874367  7001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:11.874436  7002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:11.875566  7004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:11.878621  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:11.879988  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:11.880234  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:11.880519  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599851880479 us; error 0 us; skew 500 ppm
I20260709 12:24:11.881214  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:11.884490  6451 webserver.cc:533] Webserver started at http://127.6.76.254:39443/ using document root <none> and password file <none>
I20260709 12:24:11.885013  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:11.885188  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:11.885428  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:11.886667  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "dc0cca05c1cd4890a3a33c990befc109"
format_stamp: "Formatted at 2026-07-09 12:24:11 on dist-test-slave-n447"
I20260709 12:24:11.891795  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:24:11.896299  7009 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:11.897235  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.003s
I20260709 12:24:11.897607  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "dc0cca05c1cd4890a3a33c990befc109"
format_stamp: "Formatted at 2026-07-09 12:24:11 on dist-test-slave-n447"
I20260709 12:24:11.897943  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:11.922377  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:11.924165  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:11.980346  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:39151
I20260709 12:24:11.980487  7060 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:39151 every 8 connection(s)
I20260709 12:24:11.985508  7061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:11.996529  7061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109: Bootstrap starting.
I20260709 12:24:12.001906  7061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:12.007179  7061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109: No bootstrap required, opened a new log
I20260709 12:24:12.009829  7061 raft_consensus.cc:359] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER }
I20260709 12:24:12.010351  7061 raft_consensus.cc:385] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:12.010654  7061 raft_consensus.cc:740] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0cca05c1cd4890a3a33c990befc109, State: Initialized, Role: FOLLOWER
I20260709 12:24:12.011438  7061 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER }
I20260709 12:24:12.011917  7061 raft_consensus.cc:399] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:24:12.012223  7061 raft_consensus.cc:493] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:24:12.012645  7061 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:12.017933  7061 raft_consensus.cc:515] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER }
I20260709 12:24:12.018775  7061 leader_election.cc:304] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc0cca05c1cd4890a3a33c990befc109; no voters: 
I20260709 12:24:12.020329  7061 leader_election.cc:290] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:24:12.020815  7064 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:12.022363  7064 raft_consensus.cc:697] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 1 LEADER]: Becoming Leader. State: Replica: dc0cca05c1cd4890a3a33c990befc109, State: Running, Role: LEADER
I20260709 12:24:12.023290  7064 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER }
I20260709 12:24:12.024528  7061 sys_catalog.cc:565] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:24:12.030884  7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc0cca05c1cd4890a3a33c990befc109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER } }
I20260709 12:24:12.031334  7066 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0cca05c1cd4890a3a33c990befc109. Latest consensus state: current_term: 1 leader_uuid: "dc0cca05c1cd4890a3a33c990befc109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0cca05c1cd4890a3a33c990befc109" member_type: VOTER } }
I20260709 12:24:12.031665  7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:12.032198  7066 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:12.036430  7072 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:24:12.044600  7072 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:24:12.056309  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:24:12.062778  7072 catalog_manager.cc:1383] Generated new cluster ID: 8ef6b787a7c94d24936192d79019adf7
I20260709 12:24:12.063375  7072 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:24:12.092199  7072 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:24:12.093928  7072 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:24:12.110298  7072 catalog_manager.cc:6070] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109: Generated new TSK 0
I20260709 12:24:12.111006  7072 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:24:12.132093  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:12.140713  7082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:12.141423  7083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:12.146263  6451 server_base.cc:1061] running on GCE node
W20260709 12:24:12.146302  7085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:12.147858  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:12.148106  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:12.148326  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599852148304 us; error 0 us; skew 500 ppm
I20260709 12:24:12.148999  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:12.151971  6451 webserver.cc:533] Webserver started at http://127.6.76.193:36131/ using document root <none> and password file <none>
I20260709 12:24:12.152599  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:12.152796  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:12.153177  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:12.154788  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "4441f407c0394f348866d5436b855a76"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.160939  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:24:12.165935  7090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.166987  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:24:12.167425  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "4441f407c0394f348866d5436b855a76"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.167786  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:12.184448  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:12.185828  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:12.187542  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:12.190215  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:12.190503  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.190810  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:12.191018  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.246670  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:33139
I20260709 12:24:12.246789  7152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:33139 every 8 connection(s)
I20260709 12:24:12.252292  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:12.260504  7157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:12.263922  7158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:12.266422  7160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:12.267973  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:12.269577  7153 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39151
I20260709 12:24:12.269899  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:12.270006  7153 heartbeater.cc:461] Registering TS with master...
W20260709 12:24:12.270217  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:12.270473  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599852270455 us; error 0 us; skew 500 ppm
I20260709 12:24:12.271036  7153 heartbeater.cc:507] Master 127.6.76.254:39151 requested a full tablet report, sending...
I20260709 12:24:12.271154  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:12.274463  7026 ts_manager.cc:194] Registered new tserver with Master: 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139)
I20260709 12:24:12.275079  6451 webserver.cc:533] Webserver started at http://127.6.76.194:36137/ using document root <none> and password file <none>
I20260709 12:24:12.275907  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:12.276206  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:12.276551  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:12.277088  7026 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60588
I20260709 12:24:12.278228  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "94cc4d2d82804b39bf733d2e70599c6b"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.284777  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 12:24:12.289552  7165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.290647  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:24:12.291082  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "94cc4d2d82804b39bf733d2e70599c6b"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.291498  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:12.319640  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:12.321097  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:12.323181  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:12.326653  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:12.327095  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.327441  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:12.327663  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.389918  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:34739
I20260709 12:24:12.390038  7227 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:34739 every 8 connection(s)
I20260709 12:24:12.395154  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:12.403997  7231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:12.405609  7232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:12.409894  6451 server_base.cc:1061] running on GCE node
W20260709 12:24:12.410820  7234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:12.411819  7228 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39151
I20260709 12:24:12.412292  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:12.412330  7228 heartbeater.cc:461] Registering TS with master...
W20260709 12:24:12.412688  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:12.412945  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599852412922 us; error 0 us; skew 500 ppm
I20260709 12:24:12.413627  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:12.413834  7228 heartbeater.cc:507] Master 127.6.76.254:39151 requested a full tablet report, sending...
I20260709 12:24:12.417246  6451 webserver.cc:533] Webserver started at http://127.6.76.195:34103/ using document root <none> and password file <none>
I20260709 12:24:12.417589  7026 ts_manager.cc:194] Registered new tserver with Master: 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:12.418318  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:12.418665  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:12.419047  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:12.419869  7026 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60596
I20260709 12:24:12.420742  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "66b8b104e71f4eb2b126c418c93eb29e"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.426751  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:24:12.431012  7239 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.432173  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:12.432480  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "66b8b104e71f4eb2b126c418c93eb29e"
format_stamp: "Formatted at 2026-07-09 12:24:12 on dist-test-slave-n447"
I20260709 12:24:12.432758  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:12.465529  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:12.466832  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:12.468420  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:12.471185  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:12.471488  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 12:24:12.471820  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:12.471975  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:12.522871  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:33427
I20260709 12:24:12.523006  7302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:33427 every 8 connection(s)
I20260709 12:24:12.545905  7303 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39151
I20260709 12:24:12.546412  7303 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:12.547363  7303 heartbeater.cc:507] Master 127.6.76.254:39151 requested a full tablet report, sending...
I20260709 12:24:12.549732  7026 ts_manager.cc:194] Registered new tserver with Master: 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:12.550060  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023203253s
I20260709 12:24:12.551327  7026 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60604
I20260709 12:24:12.581601  7026 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60608:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:24:12.585270  7026 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:24:12.685384  7118 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5a898c0d854d69a0ebd2813b833d79 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:24:12.687285  7118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5a898c0d854d69a0ebd2813b833d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.689615  7192 tablet_service.cc:1511] Processing CreateTablet for tablet f58f58a18e5240edbda87fd3a69cbb87 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:24:12.692865  7193 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5a898c0d854d69a0ebd2813b833d79 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:24:12.695207  7192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58f58a18e5240edbda87fd3a69cbb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.696346  7117 tablet_service.cc:1511] Processing CreateTablet for tablet f58f58a18e5240edbda87fd3a69cbb87 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:24:12.697288  7193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5a898c0d854d69a0ebd2813b833d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.698223  7117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58f58a18e5240edbda87fd3a69cbb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.704725  7191 tablet_service.cc:1511] Processing CreateTablet for tablet e58d799818bc4e16bde5239ceba7a57e (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:24:12.706478  7191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58d799818bc4e16bde5239ceba7a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.708600  7190 tablet_service.cc:1511] Processing CreateTablet for tablet 53cc14a7fdd847fa87addbd10ff9f32b (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:24:12.710227  7190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cc14a7fdd847fa87addbd10ff9f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.712867  7189 tablet_service.cc:1511] Processing CreateTablet for tablet 28b983178bd64ca89b590ae17897947d (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:24:12.715205  7189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b983178bd64ca89b590ae17897947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.731312  7265 tablet_service.cc:1511] Processing CreateTablet for tablet e58d799818bc4e16bde5239ceba7a57e (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:24:12.732918  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58d799818bc4e16bde5239ceba7a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.717219  7187 tablet_service.cc:1511] Processing CreateTablet for tablet ce6bc1447f794ed08bcae619ec50f012 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:24:12.737224  7185 tablet_service.cc:1511] Processing CreateTablet for tablet 702b90997ffe4e2c80d49becf831ca77 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:24:12.739207  7187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6bc1447f794ed08bcae619ec50f012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.717988  7267 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5a898c0d854d69a0ebd2813b833d79 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:24:12.742014  7267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5a898c0d854d69a0ebd2813b833d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.718837  7188 tablet_service.cc:1511] Processing CreateTablet for tablet 91534a4d48b747588dff34ff5cf163a5 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:24:12.743786  7264 tablet_service.cc:1511] Processing CreateTablet for tablet 53cc14a7fdd847fa87addbd10ff9f32b (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:24:12.722807  7116 tablet_service.cc:1511] Processing CreateTablet for tablet e58d799818bc4e16bde5239ceba7a57e (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:24:12.745014  7262 tablet_service.cc:1511] Processing CreateTablet for tablet 91534a4d48b747588dff34ff5cf163a5 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:24:12.745625  7264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cc14a7fdd847fa87addbd10ff9f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.746568  7262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91534a4d48b747588dff34ff5cf163a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.746611  7116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58d799818bc4e16bde5239ceba7a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.724867  7114 tablet_service.cc:1511] Processing CreateTablet for tablet 28b983178bd64ca89b590ae17897947d (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:24:12.748327  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 28b983178bd64ca89b590ae17897947d (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:24:12.726163  7186 tablet_service.cc:1511] Processing CreateTablet for tablet 95527ee5a41a4461a29e72813e1e37e3 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:24:12.750574  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b983178bd64ca89b590ae17897947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.726272  7113 tablet_service.cc:1511] Processing CreateTablet for tablet 91534a4d48b747588dff34ff5cf163a5 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:24:12.727777  7112 tablet_service.cc:1511] Processing CreateTablet for tablet ce6bc1447f794ed08bcae619ec50f012 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:24:12.729249  7115 tablet_service.cc:1511] Processing CreateTablet for tablet 53cc14a7fdd847fa87addbd10ff9f32b (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:24:12.729254  7266 tablet_service.cc:1511] Processing CreateTablet for tablet f58f58a18e5240edbda87fd3a69cbb87 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:24:12.759279  7266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58f58a18e5240edbda87fd3a69cbb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.761659  7188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91534a4d48b747588dff34ff5cf163a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.763770  7186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95527ee5a41a4461a29e72813e1e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.766350  7185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 702b90997ffe4e2c80d49becf831ca77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.781443  7184 tablet_service.cc:1511] Processing CreateTablet for tablet 4c396708f0764f1a82f6ef56fc840a12 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:24:12.782979  7184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c396708f0764f1a82f6ef56fc840a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.789657  7259 tablet_service.cc:1511] Processing CreateTablet for tablet 702b90997ffe4e2c80d49becf831ca77 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:24:12.789806  7260 tablet_service.cc:1511] Processing CreateTablet for tablet 95527ee5a41a4461a29e72813e1e37e3 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:24:12.791525  7259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 702b90997ffe4e2c80d49becf831ca77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.793251  7110 tablet_service.cc:1511] Processing CreateTablet for tablet 702b90997ffe4e2c80d49becf831ca77 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:24:12.793815  7113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91534a4d48b747588dff34ff5cf163a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.794927  7110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 702b90997ffe4e2c80d49becf831ca77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.801316  7111 tablet_service.cc:1511] Processing CreateTablet for tablet 95527ee5a41a4461a29e72813e1e37e3 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:24:12.804183  7261 tablet_service.cc:1511] Processing CreateTablet for tablet ce6bc1447f794ed08bcae619ec50f012 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:24:12.806907  7260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95527ee5a41a4461a29e72813e1e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.815651  7114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b983178bd64ca89b590ae17897947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.823503  7261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6bc1447f794ed08bcae619ec50f012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.833881  7319 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:12.841126  7258 tablet_service.cc:1511] Processing CreateTablet for tablet 4c396708f0764f1a82f6ef56fc840a12 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:24:12.842921  7258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c396708f0764f1a82f6ef56fc840a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.864374  7320 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:12.843349  7319 tablet_bootstrap.cc:654] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:12.844288  7111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95527ee5a41a4461a29e72813e1e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.834671  7109 tablet_service.cc:1511] Processing CreateTablet for tablet 4c396708f0764f1a82f6ef56fc840a12 (DEFAULT_TABLE table=test-workload [id=5cf7cfca49864d27857edd8ece79292f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:24:12.876474  7109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c396708f0764f1a82f6ef56fc840a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.878882  7115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cc14a7fdd847fa87addbd10ff9f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.883821  7112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6bc1447f794ed08bcae619ec50f012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.947481  7320 tablet_bootstrap.cc:654] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:12.982291  7321 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.003746  7320 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.004573  7320 ts_tablet_manager.cc:1403] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.023s
I20260709 12:24:13.008500  7320 raft_consensus.cc:359] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.009622  7320 raft_consensus.cc:385] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.010008  7320 raft_consensus.cc:740] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.016474  7321 tablet_bootstrap.cc:654] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.010862  7320 consensus_queue.cc:260] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.031327  7321 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.031968  7321 ts_tablet_manager.cc:1403] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.004s
I20260709 12:24:13.032824  7319 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.033716  7319 ts_tablet_manager.cc:1403] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.201s	user 0.012s	sys 0.017s
I20260709 12:24:13.035796  7321 raft_consensus.cc:359] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.036743  7321 raft_consensus.cc:385] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.037101  7321 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.037277  7153 heartbeater.cc:499] Master 127.6.76.254:39151 was elected leader, sending a full tablet report...
I20260709 12:24:13.037873  7320 ts_tablet_manager.cc:1434] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.033s	user 0.017s	sys 0.004s
I20260709 12:24:13.039044  7320 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.037636  7319 raft_consensus.cc:359] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.040264  7319 raft_consensus.cc:385] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.038409  7321 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.040688  7319 raft_consensus.cc:740] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
W20260709 12:24:13.041929  7074 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:13.042096  7319 consensus_queue.cc:260] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.050660  7321 ts_tablet_manager.cc:1434] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 12:24:13.051677  7303 heartbeater.cc:499] Master 127.6.76.254:39151 was elected leader, sending a full tablet report...
I20260709 12:24:13.052052  7321 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.053191  7320 tablet_bootstrap.cc:654] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.061705  7321 tablet_bootstrap.cc:654] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.067732  7228 heartbeater.cc:499] Master 127.6.76.254:39151 was elected leader, sending a full tablet report...
I20260709 12:24:13.068158  7319 ts_tablet_manager.cc:1434] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.003s
I20260709 12:24:13.069358  7319 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.072274  7321 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.073195  7321 ts_tablet_manager.cc:1403] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 12:24:13.077303  7320 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.076990  7321 raft_consensus.cc:359] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.077921  7320 ts_tablet_manager.cc:1403] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.007s
I20260709 12:24:13.078174  7321 raft_consensus.cc:385] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.078732  7321 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.079766  7321 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.081543  7320 raft_consensus.cc:359] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.082511  7320 raft_consensus.cc:385] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.082931  7320 raft_consensus.cc:740] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.083613  7321 ts_tablet_manager.cc:1434] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:24:13.084452  7320 consensus_queue.cc:260] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.085631  7321 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.087559  7320 ts_tablet_manager.cc:1434] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 12:24:13.089082  7320 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.089151  7319 tablet_bootstrap.cc:654] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.091439  7326 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.092057  7326 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.097546  7320 tablet_bootstrap.cc:654] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.099359  7321 tablet_bootstrap.cc:654] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.101814  7326 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.125794  7319 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.126456  7319 ts_tablet_manager.cc:1403] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.057s	user 0.023s	sys 0.013s
I20260709 12:24:13.134984  7321 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.135733  7321 ts_tablet_manager.cc:1403] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.015s
I20260709 12:24:13.139062  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.129873  7319 raft_consensus.cc:359] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.141419  7319 raft_consensus.cc:385] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:13.141603  7242 leader_election.cc:343] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.141927  7319 raft_consensus.cc:740] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.142940  7319 consensus_queue.cc:260] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.143474  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.139366  7321 raft_consensus.cc:359] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.144855  7321 raft_consensus.cc:385] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.145304  7321 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
W20260709 12:24:13.145428  7229 tablet.cc:2406] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:24:13.145579  7319 ts_tablet_manager.cc:1434] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
W20260709 12:24:13.145926  7240 leader_election.cc:343] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.146844  7240 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e; no voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.147965  7319 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.148411  7326 raft_consensus.cc:2749] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.149387  7320 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.150032  7320 ts_tablet_manager.cc:1403] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.015s
I20260709 12:24:13.152031  7321 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.153801  7320 raft_consensus.cc:359] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.154779  7320 raft_consensus.cc:385] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.155155  7320 raft_consensus.cc:740] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.156170  7321 ts_tablet_manager.cc:1434] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 12:24:13.156237  7320 consensus_queue.cc:260] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.157986  7321 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.159475  7320 ts_tablet_manager.cc:1434] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 12:24:13.160954  7320 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.163081  7319 tablet_bootstrap.cc:654] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.168488  7321 tablet_bootstrap.cc:654] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.168742  7320 tablet_bootstrap.cc:654] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.182798  7319 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.183666  7319 ts_tablet_manager.cc:1403] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.006s
I20260709 12:24:13.188869  7319 raft_consensus.cc:359] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.190290  7319 raft_consensus.cc:385] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.191052  7319 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.194523  7319 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.196198  7321 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.196916  7321 ts_tablet_manager.cc:1403] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.039s	user 0.028s	sys 0.008s
I20260709 12:24:13.197695  7319 ts_tablet_manager.cc:1434] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:24:13.198962  7319 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.200533  7321 raft_consensus.cc:359] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.201512  7321 raft_consensus.cc:385] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.201896  7321 raft_consensus.cc:740] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.202908  7321 consensus_queue.cc:260] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.208655  7321 ts_tablet_manager.cc:1434] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:24:13.209333  7319 tablet_bootstrap.cc:654] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.210393  7321 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.220269  7321 tablet_bootstrap.cc:654] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.221961  7319 tablet_bootstrap.cc:492] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.222616  7319 ts_tablet_manager.cc:1403] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.006s
I20260709 12:24:13.229856  7325 raft_consensus.cc:493] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.231010  7325 raft_consensus.cc:515] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.238724  7320 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.239471  7320 ts_tablet_manager.cc:1403] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.079s	user 0.047s	sys 0.027s
I20260709 12:24:13.249455  7320 raft_consensus.cc:359] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.250000  7325 leader_election.cc:290] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.250344  7320 raft_consensus.cc:385] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.251549  7320 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.252931  7320 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.257011  7320 ts_tablet_manager.cc:1434] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 12:24:13.260380  7320 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.261224  7328 raft_consensus.cc:493] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.262158  7328 raft_consensus.cc:515] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.269694  7320 tablet_bootstrap.cc:654] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.273213  7328 leader_election.cc:290] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.276203  7326 raft_consensus.cc:493] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.277041  7326 raft_consensus.cc:515] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.282191  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.227203  7319 raft_consensus.cc:359] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.283267  7128 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 0.
I20260709 12:24:13.283569  7319 raft_consensus.cc:385] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.284196  7319 raft_consensus.cc:740] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.285516  7240 leader_election.cc:304] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:13.289842  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.290975  7326 leader_election.cc:290] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.291813  7326 raft_consensus.cc:2804] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.292308  7326 raft_consensus.cc:493] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.292790  7326 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.289824  7319 consensus_queue.cc:260] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.314236  7325 raft_consensus.cc:493] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.316048  7325 raft_consensus.cc:515] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.320128  7203 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 0.
I20260709 12:24:13.321651  7319 ts_tablet_manager.cc:1434] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.099s	user 0.013s	sys 0.000s
I20260709 12:24:13.322453  7321 tablet_bootstrap.cc:492] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.322922  7319 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.323025  7321 ts_tablet_manager.cc:1403] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.113s	user 0.051s	sys 0.055s
W20260709 12:24:13.335114  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:33139, user_credentials={real_user=slave}} blocked reactor thread for 49776.4us
I20260709 12:24:13.334214  7326 raft_consensus.cc:515] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.342268  7321 raft_consensus.cc:359] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.343547  7321 raft_consensus.cc:385] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.344089  7321 raft_consensus.cc:740] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.344197  7202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.345500  7326 leader_election.cc:290] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.345417  7321 consensus_queue.cc:260] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.349112  7321 ts_tablet_manager.cc:1434] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.006s
I20260709 12:24:13.350265  7319 tablet_bootstrap.cc:654] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.319528  7325 leader_election.cc:290] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.352787  7202 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.357272  7321 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.360831  7320 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.361466  7320 ts_tablet_manager.cc:1403] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.101s	user 0.032s	sys 0.060s
I20260709 12:24:13.362270  7328 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.363518  7328 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.367318  7321 tablet_bootstrap.cc:654] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.375330  7328 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.373008  7320 raft_consensus.cc:359] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.376544  7320 raft_consensus.cc:385] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.376943  7320 raft_consensus.cc:740] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.377784  7320 consensus_queue.cc:260] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.379148  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:13.379614  7127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.379880  7128 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.380366  7127 raft_consensus.cc:2468] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.381718  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.383453  7320 ts_tablet_manager.cc:1434] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:24:13.383701  7326 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.384443  7326 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.385063  7320 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.385274  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.385720  7202 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 1.
I20260709 12:24:13.386045  7203 raft_consensus.cc:2468] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.388024  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.388633  7166 leader_election.cc:304] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.389020  7201 raft_consensus.cc:2468] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
W20260709 12:24:13.390117  7166 leader_election.cc:343] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.396147  7242 leader_election.cc:304] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.397370  7346 raft_consensus.cc:2804] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.396881  7093 leader_election.cc:304] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.406481  7320 tablet_bootstrap.cc:654] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.400512  7326 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.409982  7093 leader_election.cc:304] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.410775  7325 raft_consensus.cc:2804] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.411988  7325 raft_consensus.cc:493] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.400238  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.412391  7325 raft_consensus.cc:3060] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:13.419956  7242 leader_election.cc:343] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.424650  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.425222  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
W20260709 12:24:13.429323  7091 leader_election.cc:343] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.429356  7326 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.425901  7278 raft_consensus.cc:2468] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.430552  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.426329  7328 raft_consensus.cc:2804] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.431902  7328 raft_consensus.cc:493] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.432956  7328 raft_consensus.cc:3060] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.431855  7326 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.440438  7326 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.444330  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
W20260709 12:24:13.446382  7242 leader_election.cc:343] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:13.443322  7240 leader_election.cc:343] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.461323  7240 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e; no voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b
W20260709 12:24:13.453336  7093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:34739, user_credentials={real_user=slave}} blocked reactor thread for 43488us
I20260709 12:24:13.458783  7128 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 1.
I20260709 12:24:13.466250  7346 raft_consensus.cc:697] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:13.467156  7326 raft_consensus.cc:2749] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.474773  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.475461  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.475636  7352 raft_consensus.cc:2804] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.476204  7278 raft_consensus.cc:2468] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.476282  7352 raft_consensus.cc:493] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.475462  7319 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.477798  7319 ts_tablet_manager.cc:1403] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.155s	user 0.017s	sys 0.017s
I20260709 12:24:13.476910  7346 consensus_queue.cc:237] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.478044  7325 raft_consensus.cc:515] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.479632  7328 raft_consensus.cc:515] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.480527  7325 leader_election.cc:290] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.480994  7320 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.481528  7320 ts_tablet_manager.cc:1403] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.097s	user 0.021s	sys 0.028s
I20260709 12:24:13.481786  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.481698  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.482494  7321 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.482437  7277 raft_consensus.cc:3060] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.477012  7352 raft_consensus.cc:3060] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.483150  7321 ts_tablet_manager.cc:1403] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.126s	user 0.030s	sys 0.037s
I20260709 12:24:13.484545  7201 raft_consensus.cc:2393] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4441f407c0394f348866d5436b855a76 in current term 1: Already voted for candidate 94cc4d2d82804b39bf733d2e70599c6b in this term.
I20260709 12:24:13.484874  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.485788  7278 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.489646  7320 raft_consensus.cc:359] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.490837  7320 raft_consensus.cc:385] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.491300  7320 raft_consensus.cc:740] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.491771  7321 raft_consensus.cc:359] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.492650  7321 raft_consensus.cc:385] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.492412  7320 consensus_queue.cc:260] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.491804  7319 raft_consensus.cc:359] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.493384  7321 raft_consensus.cc:740] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.494907  7166 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 4441f407c0394f348866d5436b855a76
I20260709 12:24:13.493629  7319 raft_consensus.cc:385] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.496778  7319 raft_consensus.cc:740] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.495769  7328 leader_election.cc:290] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.495694  7320 ts_tablet_manager.cc:1434] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:24:13.497831  7319 consensus_queue.cc:260] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.506229  7319 ts_tablet_manager.cc:1434] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.003s
I20260709 12:24:13.497766  7328 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.506824  7328 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.507334  7328 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.508524  7277 raft_consensus.cc:2468] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:13.496886  7321 consensus_queue.cc:260] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
W20260709 12:24:13.501250  7240 leader_election.cc:343] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.503968  7325 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.510385  7325 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.502846  7352 raft_consensus.cc:515] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.511420  7321 ts_tablet_manager.cc:1434] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.005s
I20260709 12:24:13.499498  7320 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.516510  7240 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e; no voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.516034  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.518359  7091 leader_election.cc:304] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.520341  7325 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
W20260709 12:24:13.521117  7093 leader_election.cc:343] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.521250  7325 raft_consensus.cc:2804] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.526443  7325 raft_consensus.cc:697] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:13.527789  7325 consensus_queue.cc:237] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.543365  7321 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.555958  7319 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.528909  7346 raft_consensus.cc:2749] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.568511  7352 leader_election.cc:290] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.570207  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.570991  7201 raft_consensus.cc:3060] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:13.566962  7166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:33427, user_credentials={real_user=slave}} blocked reactor thread for 72128.6us
I20260709 12:24:13.574159  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:13.578065  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5a898c0d854d69a0ebd2813b833d79" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.579056  7277 raft_consensus.cc:2393] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94cc4d2d82804b39bf733d2e70599c6b in current term 1: Already voted for candidate 4441f407c0394f348866d5436b855a76 in this term.
I20260709 12:24:13.580755  7166 leader_election.cc:304] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94cc4d2d82804b39bf733d2e70599c6b; no voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:13.584643  7319 tablet_bootstrap.cc:654] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.581890  7354 raft_consensus.cc:2749] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.582342  7321 tablet_bootstrap.cc:654] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.581025  7025 catalog_manager.cc:5697] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 0 to 1, leader changed from <none> to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 1 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.592216  7320 tablet_bootstrap.cc:654] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:13.584321  7091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:33427, user_credentials={real_user=slave}} blocked reactor thread for 66594.7us
I20260709 12:24:13.596774  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.583822  7328 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.605602  7328 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.607506  7321 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.608789  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
W20260709 12:24:13.610594  7166 leader_election.cc:343] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.609596  7023 catalog_manager.cc:5697] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.611016  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.611821  7277 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.614440  7320 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.615425  7320 ts_tablet_manager.cc:1403] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.116s	user 0.019s	sys 0.000s
I20260709 12:24:13.616318  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.616616  7201 raft_consensus.cc:2468] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:13.609618  7321 ts_tablet_manager.cc:1403] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.092s	user 0.022s	sys 0.000s
I20260709 12:24:13.618479  7093 leader_election.cc:304] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
W20260709 12:24:13.618433  7091 leader_election.cc:343] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.619953  7325 raft_consensus.cc:2804] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.620262  7091 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4441f407c0394f348866d5436b855a76; no voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.620620  7325 raft_consensus.cc:697] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
W20260709 12:24:13.621862  7091 leader_election.cc:343] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.621863  7325 consensus_queue.cc:237] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.623150  7356 raft_consensus.cc:2749] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.623296  7321 raft_consensus.cc:359] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.624738  7321 raft_consensus.cc:385] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.625211  7321 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.626205  7321 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.630784  7277 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 1.
I20260709 12:24:13.629616  7319 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.630668  7320 raft_consensus.cc:359] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.632364  7319 ts_tablet_manager.cc:1403] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.013s
I20260709 12:24:13.633451  7321 ts_tablet_manager.cc:1434] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:24:13.633661  7166 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 4441f407c0394f348866d5436b855a76
I20260709 12:24:13.632288  7320 raft_consensus.cc:385] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.635210  7328 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.635803  7320 raft_consensus.cc:740] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.636313  7328 raft_consensus.cc:697] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:13.636942  7319 raft_consensus.cc:359] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.637305  7328 consensus_queue.cc:237] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.634925  7321 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.637915  7320 consensus_queue.cc:260] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.643020  7320 ts_tablet_manager.cc:1434] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 12:24:13.644605  7320 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.650084  7321 tablet_bootstrap.cc:654] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.637758  7319 raft_consensus.cc:385] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.654580  7319 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.656255  7320 tablet_bootstrap.cc:654] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.656323  7319 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.658890  7025 catalog_manager.cc:5697] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.662444  7319 ts_tablet_manager.cc:1434] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 12:24:13.663722  7319 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.668493  7320 tablet_bootstrap.cc:492] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.669116  7320 ts_tablet_manager.cc:1403] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.001s
I20260709 12:24:13.671923  7321 tablet_bootstrap.cc:492] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.672676  7321 ts_tablet_manager.cc:1403] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.002s
I20260709 12:24:13.672859  7320 raft_consensus.cc:359] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.673719  7320 raft_consensus.cc:385] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.674039  7320 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.674652  7319 tablet_bootstrap.cc:654] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.674906  7320 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.676584  7321 raft_consensus.cc:359] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.677575  7321 raft_consensus.cc:385] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.677577  7320 ts_tablet_manager.cc:1434] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:13.676183  7023 catalog_manager.cc:5697] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 1 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.678128  7321 raft_consensus.cc:740] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.679452  7320 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.679406  7321 consensus_queue.cc:260] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.682265  7321 ts_tablet_manager.cc:1434] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:13.683516  7321 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.684397  7326 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.685029  7326 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.685448  7346 raft_consensus.cc:493] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.685961  7346 raft_consensus.cc:515] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.688544  7320 tablet_bootstrap.cc:654] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.689225  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.690006  7321 tablet_bootstrap.cc:654] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.689225  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.691913  7128 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 0.
I20260709 12:24:13.693248  7326 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
W20260709 12:24:13.693967  7242 leader_election.cc:343] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.697264  7346 leader_election.cc:290] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.701112  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.702113  7201 raft_consensus.cc:2393] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66b8b104e71f4eb2b126c418c93eb29e in current term 1: Already voted for candidate 4441f407c0394f348866d5436b855a76 in this term.
I20260709 12:24:13.705411  7240 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.707286  7320 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.709724  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cc14a7fdd847fa87addbd10ff9f32b" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.713227  7328 raft_consensus.cc:493] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.713912  7328 raft_consensus.cc:515] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.707547  7346 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.715114  7346 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.715536  7346 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.715693  7240 leader_election.cc:304] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e; no voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:13.716655  7326 raft_consensus.cc:3060] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.712374  7320 ts_tablet_manager.cc:1403] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.004s
I20260709 12:24:13.718170  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.718926  7128 raft_consensus.cc:2468] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.720424  7166 leader_election.cc:304] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.720942  7320 raft_consensus.cc:359] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.721452  7354 raft_consensus.cc:2804] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.721752  7320 raft_consensus.cc:385] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.721916  7354 raft_consensus.cc:493] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.722146  7320 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.722615  7354 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.723547  7320 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.726861  7319 tablet_bootstrap.cc:492] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.727690  7319 ts_tablet_manager.cc:1403] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.008s
I20260709 12:24:13.731805  7328 leader_election.cc:290] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.731884  7320 ts_tablet_manager.cc:1434] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:24:13.731534  7319 raft_consensus.cc:359] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.733016  7319 raft_consensus.cc:385] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.733461  7319 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.734187  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.734727  7319 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.735013  7346 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
W20260709 12:24:13.736019  7166 leader_election.cc:343] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.737622  7319 ts_tablet_manager.cc:1434] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:24:13.738895  7319 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.733480  7320 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: Bootstrap starting.
I20260709 12:24:13.740612  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:13.741552  7128 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.742158  7354 raft_consensus.cc:515] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.743570  7326 raft_consensus.cc:2749] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:13.750757  7319 tablet_bootstrap.cc:654] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.758160  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.756108  7320 tablet_bootstrap.cc:654] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.755777  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:13.759548  7126 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:13.759970  7166 leader_election.cc:343] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:13.767302  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.768198  7201 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.769613  7354 leader_election.cc:290] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.772600  7128 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 1.
I20260709 12:24:13.774304  7240 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:13.768270  7346 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.776142  7346 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.777209  7346 raft_consensus.cc:697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:13.777817  7201 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 1.
I20260709 12:24:13.778312  7346 consensus_queue.cc:237] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.781380  7321 tablet_bootstrap.cc:492] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.781983  7321 ts_tablet_manager.cc:1403] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.029s
I20260709 12:24:13.781018  7126 raft_consensus.cc:2468] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 1.
I20260709 12:24:13.787693  7321 raft_consensus.cc:359] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.788779  7321 raft_consensus.cc:385] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.789122  7321 raft_consensus.cc:740] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.790380  7321 consensus_queue.cc:260] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.791571  7166 leader_election.cc:304] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:13.793753  7321 ts_tablet_manager.cc:1434] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.004s
I20260709 12:24:13.794257  7354 raft_consensus.cc:2804] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.795105  7321 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: Bootstrap starting.
I20260709 12:24:13.796531  7354 raft_consensus.cc:697] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:13.797319  7023 catalog_manager.cc:5697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 0 to 1, leader changed from <none> to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 1 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:13.798097  7354 consensus_queue.cc:237] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.802127  7321 tablet_bootstrap.cc:654] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.804767  7319 tablet_bootstrap.cc:492] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.805330  7319 ts_tablet_manager.cc:1403] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.023s
I20260709 12:24:13.808640  7319 raft_consensus.cc:359] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.809379  7319 raft_consensus.cc:385] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.809669  7319 raft_consensus.cc:740] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.809810  7320 tablet_bootstrap.cc:492] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: No bootstrap required, opened a new log
I20260709 12:24:13.809717  7321 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: No bootstrap required, opened a new log
I20260709 12:24:13.810660  7321 ts_tablet_manager.cc:1403] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 12:24:13.810441  7319 consensus_queue.cc:260] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.813295  7319 ts_tablet_manager.cc:1434] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:24:13.814313  7319 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.814742  7321 raft_consensus.cc:359] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.815744  7321 raft_consensus.cc:385] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.816169  7321 raft_consensus.cc:740] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.817162  7321 consensus_queue.cc:260] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.810441  7320 ts_tablet_manager.cc:1403] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.007s
I20260709 12:24:13.820844  7321 ts_tablet_manager.cc:1434] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:13.828174  7319 tablet_bootstrap.cc:654] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.834648  7319 tablet_bootstrap.cc:492] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.834532  7320 raft_consensus.cc:359] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.835472  7319 ts_tablet_manager.cc:1403] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.002s
I20260709 12:24:13.835526  7320 raft_consensus.cc:385] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.835918  7320 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.837028  7320 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.839370  7319 raft_consensus.cc:359] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.842082  7319 raft_consensus.cc:385] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.842569  7319 raft_consensus.cc:740] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.843672  7319 consensus_queue.cc:260] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.846146  7319 ts_tablet_manager.cc:1434] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:24:13.847407  7319 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: Bootstrap starting.
I20260709 12:24:13.851989  7320 ts_tablet_manager.cc:1434] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 12:24:13.851186  7025 catalog_manager.cc:5697] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 1 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.855901  7325 raft_consensus.cc:493] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.856597  7319 tablet_bootstrap.cc:654] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:13.856556  7325 raft_consensus.cc:515] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.860940  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.861999  7277 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.864092  7091 leader_election.cc:304] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:13.866463  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.865782  7325 leader_election.cc:290] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.867174  7201 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.869364  7354 raft_consensus.cc:493] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.870311  7356 raft_consensus.cc:2804] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.869966  7354 raft_consensus.cc:515] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.870764  7356 raft_consensus.cc:493] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.871198  7356 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.873801  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "702b90997ffe4e2c80d49becf831ca77" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.874606  7126 raft_consensus.cc:2468] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.875910  7354 leader_election.cc:290] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.876870  7166 leader_election.cc:304] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.877934  7354 raft_consensus.cc:2804] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.878454  7354 raft_consensus.cc:493] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.878931  7354 raft_consensus.cc:3060] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.873097  7319 tablet_bootstrap.cc:492] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: No bootstrap required, opened a new log
I20260709 12:24:13.878794  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "702b90997ffe4e2c80d49becf831ca77" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.879777  7277 raft_consensus.cc:2468] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 0.
I20260709 12:24:13.882055  7319 ts_tablet_manager.cc:1403] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.000s
I20260709 12:24:13.883294  7356 raft_consensus.cc:515] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.885866  7319 raft_consensus.cc:359] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.886793  7319 raft_consensus.cc:385] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:13.887123  7319 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Initialized, Role: FOLLOWER
I20260709 12:24:13.887121  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.888569  7201 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.888845  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.889923  7277 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.889964  7354 raft_consensus.cc:515] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.892167  7319 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.893039  7354 leader_election.cc:290] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.895195  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "702b90997ffe4e2c80d49becf831ca77" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:13.896034  7126 raft_consensus.cc:3060] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.896457  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "702b90997ffe4e2c80d49becf831ca77" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.897125  7278 raft_consensus.cc:3060] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.898485  7277 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:13.899379  7356 leader_election.cc:290] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.900846  7091 leader_election.cc:304] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:13.905972  7126 raft_consensus.cc:2468] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 1.
I20260709 12:24:13.907555  7166 leader_election.cc:304] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.908775  7354 raft_consensus.cc:2804] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.910188  7325 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.910256  7354 raft_consensus.cc:697] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:13.895493  7319 ts_tablet_manager.cc:1434] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:24:13.911636  7325 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.912240  7354 consensus_queue.cc:237] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.913147  7278 raft_consensus.cc:2468] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 1.
I20260709 12:24:13.915519  7346 consensus_queue.cc:1048] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:13.915970  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.916966  7325 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.915865  7201 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:13.902117  7356 raft_consensus.cc:2804] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.918432  7352 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.917470  7203 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.920636  7352 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.923267  7093 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.924337  7352 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.925385  7352 raft_consensus.cc:2804] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.925992  7352 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.926538  7352 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.927407  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.928450  7278 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.930824  7091 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:13.919193  7356 raft_consensus.cc:697] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:13.932492  7356 consensus_queue.cc:237] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.933373  7325 raft_consensus.cc:2804] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:13.933833  7325 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.934212  7325 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.936765  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:13.936173  7025 catalog_manager.cc:5697] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 1 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.937439  7203 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.937500  7352 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.940650  7352 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.945037  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.945806  7203 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.952395  7356 consensus_queue.cc:1048] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:13.951335  7026 catalog_manager.cc:5697] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:13.953074  7203 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:13.954838  7093 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:13.955463  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:13.956132  7277 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 0.
I20260709 12:24:13.956990  7356 raft_consensus.cc:2804] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:13.957516  7356 raft_consensus.cc:697] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:13.958541  7356 consensus_queue.cc:237] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.964701  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.965454  7277 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.971130  7325 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.978993  7346 raft_consensus.cc:493] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.980171  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:13.979982  7346 raft_consensus.cc:515] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:13.981124  7201 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.978965  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:13.986402  7278 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:13.987185  7346 leader_election.cc:290] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:13.977535  7325 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:13.989394  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4441f407c0394f348866d5436b855a76" is_pre_election: true
I20260709 12:24:13.994446  7126 raft_consensus.cc:2393] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66b8b104e71f4eb2b126c418c93eb29e in current term 1: Already voted for candidate 94cc4d2d82804b39bf733d2e70599c6b in this term.
I20260709 12:24:14.014916  7325 consensus_queue.cc:1048] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.032217  7023 catalog_manager.cc:5697] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.037302  7354 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.044307  7386 consensus_queue.cc:1048] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:14.051192  7242 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:34739, user_credentials={real_user=slave}} blocked reactor thread for 66540.2us
W20260709 12:24:14.053013  7074 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:14.053658  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:14.057183  7356 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.057838  7356 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.059274  7278 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:14.060582  7356 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.060802  7091 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:14.057868  7242 leader_election.cc:304] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e; no voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:14.061456  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b" is_pre_election: true
I20260709 12:24:14.063126  7356 raft_consensus.cc:2804] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:14.063742  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e" is_pre_election: true
I20260709 12:24:14.063970  7356 raft_consensus.cc:697] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.066927  7093 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4441f407c0394f348866d5436b855a76; no voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:14.067756  7356 consensus_queue.cc:237] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.070659  7277 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:14.072576  7326 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:14.073258  7201 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 1.
I20260709 12:24:14.068284  7325 raft_consensus.cc:2749] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:14.107452  7326 raft_consensus.cc:2749] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:14.108070  7126 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:14.120743  7396 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.120797  7023 catalog_manager.cc:5697] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.136098  7356 consensus_queue.cc:1048] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.151001  7380 consensus_queue.cc:1048] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.198271  7276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:50328
I20260709 12:24:14.198899  7276 raft_consensus.cc:606] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:14.218247  7203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:55232
I20260709 12:24:14.218915  7203 raft_consensus.cc:606] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:14.224184  7326 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.229099  7276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:50328
I20260709 12:24:14.229920  7276 raft_consensus.cc:606] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:14.237713  7389 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.257093  7388 raft_consensus.cc:993] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.257668  7388 raft_consensus.cc:1081] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.259914  7126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57036
I20260709 12:24:14.260687  7126 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.261130  7126 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.264603  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:57046
I20260709 12:24:14.265486  7128 raft_consensus.cc:606] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Received request to transfer leadership to TS 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:14.269052  7126 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.273313  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:14.274195  7203 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.274118  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:57046
I20260709 12:24:14.274382  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:14.275048  7128 raft_consensus.cc:606] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Received request to transfer leadership to TS 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:14.275342  7276 raft_consensus.cc:3055] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.275751  7126 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.275871  7276 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:14.277073  7276 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.278187  7276 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.283883  7203 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.285501  7093 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:14.286616  7380 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.287251  7380 raft_consensus.cc:697] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.288280  7380 consensus_queue.cc:237] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.293010  7126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:57046
I20260709 12:24:14.294018  7126 raft_consensus.cc:606] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Received request to transfer leadership to TS 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:14.294183  7276 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.298144  7025 catalog_manager.cc:5697] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 1 to 2, leader changed from 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.329457  7396 consensus_queue.cc:1048] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.345029  7328 consensus_queue.cc:1048] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.384514  7328 consensus_queue.cc:1048] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.400215  7328 consensus_queue.cc:1048] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.408161  7380 consensus_queue.cc:1048] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.418406  7325 raft_consensus.cc:993] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: : Instructing follower 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:14.419040  7352 raft_consensus.cc:1081] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Signalling peer 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:14.422415  7276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:50302
I20260709 12:24:14.424980  7276 raft_consensus.cc:493] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.425422  7276 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.428004  7325 consensus_queue.cc:1048] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.432993  7276 raft_consensus.cc:515] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.434428  7274 raft_consensus.cc:1240] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Rejecting Update request from peer 4441f407c0394f348866d5436b855a76 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:14.436789  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:14.437498  7126 raft_consensus.cc:3055] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.437865  7126 raft_consensus.cc:740] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.438750  7126 consensus_queue.cc:260] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.439283  7276 leader_election.cc:290] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 2 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:14.439860  7126 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.442034  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c396708f0764f1a82f6ef56fc840a12" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:14.442759  7203 raft_consensus.cc:3060] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.449885  7126 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 2.
I20260709 12:24:14.450157  7203 raft_consensus.cc:2468] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 2.
I20260709 12:24:14.451664  7240 leader_election.cc:304] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:14.452737  7389 raft_consensus.cc:2804] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.453595  7389 raft_consensus.cc:697] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:14.454859  7389 consensus_queue.cc:237] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.463097  7025 catalog_manager.cc:5697] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 1 to 2, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 2 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:14.476266  7325 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.492030  7356 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.500902  7380 raft_consensus.cc:993] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: : Instructing follower 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:14.501551  7356 raft_consensus.cc:1081] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Signalling peer 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:14.503786  7203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:55220
I20260709 12:24:14.504528  7203 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.504953  7203 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.510308  7203 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.513029  7203 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 2 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.514299  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:14.514650  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:14.515143  7126 raft_consensus.cc:3055] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.515658  7274 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.515632  7126 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.517282  7126 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.519228  7126 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.524684  7274 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 2.
I20260709 12:24:14.525056  7126 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 2.
I20260709 12:24:14.527103  7166 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:14.528116  7328 raft_consensus.cc:2804] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.528692  7328 raft_consensus.cc:697] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:14.529662  7328 consensus_queue.cc:237] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.537477  7356 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:14.538414  7380 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:14.539491  7023 catalog_manager.cc:5697] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 1 to 2, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 2 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.555438  7380 raft_consensus.cc:993] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: : Instructing follower 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:14.556079  7380 raft_consensus.cc:1081] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Signalling peer 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:14.561290  7202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:55220
I20260709 12:24:14.562605  7202 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.563112  7202 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.572412  7202 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.574699  7202 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 2 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.575841  7126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:14.576675  7126 raft_consensus.cc:3055] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.577162  7126 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.576185  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:14.578141  7126 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.578109  7274 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.580020  7126 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.581516  7389 raft_consensus.cc:993] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.582083  7389 raft_consensus.cc:1081] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.584396  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57036
I20260709 12:24:14.585253  7128 raft_consensus.cc:493] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.585716  7128 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.587312  7274 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 2.
I20260709 12:24:14.589028  7166 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:14.589641  7126 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 2.
I20260709 12:24:14.590261  7328 raft_consensus.cc:2804] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.591284  7328 raft_consensus.cc:697] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:14.592283  7328 consensus_queue.cc:237] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.593463  7128 raft_consensus.cc:515] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.596002  7128 leader_election.cc:290] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.597821  7202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:14.598453  7202 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.598786  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f58f58a18e5240edbda87fd3a69cbb87" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:14.599505  7274 raft_consensus.cc:3055] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.599902  7274 raft_consensus.cc:740] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:14.601315  7274 consensus_queue.cc:260] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.602355  7274 raft_consensus.cc:3060] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.603979  7026 catalog_manager.cc:5697] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 1 to 2, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 2 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.605794  7202 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.607254  7093 leader_election.cc:304] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:14.608300  7356 raft_consensus.cc:2804] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.608968  7356 raft_consensus.cc:697] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.609957  7356 consensus_queue.cc:237] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.610674  7274 raft_consensus.cc:2468] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.619100  7026 catalog_manager.cc:5697] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 1 to 2, leader changed from 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.666462  7328 raft_consensus.cc:993] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.667059  7354 raft_consensus.cc:1081] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:14.669243  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57044
I20260709 12:24:14.669867  7274 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.669930  7128 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.670683  7128 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.672132  7356 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:14.678696  7128 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.682714  7202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:14.683305  7203 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.683544  7202 raft_consensus.cc:3055] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.683722  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:14.684070  7202 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:14.684413  7274 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.685045  7128 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:14.685588  7352 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:14.686792  7202 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.688171  7202 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.694620  7274 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.696573  7091 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:14.700091  7356 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.700950  7356 raft_consensus.cc:697] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:14.702103  7356 consensus_queue.cc:237] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:14.704972  7202 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:14.712767  7025 catalog_manager.cc:5697] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 1 to 2, leader changed from 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:14.844082  7128 raft_consensus.cc:1275] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.846128  7389 consensus_queue.cc:1048] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:14.857329  7202 raft_consensus.cc:1275] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.859041  7346 consensus_queue.cc:1048] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:14.923404  7128 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.925506  7354 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:14.936544  7274 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.938613  7354 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:15.018397  7274 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.020205  7354 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.032208  7128 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.034927  7354 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.061393  7202 raft_consensus.cc:1275] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.063292  7352 consensus_queue.cc:1048] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.074512  7274 raft_consensus.cc:1275] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.076885  7352 consensus_queue.cc:1048] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:15.126824  7202 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.128791  7325 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.140424  7276 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.142311  7356 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:17.344149  6451 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:24:17.345206  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:17.368882  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:17.369589  7128 raft_consensus.cc:606] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Received request to transfer leadership to TS 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:17.372478  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:17.373035  7128 raft_consensus.cc:606] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Received request to transfer leadership to TS 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:17.374457  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:17.375036  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:17.778532  7421 raft_consensus.cc:993] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: : Instructing follower 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:17.779141  7424 raft_consensus.cc:1081] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Signalling peer 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:17.781770  7274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:50302
I20260709 12:24:17.782369  7274 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:17.782743  7274 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.788481  7274 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.790520  7274 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 3 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:17.791558  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:17.791976  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:17.792286  7128 raft_consensus.cc:3055] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:17.792604  7203 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.792680  7128 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:17.793709  7128 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.795158  7128 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.799086  7203 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 3.
I20260709 12:24:17.800830  7242 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:17.801865  7420 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:17.803330  7420 raft_consensus.cc:697] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:17.804414  7128 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 3.
I20260709 12:24:17.804476  7420 consensus_queue.cc:237] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.821723  7026 catalog_manager.cc:5697] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 2 to 3, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 3 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:17.877872  7421 raft_consensus.cc:993] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: : Instructing follower 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:17.878412  7424 raft_consensus.cc:1081] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Signalling peer 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:17.881867  7274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:50302
I20260709 12:24:17.882449  7274 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:17.882764  7274 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.888849  7274 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.890753  7274 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 3 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:17.892046  7202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:17.892025  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:17.892901  7202 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.892933  7128 raft_consensus.cc:3055] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:17.893549  7128 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:17.894505  7128 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.896220  7128 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.901472  7202 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 3.
I20260709 12:24:17.903486  7242 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:17.903911  7128 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 3.
I20260709 12:24:17.904695  7420 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:17.905268  7420 raft_consensus.cc:697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:17.906071  7420 consensus_queue.cc:237] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:17.913952  7026 catalog_manager.cc:5697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 2 to 3, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 3 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.232443  7128 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:18.234037  7420 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:18.249982  7202 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:18.252964  7420 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:18.376248  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:18.381829  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:18.382272  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:18.403476  7202 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:18.405505  7420 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:18.417690  7126 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:18.420223  7436 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:19.383394  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:19.389947  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:19.390552  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:20.392063  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:20.398401  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:20.398985  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:21.400534  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.406730  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:21.407615  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:22.408826  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:22.416052  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:22.416546  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:23.417866  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:23.425107  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:23.425657  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:24.427446  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:24.434325  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:24.434980  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:25.436105  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:25.442222  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:25.442749  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:27.094359  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:27.100277  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:27.100740  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:27.354008  7224 debug-util.cc:398] Leaking SignalData structure 0x7b08000fffe0 after lost signal to thread 6454
W20260709 12:24:27.355484  7224 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc6c0 after lost signal to thread 7060
W20260709 12:24:27.356707  7224 debug-util.cc:398] Leaking SignalData structure 0x7b080009f3c0 after lost signal to thread 7152
W20260709 12:24:27.357723  7224 debug-util.cc:398] Leaking SignalData structure 0x7b080009eee0 after lost signal to thread 7227
W20260709 12:24:27.358716  7224 debug-util.cc:398] Leaking SignalData structure 0x7b08001c85e0 after lost signal to thread 7302
I20260709 12:24:28.149987  6451 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:24:28.164786  7276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:50328
I20260709 12:24:28.166249  7276 raft_consensus.cc:606] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:28.176393  7462 raft_consensus.cc:993] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.179880  7450 raft_consensus.cc:1081] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.182569  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57036
I20260709 12:24:28.183413  7128 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:28.183938  7128 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.184504  7276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:50328
I20260709 12:24:28.185297  7276 raft_consensus.cc:606] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:28.193805  7201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:55232
I20260709 12:24:28.194537  7201 raft_consensus.cc:606] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:28.194213  7128 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.200516  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:28.201648  7276 raft_consensus.cc:3055] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:28.202127  7276 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:28.203776  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:28.204594  7201 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.203668  7276 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.204587  7203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:55232
I20260709 12:24:28.205570  7203 raft_consensus.cc:606] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:28.205859  7276 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.213464  7276 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 4.
I20260709 12:24:28.215291  7091 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:28.216838  7457 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:28.217417  7457 raft_consensus.cc:697] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:28.219033  7203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
mode: GRACEFUL
new_leader_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:55232
I20260709 12:24:28.218575  7457 consensus_queue.cc:237] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.219726  7203 raft_consensus.cc:606] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Received request to transfer leadership to TS 4441f407c0394f348866d5436b855a76
I20260709 12:24:28.221304  7128 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 4 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:28.216070  7201 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 4.
I20260709 12:24:28.237044  7025 catalog_manager.cc:5697] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 3 to 4, leader changed from 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.272122  7458 raft_consensus.cc:993] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.272945  7463 raft_consensus.cc:1081] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.275163  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57044
I20260709 12:24:28.276167  7128 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:28.276614  7128 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.282650  7128 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.285655  7128 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 3 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:28.286871  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:28.287415  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:28.287761  7201 raft_consensus.cc:3055] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:28.288187  7276 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.288216  7201 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:28.289765  7201 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.291651  7201 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.297959  7276 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 3.
I20260709 12:24:28.298787  7201 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 3.
I20260709 12:24:28.299598  7091 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 66b8b104e71f4eb2b126c418c93eb29e; no voters: 
I20260709 12:24:28.300659  7457 raft_consensus.cc:2804] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:28.301231  7457 raft_consensus.cc:697] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:28.302232  7457 consensus_queue.cc:237] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.312441  7025 catalog_manager.cc:5697] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 2 to 3, leader changed from 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.365865  7458 raft_consensus.cc:993] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.366466  7463 raft_consensus.cc:1081] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.368495  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57044
I20260709 12:24:28.369038  7128 raft_consensus.cc:493] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:28.369355  7128 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:28.374421  7128 raft_consensus.cc:515] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.376396  7128 leader_election.cc:290] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:28.377648  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:28.377812  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95527ee5a41a4461a29e72813e1e37e3" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:28.378433  7201 raft_consensus.cc:3055] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:28.378794  7201 raft_consensus.cc:740] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:28.378515  7276 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:28.380110  7201 consensus_queue.cc:260] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.381291  7201 raft_consensus.cc:3060] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:28.388594  7201 raft_consensus.cc:2468] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:28.389436  7276 raft_consensus.cc:2468] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 2.
I20260709 12:24:28.390060  7093 leader_election.cc:304] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:28.391125  7457 raft_consensus.cc:2804] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:28.391727  7457 raft_consensus.cc:697] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:28.392658  7457 consensus_queue.cc:237] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.402834  7025 catalog_manager.cc:5697] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 1 to 2, leader changed from 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.464932  7462 raft_consensus.cc:993] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.465641  7459 raft_consensus.cc:1081] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.467969  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57036
I20260709 12:24:28.468561  7128 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:28.469111  7128 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.475441  7128 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.477201  7128 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 4 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:28.478459  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:28.478592  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:28.479416  7201 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.479609  7276 raft_consensus.cc:3055] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:28.480154  7276 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:28.481185  7276 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.482620  7276 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.486691  7201 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 4.
I20260709 12:24:28.488476  7093 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:28.490481  7457 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:28.491894  7457 raft_consensus.cc:697] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:28.492769  7276 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 4.
I20260709 12:24:28.493753  7457 consensus_queue.cc:237] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.504695  7458 raft_consensus.cc:993] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: : Instructing follower 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.505296  7463 raft_consensus.cc:1081] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Signalling peer 4441f407c0394f348866d5436b855a76 to start an election
I20260709 12:24:28.507208  7128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "4441f407c0394f348866d5436b855a76"
 from {username='slave'} at 127.0.0.1:57044
I20260709 12:24:28.506313  7025 catalog_manager.cc:5697] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 3 to 4, leader changed from 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.507903  7128 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:28.508296  7128 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.516516  7128 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.519804  7128 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 3 election: Requested vote from peers 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:28.520334  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:28.521034  7201 raft_consensus.cc:3055] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:28.521679  7201 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:28.522814  7201 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.526417  7201 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.520673  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "4441f407c0394f348866d5436b855a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:28.535025  7276 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.537882  7201 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 3.
I20260709 12:24:28.539568  7093 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4441f407c0394f348866d5436b855a76, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:28.540635  7457 raft_consensus.cc:2804] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:28.541317  7457 raft_consensus.cc:697] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Becoming Leader. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:28.542450  7276 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4441f407c0394f348866d5436b855a76 in term 3.
I20260709 12:24:28.542454  7457 consensus_queue.cc:237] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:28.556175  7025 catalog_manager.cc:5697] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 reported cstate change: term changed from 2 to 3, leader changed from 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194) to 4441f407c0394f348866d5436b855a76 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "4441f407c0394f348866d5436b855a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.668298  7201 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:28.670135  7457 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:28.671062  7276 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:28.673738  7457 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:28.863587  7203 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:28.865540  7457 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:28.882278  7276 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:28.884361  7457 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:28.975445  7276 raft_consensus.cc:1275] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:28.977524  7457 consensus_queue.cc:1048] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:28.990219  7203 raft_consensus.cc:1275] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:28.992278  7460 consensus_queue.cc:1048] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:28.997103  7201 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:29.000175  7460 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:29.019438  7276 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:29.022475  7457 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:29.049906  7276 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:29.051573  7460 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:29.068977  7201 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Refusing update from remote peer 4441f407c0394f348866d5436b855a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:29.070886  7477 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.272806  6451 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:24:31.273765  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:31.282205  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:31.283000  7128 raft_consensus.cc:606] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Received request to transfer leadership to TS 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:31.286758  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:31.287398  7128 raft_consensus.cc:606] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Received request to transfer leadership to TS 66b8b104e71f4eb2b126c418c93eb29e
I20260709 12:24:31.290112  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:31.290609  7128 raft_consensus.cc:606] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Received request to transfer leadership to TS 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:31.293524  7128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "4441f407c0394f348866d5436b855a76"
mode: GRACEFUL
new_leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:57054
I20260709 12:24:31.294080  7128 raft_consensus.cc:606] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Received request to transfer leadership to TS 94cc4d2d82804b39bf733d2e70599c6b
I20260709 12:24:31.295671  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:24:31.296242  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:31.345206  7457 raft_consensus.cc:993] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: : Instructing follower 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:31.345737  7457 raft_consensus.cc:1081] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Signalling peer 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:31.347661  7203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:55220
I20260709 12:24:31.348244  7203 raft_consensus.cc:493] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:31.348722  7203 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.355350  7203 raft_consensus.cc:515] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.358741  7203 leader_election.cc:290] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 4 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:31.360317  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:31.361012  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6bc1447f794ed08bcae619ec50f012" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:31.361253  7128 raft_consensus.cc:3055] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:31.361771  7277 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.361788  7128 raft_consensus.cc:740] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:31.363119  7128 consensus_queue.cc:260] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.364521  7128 raft_consensus.cc:3060] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.369510  7277 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 4.
I20260709 12:24:31.370653  7128 raft_consensus.cc:2468] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 4.
I20260709 12:24:31.371688  7166 leader_election.cc:304] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:31.372857  7463 raft_consensus.cc:2804] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:31.374017  7463 raft_consensus.cc:697] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:31.374981  7463 consensus_queue.cc:237] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.384130  7025 catalog_manager.cc:5697] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 3 to 4, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 4 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.546623  7457 raft_consensus.cc:993] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: : Instructing follower 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:31.548635  7457 raft_consensus.cc:1081] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Signalling peer 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:31.549662  7497 raft_consensus.cc:993] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: : Instructing follower 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:31.550377  7457 raft_consensus.cc:1081] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Signalling peer 66b8b104e71f4eb2b126c418c93eb29e to start an election
I20260709 12:24:31.551200  7277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28b983178bd64ca89b590ae17897947d"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:50302
I20260709 12:24:31.551954  7277 raft_consensus.cc:493] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:31.552413  7277 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.552587  7276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5"
dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
 from {username='slave'} at 127.0.0.1:50302
I20260709 12:24:31.553205  7276 raft_consensus.cc:493] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:31.553561  7276 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.558809  7277 raft_consensus.cc:515] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.559715  7276 raft_consensus.cc:515] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.561506  7277 leader_election.cc:290] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 5 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:31.562356  7276 leader_election.cc:290] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 4 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739)
I20260709 12:24:31.563283  7203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:31.564018  7203 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.564051  7128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b983178bd64ca89b590ae17897947d" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:31.564766  7128 raft_consensus.cc:3055] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:31.564484  7201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
I20260709 12:24:31.565222  7128 raft_consensus.cc:740] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:31.565510  7201 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.564496  7127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91534a4d48b747588dff34ff5cf163a5" candidate_uuid: "66b8b104e71f4eb2b126c418c93eb29e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:31.566123  7128 consensus_queue.cc:260] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.567339  7127 raft_consensus.cc:3055] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:31.567803  7127 raft_consensus.cc:740] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:31.568111  7128 raft_consensus.cc:3060] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.568881  7127 consensus_queue.cc:260] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.571024  7127 raft_consensus.cc:3060] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.572582  7203 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 5.
I20260709 12:24:31.574255  7242 leader_election.cc:304] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:31.575855  7459 raft_consensus.cc:2804] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.577350  7128 raft_consensus.cc:2468] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 5.
I20260709 12:24:31.578799  7459 raft_consensus.cc:697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 5 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:31.579257  7201 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 4.
I20260709 12:24:31.580524  7242 leader_election.cc:304] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:31.580188  7459 consensus_queue.cc:237] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.581784  7499 raft_consensus.cc:2804] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:31.583189  7127 raft_consensus.cc:2468] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b8b104e71f4eb2b126c418c93eb29e in term 4.
I20260709 12:24:31.584764  7499 raft_consensus.cc:697] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 LEADER]: Becoming Leader. State: Replica: 66b8b104e71f4eb2b126c418c93eb29e, State: Running, Role: LEADER
I20260709 12:24:31.586016  7499 consensus_queue.cc:237] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.592624  7025 catalog_manager.cc:5697] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 4 to 5, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 5 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.607196  7025 catalog_manager.cc:5697] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e reported cstate change: term changed from 3 to 4, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195). New cstate: current_term: 4 leader_uuid: "66b8b104e71f4eb2b126c418c93eb29e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.671782  7497 raft_consensus.cc:993] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: : Instructing follower 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:31.672358  7457 raft_consensus.cc:1081] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Signalling peer 94cc4d2d82804b39bf733d2e70599c6b to start an election
I20260709 12:24:31.673976  7201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e"
dest_uuid: "94cc4d2d82804b39bf733d2e70599c6b"
 from {username='slave'} at 127.0.0.1:55220
I20260709 12:24:31.674458  7201 raft_consensus.cc:493] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:31.674746  7201 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.679914  7201 raft_consensus.cc:515] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.681799  7201 leader_election.cc:290] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 5 election: Requested vote from peers 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139), 66b8b104e71f4eb2b126c418c93eb29e (127.6.76.195:33427)
I20260709 12:24:31.683037  7127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4441f407c0394f348866d5436b855a76"
I20260709 12:24:31.683037  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58d799818bc4e16bde5239ceba7a57e" candidate_uuid: "94cc4d2d82804b39bf733d2e70599c6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "66b8b104e71f4eb2b126c418c93eb29e"
I20260709 12:24:31.684077  7277 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.684077  7127 raft_consensus.cc:3055] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:31.684655  7127 raft_consensus.cc:740] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4441f407c0394f348866d5436b855a76, State: Running, Role: LEADER
I20260709 12:24:31.685912  7127 consensus_queue.cc:260] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.687420  7127 raft_consensus.cc:3060] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.692610  7277 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 5.
I20260709 12:24:31.694243  7166 leader_election.cc:304] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b8b104e71f4eb2b126c418c93eb29e, 94cc4d2d82804b39bf733d2e70599c6b; no voters: 
I20260709 12:24:31.694988  7127 raft_consensus.cc:2468] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cc4d2d82804b39bf733d2e70599c6b in term 5.
I20260709 12:24:31.695163  7463 raft_consensus.cc:2804] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.695794  7463 raft_consensus.cc:697] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 5 LEADER]: Becoming Leader. State: Replica: 94cc4d2d82804b39bf733d2e70599c6b, State: Running, Role: LEADER
I20260709 12:24:31.696924  7463 consensus_queue.cc:237] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } }
I20260709 12:24:31.705119  7025 catalog_manager.cc:5697] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b reported cstate change: term changed from 4 to 5, leader changed from 4441f407c0394f348866d5436b855a76 (127.6.76.193) to 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194). New cstate: current_term: 5 leader_uuid: "94cc4d2d82804b39bf733d2e70599c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.803841  7277 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:31.805871  7463 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.819299  7127 raft_consensus.cc:1275] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:31.821640  7506 consensus_queue.cc:1048] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.970979  7127 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:31.972513  7459 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:31.984475  7201 raft_consensus.cc:1275] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:31.986759  7459 consensus_queue.cc:1048] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:32.034411  7127 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:32.035887  7459 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:32.051694  7201 raft_consensus.cc:1275] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Refusing update from remote peer 66b8b104e71f4eb2b126c418c93eb29e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:32.053495  7499 consensus_queue.cc:1048] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94cc4d2d82804b39bf733d2e70599c6b" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:32.144399  7127 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:32.146378  7506 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4441f407c0394f348866d5436b855a76" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:32.158244  7277 raft_consensus.cc:1275] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Refusing update from remote peer 94cc4d2d82804b39bf733d2e70599c6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:32.160606  7515 consensus_queue.cc:1048] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b8b104e71f4eb2b126c418c93eb29e" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:32.297600  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:32.303977  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:32.304610  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:33.306025  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:33.313621  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:33.314291  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:34.315810  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:34.322499  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:34.322973  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:35.324384  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:35.330896  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:35.331487  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:36.332563  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:36.340735  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:36.341189  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:37.342593  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:37.348623  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:37.349254  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:38.350687  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:38.357991  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:38.358614  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:39.359975  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:39.367781  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:39.368357  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:40.369640  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:40.376557  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:40.377007  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:41.421290  6451 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:24:41.421862  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
I20260709 12:24:41.462242  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:41.463152  6451 tablet_replica.cc:333] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.464408  6451 raft_consensus.cc:2243] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.465910  6451 raft_consensus.cc:2272] T ce6bc1447f794ed08bcae619ec50f012 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.468806  6451 tablet_replica.cc:333] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.469576  6451 raft_consensus.cc:2243] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.470158  6451 raft_consensus.cc:2272] T e58d799818bc4e16bde5239ceba7a57e P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.472625  6451 tablet_replica.cc:333] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.473281  6451 raft_consensus.cc:2243] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.473798  6451 raft_consensus.cc:2272] T 91534a4d48b747588dff34ff5cf163a5 P 4441f407c0394f348866d5436b855a76 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.476307  6451 tablet_replica.cc:333] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.476912  6451 raft_consensus.cc:2243] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.477406  6451 raft_consensus.cc:2272] T 702b90997ffe4e2c80d49becf831ca77 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.479709  6451 tablet_replica.cc:333] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.480305  6451 raft_consensus.cc:2243] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.481218  6451 raft_consensus.cc:2272] T 95527ee5a41a4461a29e72813e1e37e3 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.484028  6451 tablet_replica.cc:333] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.484668  6451 raft_consensus.cc:2243] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.485601  6451 raft_consensus.cc:2272] T 9a5a898c0d854d69a0ebd2813b833d79 P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.488163  6451 tablet_replica.cc:333] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.488991  6451 raft_consensus.cc:2243] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.489563  6451 raft_consensus.cc:2272] T 28b983178bd64ca89b590ae17897947d P 4441f407c0394f348866d5436b855a76 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.491675  6451 tablet_replica.cc:333] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.492219  6451 raft_consensus.cc:2243] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.493158  6451 raft_consensus.cc:2272] T f58f58a18e5240edbda87fd3a69cbb87 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.495261  6451 tablet_replica.cc:333] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.495977  6451 raft_consensus.cc:2243] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.496598  6451 raft_consensus.cc:2272] T 4c396708f0764f1a82f6ef56fc840a12 P 4441f407c0394f348866d5436b855a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.498584  6451 tablet_replica.cc:333] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76: stopping tablet replica
I20260709 12:24:41.499125  6451 raft_consensus.cc:2243] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.500160  6451 raft_consensus.cc:2272] T 53cc14a7fdd847fa87addbd10ff9f32b P 4441f407c0394f348866d5436b855a76 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:41.523026  7166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108) [suppressed 4 similar messages]
W20260709 12:24:41.529369  7166 consensus_peers.cc:597] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b -> Peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Couldn't send request to peer 4441f407c0394f348866d5436b855a76. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:33139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:41.539476  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
W20260709 12:24:41.543761  7240 consensus_peers.cc:597] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e -> Peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Couldn't send request to peer 4441f407c0394f348866d5436b855a76. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:33139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:41.570197  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
W20260709 12:24:41.596453  7240 consensus_peers.cc:597] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e -> Peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Couldn't send request to peer 4441f407c0394f348866d5436b855a76. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:33139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:41.613124  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:41.614082  6451 tablet_replica.cc:333] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.614924  6451 raft_consensus.cc:2243] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.616120  6451 raft_consensus.cc:2272] T ce6bc1447f794ed08bcae619ec50f012 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.618592  6451 tablet_replica.cc:333] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.619378  6451 raft_consensus.cc:2243] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.619998  6451 raft_consensus.cc:2272] T 91534a4d48b747588dff34ff5cf163a5 P 94cc4d2d82804b39bf733d2e70599c6b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.622430  6451 tablet_replica.cc:333] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.623260  6451 raft_consensus.cc:2243] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.624442  6451 raft_consensus.cc:2272] T 702b90997ffe4e2c80d49becf831ca77 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.626868  6451 tablet_replica.cc:333] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.627624  6451 raft_consensus.cc:2243] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.628154  6451 raft_consensus.cc:2272] T 28b983178bd64ca89b590ae17897947d P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.630270  6451 tablet_replica.cc:333] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.630918  6451 raft_consensus.cc:2243] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.631474  6451 raft_consensus.cc:2272] T 9a5a898c0d854d69a0ebd2813b833d79 P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.633935  6451 tablet_replica.cc:333] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.634825  6451 raft_consensus.cc:2243] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.635601  6451 raft_consensus.cc:2272] T 95527ee5a41a4461a29e72813e1e37e3 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.637969  6451 tablet_replica.cc:333] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.638684  6451 raft_consensus.cc:2243] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.639966  6451 raft_consensus.cc:2272] T e58d799818bc4e16bde5239ceba7a57e P 94cc4d2d82804b39bf733d2e70599c6b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.643357  6451 tablet_replica.cc:333] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.644104  6451 raft_consensus.cc:2243] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.644791  6451 raft_consensus.cc:2272] T 4c396708f0764f1a82f6ef56fc840a12 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.647148  6451 tablet_replica.cc:333] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.647671  6451 raft_consensus.cc:2243] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.648097  6451 raft_consensus.cc:2272] T f58f58a18e5240edbda87fd3a69cbb87 P 94cc4d2d82804b39bf733d2e70599c6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.649912  6451 tablet_replica.cc:333] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b: stopping tablet replica
I20260709 12:24:41.650502  6451 raft_consensus.cc:2243] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.650978  6451 raft_consensus.cc:2272] T 53cc14a7fdd847fa87addbd10ff9f32b P 94cc4d2d82804b39bf733d2e70599c6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:41.668197  7242 consensus_peers.cc:597] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e -> Peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Couldn't send request to peer 94cc4d2d82804b39bf733d2e70599c6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:41.674279  7240 consensus_peers.cc:597] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e -> Peer 4441f407c0394f348866d5436b855a76 (127.6.76.193:33139): Couldn't send request to peer 4441f407c0394f348866d5436b855a76. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:33139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:41.692617  7242 consensus_peers.cc:597] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e -> Peer 94cc4d2d82804b39bf733d2e70599c6b (127.6.76.194:34739): Couldn't send request to peer 94cc4d2d82804b39bf733d2e70599c6b. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:34739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:41.708585  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
I20260709 12:24:41.735591  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
I20260709 12:24:41.774257  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:41.775558  6451 tablet_replica.cc:333] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.776396  6451 raft_consensus.cc:2243] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.777033  6451 raft_consensus.cc:2272] T 53cc14a7fdd847fa87addbd10ff9f32b P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.779356  6451 tablet_replica.cc:333] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.780038  6451 raft_consensus.cc:2243] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.780558  6451 raft_consensus.cc:2272] T 95527ee5a41a4461a29e72813e1e37e3 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.782455  6451 tablet_replica.cc:333] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.783064  6451 raft_consensus.cc:2243] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.784041  6451 raft_consensus.cc:2272] T 28b983178bd64ca89b590ae17897947d P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.786010  6451 tablet_replica.cc:333] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.786662  6451 raft_consensus.cc:2243] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.787639  6451 raft_consensus.cc:2272] T 702b90997ffe4e2c80d49becf831ca77 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.789933  6451 tablet_replica.cc:333] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.790580  6451 raft_consensus.cc:2243] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.791556  6451 raft_consensus.cc:2272] T 91534a4d48b747588dff34ff5cf163a5 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.793537  6451 tablet_replica.cc:333] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.794066  6451 raft_consensus.cc:2243] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.794502  6451 raft_consensus.cc:2272] T 9a5a898c0d854d69a0ebd2813b833d79 P 66b8b104e71f4eb2b126c418c93eb29e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.796764  6451 tablet_replica.cc:333] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.797370  6451 raft_consensus.cc:2243] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.798226  6451 raft_consensus.cc:2272] T 4c396708f0764f1a82f6ef56fc840a12 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.800218  6451 tablet_replica.cc:333] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.800729  6451 raft_consensus.cc:2243] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.801156  6451 raft_consensus.cc:2272] T f58f58a18e5240edbda87fd3a69cbb87 P 66b8b104e71f4eb2b126c418c93eb29e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.802975  6451 tablet_replica.cc:333] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.803552  6451 raft_consensus.cc:2243] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.804013  6451 raft_consensus.cc:2272] T ce6bc1447f794ed08bcae619ec50f012 P 66b8b104e71f4eb2b126c418c93eb29e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.806440  6451 tablet_replica.cc:333] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e: stopping tablet replica
I20260709 12:24:41.807047  6451 raft_consensus.cc:2243] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:41.807624  6451 raft_consensus.cc:2272] T e58d799818bc4e16bde5239ceba7a57e P 66b8b104e71f4eb2b126c418c93eb29e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.842165  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:24:41.865888  6451 master.cc:562] Master@127.6.76.254:39151 shutting down...
I20260709 12:24:41.897610  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:41.898607  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:41.899063  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P dc0cca05c1cd4890a3a33c990befc109: stopping tablet replica
I20260709 12:24:41.920608  6451 master.cc:584] Master@127.6.76.254:39151 shutdown complete.
4441f407c0394f348866d5436b855a76  5
66b8b104e71f4eb2b126c418c93eb29e  1
94cc4d2d82804b39bf733d2e70599c6b  4
4441f407c0394f348866d5436b855a76  3
66b8b104e71f4eb2b126c418c93eb29e  3
94cc4d2d82804b39bf733d2e70599c6b  4
4441f407c0394f348866d5436b855a76  8
66b8b104e71f4eb2b126c418c93eb29e  1
94cc4d2d82804b39bf733d2e70599c6b  1
4441f407c0394f348866d5436b855a76  4
66b8b104e71f4eb2b126c418c93eb29e  3
94cc4d2d82804b39bf733d2e70599c6b  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (30104 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:24:41.970818  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:33941
I20260709 12:24:41.972180  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:41.978214  7534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:41.978377  7535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:41.981772  7537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:41.982751  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:41.984467  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:41.984714  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:41.984829  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599881984817 us; error 0 us; skew 500 ppm
I20260709 12:24:41.985453  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:41.988991  6451 webserver.cc:533] Webserver started at http://127.6.76.254:43391/ using document root <none> and password file <none>
I20260709 12:24:41.989574  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:41.989764  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:41.990036  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:41.991416  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "004cbdb090d14f8ca89ed535ed34a625"
format_stamp: "Formatted at 2026-07-09 12:24:41 on dist-test-slave-n447"
I20260709 12:24:41.997017  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:24:42.002163  7542 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.003118  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:24:42.003542  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "004cbdb090d14f8ca89ed535ed34a625"
format_stamp: "Formatted at 2026-07-09 12:24:41 on dist-test-slave-n447"
I20260709 12:24:42.003932  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:42.023044  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:42.024782  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:42.075659  7593 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:33941 every 8 connection(s)
I20260709 12:24:42.075659  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:33941
I20260709 12:24:42.080412  7594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.093139  7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Bootstrap starting.
I20260709 12:24:42.098711  7594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.104729  7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: No bootstrap required, opened a new log
I20260709 12:24:42.108156  7594 raft_consensus.cc:359] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER }
I20260709 12:24:42.108772  7594 raft_consensus.cc:385] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.109090  7594 raft_consensus.cc:740] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004cbdb090d14f8ca89ed535ed34a625, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.109858  7594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER }
I20260709 12:24:42.110584  7594 raft_consensus.cc:399] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:24:42.110934  7594 raft_consensus.cc:493] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:24:42.111323  7594 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.117553  7594 raft_consensus.cc:515] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER }
I20260709 12:24:42.118296  7594 leader_election.cc:304] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 004cbdb090d14f8ca89ed535ed34a625; no voters: 
I20260709 12:24:42.119783  7594 leader_election.cc:290] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:24:42.120316  7597 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.121896  7597 raft_consensus.cc:697] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 1 LEADER]: Becoming Leader. State: Replica: 004cbdb090d14f8ca89ed535ed34a625, State: Running, Role: LEADER
I20260709 12:24:42.122665  7597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER }
I20260709 12:24:42.123185  7594 sys_catalog.cc:565] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:24:42.127846  7599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004cbdb090d14f8ca89ed535ed34a625. Latest consensus state: current_term: 1 leader_uuid: "004cbdb090d14f8ca89ed535ed34a625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER } }
I20260709 12:24:42.128710  7599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:42.128592  7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "004cbdb090d14f8ca89ed535ed34a625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004cbdb090d14f8ca89ed535ed34a625" member_type: VOTER } }
I20260709 12:24:42.129799  7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:42.134737  7604 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:24:42.144704  7604 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:24:42.146605  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:24:42.156905  7604 catalog_manager.cc:1383] Generated new cluster ID: e57b08482be44cc4b858348648a4d972
I20260709 12:24:42.157295  7604 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:24:42.206380  7604 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:24:42.208042  7604 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:24:42.222064  7604 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Generated new TSK 0
I20260709 12:24:42.223011  7604 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:24:42.278987  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:42.286545  7615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:42.287643  7616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:42.290030  6451 server_base.cc:1061] running on GCE node
W20260709 12:24:42.289986  7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:42.291414  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:42.291690  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:42.291867  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599882291847 us; error 0 us; skew 500 ppm
I20260709 12:24:42.292565  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:42.295795  6451 webserver.cc:533] Webserver started at http://127.6.76.193:38543/ using document root <none> and password file <none>
I20260709 12:24:42.296445  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:42.296685  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:42.297010  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:42.298331  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.304502  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:24:42.309434  7623 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.310449  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:24:42.310899  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.311197  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:42.328287  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:42.330051  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:42.331954  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:42.334677  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:42.334944  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.335261  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:42.335496  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:24:42.392078  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:43519
I20260709 12:24:42.392210  7686 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:43519 every 8 connection(s)
I20260709 12:24:42.397794  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:42.405380  7691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:42.406989  7692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:42.412002  7694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:42.412751  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:42.414451  7687 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33941
I20260709 12:24:42.414922  7687 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:42.415037  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:42.415380  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:42.415619  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599882415595 us; error 0 us; skew 500 ppm
I20260709 12:24:42.416038  7687 heartbeater.cc:507] Master 127.6.76.254:33941 requested a full tablet report, sending...
I20260709 12:24:42.416529  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:42.419453  7559 ts_manager.cc:194] Registered new tserver with Master: be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:24:42.420406  6451 webserver.cc:533] Webserver started at http://127.6.76.194:45249/ using document root <none> and password file <none>
I20260709 12:24:42.421129  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:42.421427  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:42.421772  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:42.422262  7559 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60404
I20260709 12:24:42.424000  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.429981  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:24:42.434219  7699 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.435524  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:42.435994  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.436525  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:42.450739  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:42.452198  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:42.453868  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:42.456660  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:42.457022  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.457326  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:42.457545  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.511234  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:35283
I20260709 12:24:42.511430  7761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:35283 every 8 connection(s)
I20260709 12:24:42.516808  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:42.526317  7766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:42.528826  7768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:42.529258  7765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:42.530438  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:42.531636  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:42.531936  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:42.532140  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599882532093 us; error 0 us; skew 500 ppm
I20260709 12:24:42.532730  7762 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33941
I20260709 12:24:42.532871  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:42.533150  7762 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:42.533924  7762 heartbeater.cc:507] Master 127.6.76.254:33941 requested a full tablet report, sending...
I20260709 12:24:42.536183  6451 webserver.cc:533] Webserver started at http://127.6.76.195:38095/ using document root <none> and password file <none>
I20260709 12:24:42.536815  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:42.537034  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:42.537134  7559 ts_manager.cc:194] Registered new tserver with Master: fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:42.537360  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:42.538815  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "d72176ccaed049958ac68f706d4f052d"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.539182  7559 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60410
I20260709 12:24:42.545449  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:24:42.550719  7774 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.551849  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:24:42.552237  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "d72176ccaed049958ac68f706d4f052d"
format_stamp: "Formatted at 2026-07-09 12:24:42 on dist-test-slave-n447"
I20260709 12:24:42.552572  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:42.573338  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:42.574975  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:42.576766  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:42.579584  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:42.579849  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.580102  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:42.580291  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:42.644311  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:37801
I20260709 12:24:42.644470  7836 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:37801 every 8 connection(s)
I20260709 12:24:42.663064  7837 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33941
I20260709 12:24:42.663668  7837 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:42.664752  7837 heartbeater.cc:507] Master 127.6.76.254:33941 requested a full tablet report, sending...
I20260709 12:24:42.667572  7559 ts_manager.cc:194] Registered new tserver with Master: d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:42.667796  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019775185s
I20260709 12:24:42.669271  7559 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60420
I20260709 12:24:42.703423  7559 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60422:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:24:42.710328  7559 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:24:42.949702  7802 tablet_service.cc:1511] Processing CreateTablet for tablet cbbae2856e41420d91e3ccfbf507eacf (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:42.950121  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 691f0d885861432f91375d5f288af312 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:42.951504  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbbae2856e41420d91e3ccfbf507eacf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.952334  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691f0d885861432f91375d5f288af312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.958634  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 31230d31601445449fab8f0f9943db57 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:42.960065  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 6563fdb577f64184ac4e28f7c43e8c30 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:42.960283  7727 tablet_service.cc:1511] Processing CreateTablet for tablet cbbae2856e41420d91e3ccfbf507eacf (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:42.960593  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31230d31601445449fab8f0f9943db57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.962229  7724 tablet_service.cc:1511] Processing CreateTablet for tablet a3d64be480d9430189f5e290bab7c3b8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:42.963491  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbbae2856e41420d91e3ccfbf507eacf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.965392  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 62d1255be1b348739b8310c8f1a903d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:42.966624  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6563fdb577f64184ac4e28f7c43e8c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.968284  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 24790f3be4e8465581029a2e0d526117 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:42.970060  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24790f3be4e8465581029a2e0d526117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.962363  7726 tablet_service.cc:1511] Processing CreateTablet for tablet 691f0d885861432f91375d5f288af312 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:42.971624  7724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d64be480d9430189f5e290bab7c3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.972605  7726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691f0d885861432f91375d5f288af312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.974013  7800 tablet_service.cc:1511] Processing CreateTablet for tablet 31230d31601445449fab8f0f9943db57 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:42.975809  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31230d31601445449fab8f0f9943db57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.975453  7720 tablet_service.cc:1511] Processing CreateTablet for tablet 033af988cc8f496ca24e4443c21ea5d3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:42.977363  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d1255be1b348739b8310c8f1a903d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.978456  7718 tablet_service.cc:1511] Processing CreateTablet for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:42.980073  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.979854  7799 tablet_service.cc:1511] Processing CreateTablet for tablet a3d64be480d9430189f5e290bab7c3b8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:42.981068  7798 tablet_service.cc:1511] Processing CreateTablet for tablet 62d1255be1b348739b8310c8f1a903d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:42.981843  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d64be480d9430189f5e290bab7c3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.982769  7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d1255be1b348739b8310c8f1a903d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.986562  7719 tablet_service.cc:1511] Processing CreateTablet for tablet e8d96c92c58a4ad683334c5afc8e9a67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:42.988333  7719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d96c92c58a4ad683334c5afc8e9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.991421  7720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033af988cc8f496ca24e4443c21ea5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:42.995244  7797 tablet_service.cc:1511] Processing CreateTablet for tablet 24790f3be4e8465581029a2e0d526117 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:43.001220  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 6563fdb577f64184ac4e28f7c43e8c30 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:43.006839  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6563fdb577f64184ac4e28f7c43e8c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.006717  7795 tablet_service.cc:1511] Processing CreateTablet for tablet 033af988cc8f496ca24e4443c21ea5d3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:43.008607  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033af988cc8f496ca24e4443c21ea5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.009586  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24790f3be4e8465581029a2e0d526117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.021719  7794 tablet_service.cc:1511] Processing CreateTablet for tablet e8d96c92c58a4ad683334c5afc8e9a67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:43.024127  7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d96c92c58a4ad683334c5afc8e9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.050812  7652 tablet_service.cc:1511] Processing CreateTablet for tablet cbbae2856e41420d91e3ccfbf507eacf (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:43.053404  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbbae2856e41420d91e3ccfbf507eacf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.073025  7793 tablet_service.cc:1511] Processing CreateTablet for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:43.074775  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.076551  7722 tablet_service.cc:1511] Processing CreateTablet for tablet e78ad8f196c145088ee9ca42482e70ce (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:43.086560  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ad8f196c145088ee9ca42482e70ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.097644  7651 tablet_service.cc:1511] Processing CreateTablet for tablet 691f0d885861432f91375d5f288af312 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:43.099489  7651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691f0d885861432f91375d5f288af312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.108780  7856 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:43.109594  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 31230d31601445449fab8f0f9943db57 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:43.111416  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31230d31601445449fab8f0f9943db57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.121188  7801 tablet_service.cc:1511] Processing CreateTablet for tablet e78ad8f196c145088ee9ca42482e70ce (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:43.121937  7649 tablet_service.cc:1511] Processing CreateTablet for tablet a3d64be480d9430189f5e290bab7c3b8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:43.122808  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ad8f196c145088ee9ca42482e70ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.114378  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 268874518d1d4544a8500be048628b67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:43.125041  7648 tablet_service.cc:1511] Processing CreateTablet for tablet 62d1255be1b348739b8310c8f1a903d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:43.126130  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268874518d1d4544a8500be048628b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.128240  7647 tablet_service.cc:1511] Processing CreateTablet for tablet 24790f3be4e8465581029a2e0d526117 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:43.129531  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:43.131016  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 6563fdb577f64184ac4e28f7c43e8c30 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:43.134974  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 033af988cc8f496ca24e4443c21ea5d3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:43.131112  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5cde7dcc814e5d90f0b8e433f8f452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.140467  7643 tablet_service.cc:1511] Processing CreateTablet for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:43.138159  7644 tablet_service.cc:1511] Processing CreateTablet for tablet e8d96c92c58a4ad683334c5afc8e9a67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
W20260709 12:24:43.140468  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:43.143175  7649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d64be480d9430189f5e290bab7c3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.147644  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402d5cf4f7d24ac59dee1221fc2c7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.150465  7647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24790f3be4e8465581029a2e0d526117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.153734  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d96c92c58a4ad683334c5afc8e9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.155160  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6563fdb577f64184ac4e28f7c43e8c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.156718  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d1255be1b348739b8310c8f1a903d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.157891  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 539d703b24874183847fa9a8889111e9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:43.158568  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033af988cc8f496ca24e4443c21ea5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.150431  7798 tablet_service.cc:1511] Processing CreateTablet for tablet 268874518d1d4544a8500be048628b67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:43.180068  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 539d703b24874183847fa9a8889111e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.181026  7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268874518d1d4544a8500be048628b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.184883  7802 tablet_service.cc:1511] Processing CreateTablet for tablet 539d703b24874183847fa9a8889111e9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:43.185902  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:43.186491  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 539d703b24874183847fa9a8889111e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.187446  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5cde7dcc814e5d90f0b8e433f8f452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.167402  7726 tablet_service.cc:1511] Processing CreateTablet for tablet ddca3b06c0a84859adac6776a38c3518 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:43.202102  7726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddca3b06c0a84859adac6776a38c3518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.210597  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 629e345c827545598250d02a9f39e2fb (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:43.212675  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629e345c827545598250d02a9f39e2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.232249  7855 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:43.235601  7718 tablet_service.cc:1511] Processing CreateTablet for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:43.237326  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.256264  7794 tablet_service.cc:1511] Processing CreateTablet for tablet 629e345c827545598250d02a9f39e2fb (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:43.257910  7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629e345c827545598250d02a9f39e2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.259341  7856 tablet_bootstrap.cc:654] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.259625  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 0733d7d23d9041e09a5ade18807c383e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:43.266209  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 0733d7d23d9041e09a5ade18807c383e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:43.267063  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0733d7d23d9041e09a5ade18807c383e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.270583  7855 tablet_bootstrap.cc:654] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.272658  7647 tablet_service.cc:1511] Processing CreateTablet for tablet 268874518d1d4544a8500be048628b67 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:43.265226  7795 tablet_service.cc:1511] Processing CreateTablet for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:43.275413  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.275964  7650 tablet_service.cc:1511] Processing CreateTablet for tablet e78ad8f196c145088ee9ca42482e70ce (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:43.274314  7647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268874518d1d4544a8500be048628b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.223213  7799 tablet_service.cc:1511] Processing CreateTablet for tablet ddca3b06c0a84859adac6776a38c3518 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:43.290551  7651 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:43.293754  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0733d7d23d9041e09a5ade18807c383e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.287111  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 539d703b24874183847fa9a8889111e9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:43.305176  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddca3b06c0a84859adac6776a38c3518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.305683  7857 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:43.307740  7651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5cde7dcc814e5d90f0b8e433f8f452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.309453  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 116499cb3f0a4b4eba7b2ace2817bb25 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:43.311002  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116499cb3f0a4b4eba7b2ace2817bb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.313009  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ad8f196c145088ee9ca42482e70ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.319551  7648 tablet_service.cc:1511] Processing CreateTablet for tablet ddca3b06c0a84859adac6776a38c3518 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:43.321161  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddca3b06c0a84859adac6776a38c3518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.323122  7797 tablet_service.cc:1511] Processing CreateTablet for tablet 116499cb3f0a4b4eba7b2ace2817bb25 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:43.324980  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116499cb3f0a4b4eba7b2ace2817bb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.331833  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 539d703b24874183847fa9a8889111e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.343814  7857 tablet_bootstrap.cc:654] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.363787  7652 tablet_service.cc:1511] Processing CreateTablet for tablet 629e345c827545598250d02a9f39e2fb (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:43.365286  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629e345c827545598250d02a9f39e2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.368880  7855 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:43.369446  7855 ts_tablet_manager.cc:1403] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.138s	user 0.011s	sys 0.006s
I20260709 12:24:43.391971  7652 tablet_service.cc:1511] Processing CreateTablet for tablet 40860756ca234260bb6147416168cd6a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:43.393955  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40860756ca234260bb6147416168cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.405061  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 40860756ca234260bb6147416168cd6a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:43.416834  7855 raft_consensus.cc:359] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:43.417877  7855 raft_consensus.cc:385] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.418334  7855 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.419649  7855 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:43.423475  7648 tablet_service.cc:1511] Processing CreateTablet for tablet 9b24ad6f078449b6a6418a331b6099f2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:43.424202  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:43.426055  7687 heartbeater.cc:499] Master 127.6.76.254:33941 was elected leader, sending a full tablet report...
I20260709 12:24:43.429001  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 40860756ca234260bb6147416168cd6a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:43.430830  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40860756ca234260bb6147416168cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.447851  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40860756ca234260bb6147416168cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.455204  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b24ad6f078449b6a6418a331b6099f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.468569  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 9b24ad6f078449b6a6418a331b6099f2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:43.470031  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b24ad6f078449b6a6418a331b6099f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.467288  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.476209  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 9b24ad6f078449b6a6418a331b6099f2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:43.492904  7857 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:43.493072  7793 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbccb309817428d8d84eeecd4ac4410 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:43.493577  7857 ts_tablet_manager.cc:1403] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.188s	user 0.033s	sys 0.049s
I20260709 12:24:43.496832  7857 raft_consensus.cc:359] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.497723  7857 raft_consensus.cc:385] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.498044  7857 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.498816  7857 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.506678  7837 heartbeater.cc:499] Master 127.6.76.254:33941 was elected leader, sending a full tablet report...
I20260709 12:24:43.479301  7649 tablet_service.cc:1511] Processing CreateTablet for tablet 0733d7d23d9041e09a5ade18807c383e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:43.507355  7861 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.507947  7861 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:43.508479  7649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0733d7d23d9041e09a5ade18807c383e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.510392  7652 tablet_service.cc:1511] Processing CreateTablet for tablet d9c7a755001445e2895ad66ede4c8f60 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:43.512656  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c7a755001445e2895ad66ede4c8f60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.479367  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbccb309817428d8d84eeecd4ac4410 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:43.516319  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbccb309817428d8d84eeecd4ac4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.516554  7726 tablet_service.cc:1511] Processing CreateTablet for tablet d9c7a755001445e2895ad66ede4c8f60 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:43.518108  7726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c7a755001445e2895ad66ede4c8f60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.520102  7861 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:24:43.522251  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbccb309817428d8d84eeecd4ac4410 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:43.523828  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbccb309817428d8d84eeecd4ac4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.540395  7855 ts_tablet_manager.cc:1434] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.171s	user 0.020s	sys 0.012s
I20260709 12:24:43.542845  7762 heartbeater.cc:499] Master 127.6.76.254:33941 was elected leader, sending a full tablet report...
I20260709 12:24:43.545888  7856 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:43.546460  7857 ts_tablet_manager.cc:1434] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.022s
I20260709 12:24:43.546550  7800 tablet_service.cc:1511] Processing CreateTablet for tablet d9c7a755001445e2895ad66ede4c8f60 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:43.547909  7857 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:43.546439  7856 ts_tablet_manager.cc:1403] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.438s	user 0.010s	sys 0.013s
I20260709 12:24:43.561592  7643 tablet_service.cc:1511] Processing CreateTablet for tablet 116499cb3f0a4b4eba7b2ace2817bb25 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:43.563279  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b24ad6f078449b6a6418a331b6099f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.576572  7855 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:43.576613  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbccb309817428d8d84eeecd4ac4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.570674  7856 raft_consensus.cc:359] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.577863  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116499cb3f0a4b4eba7b2ace2817bb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.578101  7856 raft_consensus.cc:385] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.580528  7856 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.581773  7856 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.554927  7857 tablet_bootstrap.cc:654] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.596043  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c7a755001445e2895ad66ede4c8f60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.596722  7651 tablet_service.cc:1511] Processing CreateTablet for tablet 39c84f7f7e484afbae3c398efd5924b4 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:43.598210  7855 tablet_bootstrap.cc:654] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.598205  7651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c84f7f7e484afbae3c398efd5924b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.609792  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 39c84f7f7e484afbae3c398efd5924b4 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:43.616326  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c84f7f7e484afbae3c398efd5924b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.618206  7795 tablet_service.cc:1511] Processing CreateTablet for tablet 39c84f7f7e484afbae3c398efd5924b4 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:43.620092  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c84f7f7e484afbae3c398efd5924b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.625885  7644 tablet_service.cc:1511] Processing CreateTablet for tablet fd33cd795fd74820b3adafb3cdbc07c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:43.627632  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33cd795fd74820b3adafb3cdbc07c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:43.650842  7838 tablet.cc:2406] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:24:43.652441  7718 tablet_service.cc:1511] Processing CreateTablet for tablet fd33cd795fd74820b3adafb3cdbc07c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:43.654176  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33cd795fd74820b3adafb3cdbc07c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.659363  7799 tablet_service.cc:1511] Processing CreateTablet for tablet fd33cd795fd74820b3adafb3cdbc07c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:43.663555  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33cd795fd74820b3adafb3cdbc07c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.669195  7856 ts_tablet_manager.cc:1434] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.103s	user 0.011s	sys 0.006s
I20260709 12:24:43.670585  7856 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:43.688915  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:24:43.691149  7777 leader_election.cc:343] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.700490  7652 tablet_service.cc:1511] Processing CreateTablet for tablet ccd837ed3845462283f72c3010de59c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:43.702008  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd837ed3845462283f72c3010de59c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.703461  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:43.705678  7778 leader_election.cc:343] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.706210  7778 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:43.707834  7862 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:43.710796  7856 tablet_bootstrap.cc:654] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.744464  7723 tablet_service.cc:1511] Processing CreateTablet for tablet ccd837ed3845462283f72c3010de59c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:43.746775  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd837ed3845462283f72c3010de59c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.758745  7645 tablet_service.cc:1511] Processing CreateTablet for tablet cee2ab66e8604e5bbd1f08acbef110d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:43.760632  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee2ab66e8604e5bbd1f08acbef110d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.763993  7727 tablet_service.cc:1511] Processing CreateTablet for tablet cee2ab66e8604e5bbd1f08acbef110d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:43.761214  7799 tablet_service.cc:1511] Processing CreateTablet for tablet ccd837ed3845462283f72c3010de59c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:43.765664  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd837ed3845462283f72c3010de59c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.779896  7855 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:43.780857  7855 ts_tablet_manager.cc:1403] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.205s	user 0.010s	sys 0.009s
I20260709 12:24:43.797583  7863 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.798158  7863 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.803501  7863 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:43.803340  7855 raft_consensus.cc:359] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.804682  7855 raft_consensus.cc:385] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.805181  7855 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.819371  7795 tablet_service.cc:1511] Processing CreateTablet for tablet 189237b70e7f43458ac90a5773fd551f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:43.819731  7799 tablet_service.cc:1511] Processing CreateTablet for tablet cee2ab66e8604e5bbd1f08acbef110d2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:43.821175  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189237b70e7f43458ac90a5773fd551f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.820751  7646 tablet_service.cc:1511] Processing CreateTablet for tablet d353e70329004cc6b0c1a6bcda38e5c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:43.822358  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d353e70329004cc6b0c1a6bcda38e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.824710  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 189237b70e7f43458ac90a5773fd551f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:43.826259  7722 tablet_service.cc:1511] Processing CreateTablet for tablet d353e70329004cc6b0c1a6bcda38e5c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:43.828656  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee2ab66e8604e5bbd1f08acbef110d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.830395  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee2ab66e8604e5bbd1f08acbef110d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.834625  7857 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:43.835912  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189237b70e7f43458ac90a5773fd551f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.837613  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d353e70329004cc6b0c1a6bcda38e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.816143  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 189237b70e7f43458ac90a5773fd551f (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:43.839742  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189237b70e7f43458ac90a5773fd551f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.851353  7857 ts_tablet_manager.cc:1403] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.304s	user 0.015s	sys 0.015s
I20260709 12:24:43.863270  7856 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:43.816394  7855 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.872578  7643 tablet_service.cc:1511] Processing CreateTablet for tablet a0adec428658479f82eeacfa4eb81d5e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:43.874226  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0adec428658479f82eeacfa4eb81d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.870198  7856 ts_tablet_manager.cc:1403] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.200s	user 0.049s	sys 0.038s
I20260709 12:24:43.879436  7793 tablet_service.cc:1511] Processing CreateTablet for tablet d353e70329004cc6b0c1a6bcda38e5c9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:43.880923  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d353e70329004cc6b0c1a6bcda38e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.902732  7800 tablet_service.cc:1511] Processing CreateTablet for tablet a0adec428658479f82eeacfa4eb81d5e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:43.904359  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0adec428658479f82eeacfa4eb81d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.908448  7718 tablet_service.cc:1511] Processing CreateTablet for tablet a0adec428658479f82eeacfa4eb81d5e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:43.909597  7856 raft_consensus.cc:359] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.910231  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0adec428658479f82eeacfa4eb81d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.910501  7856 raft_consensus.cc:385] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.910907  7856 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.904397  7644 tablet_service.cc:1511] Processing CreateTablet for tablet bae510633fdb47c1a109530c37a265b0 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:43.912592  7855 ts_tablet_manager.cc:1434] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.131s	user 0.011s	sys 0.001s
I20260709 12:24:43.904642  7857 raft_consensus.cc:359] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.931335  7857 raft_consensus.cc:385] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.931784  7857 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.932637  7857 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.934685  7855 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:43.934240  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 463898dea61f4d7ab42c74b60beb15e6 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:43.935894  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463898dea61f4d7ab42c74b60beb15e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.936375  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 463898dea61f4d7ab42c74b60beb15e6 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:43.937558  7857 ts_tablet_manager.cc:1434] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.086s	user 0.011s	sys 0.001s
I20260709 12:24:43.929896  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae510633fdb47c1a109530c37a265b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.938978  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463898dea61f4d7ab42c74b60beb15e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.939834  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 79849a645ae64c3787630b03e7da6307 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:43.941145  7727 tablet_service.cc:1511] Processing CreateTablet for tablet bae510633fdb47c1a109530c37a265b0 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:43.941850  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:43.943388  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 79849a645ae64c3787630b03e7da6307 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:43.969022  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79849a645ae64c3787630b03e7da6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.970744  7795 tablet_service.cc:1511] Processing CreateTablet for tablet b69d30f987e646bfbea975a362ad70ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:43.943825  7726 tablet_service.cc:1511] Processing CreateTablet for tablet 463898dea61f4d7ab42c74b60beb15e6 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:43.972564  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b69d30f987e646bfbea975a362ad70ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:43.972898  7627 leader_election.cc:343] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.946367  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 79849a645ae64c3787630b03e7da6307 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:43.973387  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae510633fdb47c1a109530c37a265b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.974116  7726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463898dea61f4d7ab42c74b60beb15e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.974767  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79849a645ae64c3787630b03e7da6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.949743  7652 tablet_service.cc:1511] Processing CreateTablet for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:43.925937  7802 tablet_service.cc:1511] Processing CreateTablet for tablet bae510633fdb47c1a109530c37a265b0 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:43.955786  7797 tablet_service.cc:1511] Processing CreateTablet for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:43.980294  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.980597  7857 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:43.981305  7643 tablet_service.cc:1511] Processing CreateTablet for tablet d997d33a05304f31b50341cc8c4bbf9b (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:43.960364  7648 tablet_service.cc:1511] Processing CreateTablet for tablet b69d30f987e646bfbea975a362ad70ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:43.961030  7855 tablet_bootstrap.cc:654] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.963409  7868 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.972121  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79849a645ae64c3787630b03e7da6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.984131  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b69d30f987e646bfbea975a362ad70ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.947427  7856 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.983310  7868 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:43.987617  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.988507  7868 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:43.994242  7793 tablet_service.cc:1511] Processing CreateTablet for tablet d997d33a05304f31b50341cc8c4bbf9b (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:43.996145  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae510633fdb47c1a109530c37a265b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.996816  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997d33a05304f31b50341cc8c4bbf9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:43.997884  7719 tablet_service.cc:1511] Processing CreateTablet for tablet d997d33a05304f31b50341cc8c4bbf9b (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:43.999434  7719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997d33a05304f31b50341cc8c4bbf9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.004377  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:44.005981  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.008302  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997d33a05304f31b50341cc8c4bbf9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.035382  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 0741ccb73f0c4316be6682db332fd98e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:44.044711  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.045590  7812 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:44.047919  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:44.049275  7863 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.049723  7863 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.050073  7863 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.038117  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741ccb73f0c4316be6682db332fd98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.078320  7725 tablet_service.cc:1511] Processing CreateTablet for tablet b69d30f987e646bfbea975a362ad70ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:44.078579  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 0741ccb73f0c4316be6682db332fd98e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:44.080178  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b69d30f987e646bfbea975a362ad70ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.080881  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741ccb73f0c4316be6682db332fd98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.084919  7856 ts_tablet_manager.cc:1434] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.205s	user 0.017s	sys 0.005s
I20260709 12:24:44.086115  7856 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:44.122406  7855 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:44.123096  7855 ts_tablet_manager.cc:1403] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.189s	user 0.043s	sys 0.029s
I20260709 12:24:44.124711  7724 tablet_service.cc:1511] Processing CreateTablet for tablet b71af551cbdf4c63a9a0da6165082874 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:44.126209  7724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b71af551cbdf4c63a9a0da6165082874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.126480  7855 raft_consensus.cc:359] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.127492  7855 raft_consensus.cc:385] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.128094  7855 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.132382  7857 tablet_bootstrap.cc:654] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.135733  7855 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:44.145439  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:44.154335  7650 tablet_service.cc:1511] Processing CreateTablet for tablet b71af551cbdf4c63a9a0da6165082874 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:44.157641  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 0741ccb73f0c4316be6682db332fd98e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:44.159305  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741ccb73f0c4316be6682db332fd98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.164011  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b71af551cbdf4c63a9a0da6165082874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.164592  7856 tablet_bootstrap.cc:654] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.193094  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:24:44.197052  7702 leader_election.cc:343] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.199543  7855 ts_tablet_manager.cc:1434] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.005s
I20260709 12:24:44.202186  7863 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.211603  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.212627  7812 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:44.218765  7801 tablet_service.cc:1511] Processing CreateTablet for tablet b71af551cbdf4c63a9a0da6165082874 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:44.220196  7855 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:44.220474  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b71af551cbdf4c63a9a0da6165082874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.214900  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:44.237344  7875 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.237643  7857 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:44.237877  7875 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.238288  7875 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.238350  7857 ts_tablet_manager.cc:1403] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.258s	user 0.026s	sys 0.008s
I20260709 12:24:44.242782  7857 raft_consensus.cc:359] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.246707  7857 raft_consensus.cc:385] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.247160  7857 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.244741  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:44.248023  7812 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.248299  7857 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.249720  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 756aa7b7913e49aaba2a2164e45553fe (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:44.254825  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756aa7b7913e49aaba2a2164e45553fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.246318  7856 tablet_bootstrap.cc:492] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:44.243273  7863 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.267191  7644 tablet_service.cc:1511] Processing CreateTablet for tablet 756aa7b7913e49aaba2a2164e45553fe (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:44.269495  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756aa7b7913e49aaba2a2164e45553fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.271607  7856 ts_tablet_manager.cc:1403] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.010s
I20260709 12:24:44.276213  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 1e2e2ecc360e4304b9a841fbd6b8287e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:44.277282  7802 tablet_service.cc:1511] Processing CreateTablet for tablet 756aa7b7913e49aaba2a2164e45553fe (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:44.277797  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2e2ecc360e4304b9a841fbd6b8287e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.278793  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756aa7b7913e49aaba2a2164e45553fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.280376  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 1e2e2ecc360e4304b9a841fbd6b8287e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:44.281839  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2e2ecc360e4304b9a841fbd6b8287e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:44.291780  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 77035.8us
I20260709 12:24:44.262794  7875 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.295926  7875 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.259001  7812 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:44.298381  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:44.299501  7863 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.300037  7863 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:44.300899  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:44.300983  7863 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.305425  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 674d58a219c247d091f8a10ca4b75af1 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:44.307731  7881 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.307137  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674d58a219c247d091f8a10ca4b75af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.308504  7857 ts_tablet_manager.cc:1434] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.069s	user 0.023s	sys 0.017s
I20260709 12:24:44.310869  7857 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:44.308535  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 674d58a219c247d091f8a10ca4b75af1 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:44.305951  7793 tablet_service.cc:1511] Processing CreateTablet for tablet 1e2e2ecc360e4304b9a841fbd6b8287e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:24:44.315488  7627 leader_election.cc:343] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.316138  7856 raft_consensus.cc:359] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.316820  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2e2ecc360e4304b9a841fbd6b8287e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.316974  7856 raft_consensus.cc:385] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.317276  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674d58a219c247d091f8a10ca4b75af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.317444  7856 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.321643  7855 tablet_bootstrap.cc:654] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.327157  7857 tablet_bootstrap.cc:654] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.327898  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:24:44.330376  7702 leader_election.cc:343] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.331462  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:44.332427  7812 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.308293  7881 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.340631  7558 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.344002  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:44.345054  7737 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:44.347451  7802 tablet_service.cc:1511] Processing CreateTablet for tablet 674d58a219c247d091f8a10ca4b75af1 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:44.348114  7881 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.349335  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674d58a219c247d091f8a10ca4b75af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.349210  7648 tablet_service.cc:1511] Processing CreateTablet for tablet 35934889fce04bc093f07a0f03c7165c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:44.350875  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35934889fce04bc093f07a0f03c7165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.347491  7856 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.353267  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.357071  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:44.361374  7881 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.361886  7881 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:44.362090  7626 leader_election.cc:343] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.362421  7881 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.369469  7856 ts_tablet_manager.cc:1434] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.097s	user 0.004s	sys 0.003s
I20260709 12:24:44.370007  7718 tablet_service.cc:1511] Processing CreateTablet for tablet 35934889fce04bc093f07a0f03c7165c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:44.373350  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35934889fce04bc093f07a0f03c7165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.374281  7856 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:44.382033  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 35934889fce04bc093f07a0f03c7165c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:44.399252  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b27e8abd04f2ea19a70a13d1958ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:44.387809  7856 tablet_bootstrap.cc:654] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.390115  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b27e8abd04f2ea19a70a13d1958ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:44.392434  7812 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:44.395648  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b27e8abd04f2ea19a70a13d1958ec (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:44.404297  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b27e8abd04f2ea19a70a13d1958ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.406615  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35934889fce04bc093f07a0f03c7165c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.404297  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b27e8abd04f2ea19a70a13d1958ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.407464  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b27e8abd04f2ea19a70a13d1958ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.412806  7857 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:44.414016  7857 ts_tablet_manager.cc:1403] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.019s
I20260709 12:24:44.382602  7881 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.419100  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:44.419895  7737 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.419878  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 100d9b95a9cf4fab8305b579485c2332 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:44.421577  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d9b95a9cf4fab8305b579485c2332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.424360  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:44.425402  7875 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.426371  7875 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:44.427439  7875 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.430047  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 100d9b95a9cf4fab8305b579485c2332 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:44.440001  7881 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.440312  7737 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:44.451541  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:44.455518  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:24:44.457382  7626 leader_election.cc:343] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.456799  7857 raft_consensus.cc:359] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.458159  7857 raft_consensus.cc:385] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.453428  7881 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.458770  7857 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.459272  7881 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:44.484830  7881 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.460389  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d9b95a9cf4fab8305b579485c2332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.467829  7855 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:44.503144  7855 ts_tablet_manager.cc:1403] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.283s	user 0.013s	sys 0.009s
I20260709 12:24:44.504091  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 7e34ef3d78ea4d53847d21b92b07fab2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:44.504884  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 7e34ef3d78ea4d53847d21b92b07fab2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:44.505638  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e34ef3d78ea4d53847d21b92b07fab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.506253  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e34ef3d78ea4d53847d21b92b07fab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.506914  7855 raft_consensus.cc:359] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.507732  7855 raft_consensus.cc:385] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.508014  7855 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.508322  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 100d9b95a9cf4fab8305b579485c2332 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:44.509001  7855 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.509783  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d9b95a9cf4fab8305b579485c2332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.511508  7857 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.514575  7857 ts_tablet_manager.cc:1434] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.100s	user 0.002s	sys 0.009s
I20260709 12:24:44.516036  7857 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:44.515918  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 7e34ef3d78ea4d53847d21b92b07fab2 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:44.517551  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e34ef3d78ea4d53847d21b92b07fab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.517540  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2203ee13714c8bacacb6f83ec31034 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:44.524667  7856 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:44.525271  7856 ts_tablet_manager.cc:1403] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.151s	user 0.019s	sys 0.020s
I20260709 12:24:44.525527  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2203ee13714c8bacacb6f83ec31034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.534794  7644 tablet_service.cc:1511] Processing CreateTablet for tablet 38bf041bf77540baa50b5319733c5e9e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:44.537140  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bf041bf77540baa50b5319733c5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.538576  7802 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2203ee13714c8bacacb6f83ec31034 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:44.542275  7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2203ee13714c8bacacb6f83ec31034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.559124  7643 tablet_service.cc:1511] Processing CreateTablet for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:44.578152  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.552347  7797 tablet_service.cc:1511] Processing CreateTablet for tablet 38bf041bf77540baa50b5319733c5e9e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:44.579850  7793 tablet_service.cc:1511] Processing CreateTablet for tablet 7112151d65374f6690a117439cc2dc33 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:44.568024  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:44.580857  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bf041bf77540baa50b5319733c5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.581588  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.583474  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7112151d65374f6690a117439cc2dc33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.563655  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 38bf041bf77540baa50b5319733c5e9e (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:44.584573  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 7112151d65374f6690a117439cc2dc33 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:44.585321  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bf041bf77540baa50b5319733c5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.586285  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7112151d65374f6690a117439cc2dc33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.571493  7875 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.587080  7875 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.589584  7875 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.590678  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 8987ec43c27c4d578d5ad1e143553334 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:44.592011  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.592795  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8987ec43c27c4d578d5ad1e143553334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.593019  7812 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:44.593779  7648 tablet_service.cc:1511] Processing CreateTablet for tablet 8987ec43c27c4d578d5ad1e143553334 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:44.595372  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:44.595779  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 7112151d65374f6690a117439cc2dc33 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:44.597262  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7112151d65374f6690a117439cc2dc33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.597321  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:44.598743  7875 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.573117  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:44.600664  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.600999  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8987ec43c27c4d578d5ad1e143553334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.575551  7863 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.603461  7863 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.604212  7795 tablet_service.cc:1511] Processing CreateTablet for tablet 8987ec43c27c4d578d5ad1e143553334 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:44.607553  7863 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.608263  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.609503  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:44.610713  7626 leader_election.cc:343] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.611574  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8987ec43c27c4d578d5ad1e143553334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.572618  7856 raft_consensus.cc:359] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.614588  7856 raft_consensus.cc:385] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.614923  7856 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.570580  7857 tablet_bootstrap.cc:654] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.555886  7855 ts_tablet_manager.cc:1434] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.003s
I20260709 12:24:44.627926  7856 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.629256  7737 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:44.628635  7855 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:44.632977  7627 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:44.635128  7863 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.635854  7863 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.636314  7863 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.641541  7652 tablet_service.cc:1511] Processing CreateTablet for tablet 16eac2fd2da54bec8a02bb35cc376087 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:44.643129  7652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16eac2fd2da54bec8a02bb35cc376087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.540961  7881 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.657959  7721 tablet_service.cc:1511] Processing CreateTablet for tablet 16eac2fd2da54bec8a02bb35cc376087 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:44.661226  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16eac2fd2da54bec8a02bb35cc376087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.677573  7855 tablet_bootstrap.cc:654] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.575700  7718 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2203ee13714c8bacacb6f83ec31034 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:44.696542  7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2203ee13714c8bacacb6f83ec31034. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:44.703812  7545 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48081.2us
I20260709 12:24:44.679471  7881 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.721158  7856 ts_tablet_manager.cc:1434] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.196s	user 0.015s	sys 0.000s
I20260709 12:24:44.722465  7856 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:44.722916  7796 tablet_service.cc:1511] Processing CreateTablet for tablet 16eac2fd2da54bec8a02bb35cc376087 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:44.731362  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16eac2fd2da54bec8a02bb35cc376087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.731729  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 08777b07fa9d4993a24c05bb4e2db3c8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:44.733263  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08777b07fa9d4993a24c05bb4e2db3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.733016  7863 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.742632  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:44.744485  7627 leader_election.cc:343] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.747006  7559 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.748930  7881 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.749717  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.750495  7812 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:44.752808  7626 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:44.757141  7881 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.757725  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:44.758487  7737 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.763775  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 08777b07fa9d4993a24c05bb4e2db3c8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:44.783882  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08777b07fa9d4993a24c05bb4e2db3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.783489  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 3960664d9eb541f185c9eac4a231e5c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:44.785257  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3960664d9eb541f185c9eac4a231e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.757742  7881 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.786419  7881 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.791070  7737 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:44.791450  7857 tablet_bootstrap.cc:492] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:44.792282  7857 ts_tablet_manager.cc:1403] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.277s	user 0.008s	sys 0.028s
I20260709 12:24:44.792840  7627 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:44.793974  7883 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.795738  7857 raft_consensus.cc:359] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.796868  7857 raft_consensus.cc:385] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.797238  7857 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.798231  7857 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.802913  7855 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:44.803790  7855 ts_tablet_manager.cc:1403] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.175s	user 0.019s	sys 0.011s
I20260709 12:24:44.768658  7863 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.773072  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:44.807157  7800 tablet_service.cc:1511] Processing CreateTablet for tablet 3960664d9eb541f185c9eac4a231e5c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:44.776680  7856 tablet_bootstrap.cc:654] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.809104  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3960664d9eb541f185c9eac4a231e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:44.809265  7626 leader_election.cc:343] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.814015  7857 ts_tablet_manager.cc:1434] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.021s	user 0.015s	sys 0.001s
I20260709 12:24:44.815299  7857 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:44.821794  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:44.822712  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:44.823519  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f766c3d13d4360bcdeb1d5f0f318a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.824061  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f766c3d13d4360bcdeb1d5f0f318a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.826107  7883 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:44.827572  7883 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.828473  7799 tablet_service.cc:1511] Processing CreateTablet for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:44.830045  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f766c3d13d4360bcdeb1d5f0f318a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.824076  7881 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.782258  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 08777b07fa9d4993a24c05bb4e2db3c8 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:44.839920  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08777b07fa9d4993a24c05bb4e2db3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.782414  7722 tablet_service.cc:1511] Processing CreateTablet for tablet 3960664d9eb541f185c9eac4a231e5c3 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:44.844290  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3960664d9eb541f185c9eac4a231e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.845021  7857 tablet_bootstrap.cc:654] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.848698  7855 raft_consensus.cc:359] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.849603  7855 raft_consensus.cc:385] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.850134  7855 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.853842  7855 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.858204  7856 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:44.858856  7856 ts_tablet_manager.cc:1403] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.003s
I20260709 12:24:44.781963  7875 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.875362  7895 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.879750  7895 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.885912  7895 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.890394  7723 tablet_service.cc:1511] Processing CreateTablet for tablet b3b8edd0c3f4472bb922dba093edf25a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:44.893412  7895 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.893402  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:44.893622  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:44.882407  7559 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.896050  7812 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:44.898645  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:44.903611  7811 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.904259  7662 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.899326  7645 tablet_service.cc:1511] Processing CreateTablet for tablet b3b8edd0c3f4472bb922dba093edf25a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:44.906399  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b8edd0c3f4472bb922dba093edf25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.900041  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:44.908864  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:44.901197  7855 ts_tablet_manager.cc:1434] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.001s
I20260709 12:24:44.898849  7856 raft_consensus.cc:359] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.911502  7856 raft_consensus.cc:385] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.912997  7856 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.918272  7857 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:44.923455  7895 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.926553  7883 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.903132  7881 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:44.931556  7855 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:44.891829  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b8edd0c3f4472bb922dba093edf25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.949419  7811 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:44.951308  7857 ts_tablet_manager.cc:1403] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.136s	user 0.012s	sys 0.020s
I20260709 12:24:44.951632  7796 tablet_service.cc:1511] Processing CreateTablet for tablet b3b8edd0c3f4472bb922dba093edf25a (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:44.954511  7644 tablet_service.cc:1511] Processing CreateTablet for tablet 41a5afed00934a1d9878dd900b236263 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:44.955446  7857 raft_consensus.cc:359] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:44.956283  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a5afed00934a1d9878dd900b236263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.956404  7857 raft_consensus.cc:385] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.956878  7857 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.957818  7857 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:44.959558  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b8edd0c3f4472bb922dba093edf25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.964161  7727 tablet_service.cc:1511] Processing CreateTablet for tablet 41a5afed00934a1d9878dd900b236263 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:44.965557  7727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a5afed00934a1d9878dd900b236263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.966347  7855 tablet_bootstrap.cc:654] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.968178  7643 tablet_service.cc:1511] Processing CreateTablet for tablet db9652df6fb248539296e0b903cc4d1c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:44.969939  7626 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:44.970497  7856 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.970932  7881 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.971611  7737 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.969594  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9652df6fb248539296e0b903cc4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.973272  7856 ts_tablet_manager.cc:1434] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.114s	user 0.009s	sys 0.002s
I20260709 12:24:44.962334  7557 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.971606  7881 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:44.979749  7875 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.969537  7857 ts_tablet_manager.cc:1434] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:24:44.987725  7856 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:45.014925  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 41a5afed00934a1d9878dd900b236263 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:45.019001  7737 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:45.064942  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a5afed00934a1d9878dd900b236263. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:44.999061  7626 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:45.073637  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 99249.5us
I20260709 12:24:44.993613  7649 tablet_service.cc:1511] Processing CreateTablet for tablet 575c307765244f48890311159c0ee934 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:45.070315  7868 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.986153  7725 tablet_service.cc:1511] Processing CreateTablet for tablet db9652df6fb248539296e0b903cc4d1c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:45.083936  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9652df6fb248539296e0b903cc4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.988763  7881 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:44.988790  7793 tablet_service.cc:1511] Processing CreateTablet for tablet db9652df6fb248539296e0b903cc4d1c (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:45.088477  7793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9652df6fb248539296e0b903cc4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.090759  7856 tablet_bootstrap.cc:654] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:45.073160  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 74392.5us
I20260709 12:24:45.115273  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 575c307765244f48890311159c0ee934 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:45.102592  7857 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.116976  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575c307765244f48890311159c0ee934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.102037  7868 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.112094  7649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575c307765244f48890311159c0ee934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.126170  7895 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.110814  7862 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.127943  7862 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.129417  7863 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:45.127036  7895 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:45.104074  7875 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
W20260709 12:24:45.154824  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:45.157119  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:45.158988  7627 leader_election.cc:343] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.163003  7801 tablet_service.cc:1511] Processing CreateTablet for tablet 575c307765244f48890311159c0ee934 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:45.165777  7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575c307765244f48890311159c0ee934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.161767  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 248d73a0462c48fdad073c597b243806 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:45.173611  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248d73a0462c48fdad073c597b243806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.185495  7868 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.191768  7862 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:45.192715  7855 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.193360  7855 ts_tablet_manager.cc:1403] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.268s	user 0.023s	sys 0.028s
I20260709 12:24:45.193542  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:45.195495  7778 leader_election.cc:343] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.198642  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.203549  7800 tablet_service.cc:1511] Processing CreateTablet for tablet 248d73a0462c48fdad073c597b243806 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:45.204059  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.171156  7895 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:45.249644  7842 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.6.76.254:33941, user_credentials={real_user=slave}} blocked reactor thread for 47456.4us
I20260709 12:24:45.205255  7809 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:45.244917  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.241701  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.262224  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248d73a0462c48fdad073c597b243806. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:45.271579  7909 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:24:45.287515  7777 leader_election.cc:343] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.288087  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:45.289268  7862 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:45.294337  7810 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:45.295987  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:45.297292  7902 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:45.297870  7902 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.298352  7902 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.273728  7643 tablet_service.cc:1511] Processing CreateTablet for tablet 248d73a0462c48fdad073c597b243806 (DEFAULT_TABLE table=test-workload [id=07c684bbd79840cf825767ed5d335938]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:45.310469  7643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248d73a0462c48fdad073c597b243806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.278026  7855 raft_consensus.cc:359] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.318372  7855 raft_consensus.cc:385] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.328181  7626 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:45.330188  7895 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.330780  7895 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.331269  7895 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.318929  7855 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.424968  7855 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.428319  7855 ts_tablet_manager.cc:1434] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.235s	user 0.015s	sys 0.000s
I20260709 12:24:45.429579  7855 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.429719  7857 tablet_bootstrap.cc:654] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:45.431382  7545 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 145148us
I20260709 12:24:45.435719  7856 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.436379  7856 ts_tablet_manager.cc:1403] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.449s	user 0.072s	sys 0.075s
W20260709 12:24:45.421151  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 93185.4us
W20260709 12:24:45.421532  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 125657us
W20260709 12:24:45.443461  7626 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:45.444797  7856 raft_consensus.cc:359] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.445713  7856 raft_consensus.cc:385] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.446082  7856 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.451184  7855 tablet_bootstrap.cc:654] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.464221  7896 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.471578  7856 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.471578  7559 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.488915  7856 ts_tablet_manager.cc:1434] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.002s
I20260709 12:24:45.490528  7856 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
W20260709 12:24:45.495555  7545 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55185.5us
I20260709 12:24:45.503504  7896 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.505860  7856 tablet_bootstrap.cc:654] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.548678  7895 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:45.566402  7857 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.568735  7857 ts_tablet_manager.cc:1403] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.491s	user 0.016s	sys 0.012s
I20260709 12:24:45.572201  7857 raft_consensus.cc:359] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.573004  7857 raft_consensus.cc:385] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.573357  7857 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.576213  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:45.576824  7895 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:45.577911  7627 leader_election.cc:343] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.578240  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:45.578986  7812 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.563071  7902 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.582871  7736 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:45.585287  7902 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.586216  7857 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.586620  7855 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.588137  7855 ts_tablet_manager.cc:1403] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.159s	user 0.052s	sys 0.080s
I20260709 12:24:45.588686  7856 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.589550  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:45.588969  7896 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.589890  7856 ts_tablet_manager.cc:1403] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.100s	user 0.026s	sys 0.018s
I20260709 12:24:45.591270  7896 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:45.591758  7896 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:45.592686  7855 raft_consensus.cc:359] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.593029  7896 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.589857  7857 ts_tablet_manager.cc:1434] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:24:45.594007  7856 raft_consensus.cc:359] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.594882  7856 raft_consensus.cc:385] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.595283  7856 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.595449  7857 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.588562  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:45.596477  7856 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.598845  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:45.599839  7902 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.599558  7856 ts_tablet_manager.cc:1434] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:24:45.595001  7896 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.604315  7856 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:45.607460  7857 tablet_bootstrap.cc:654] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.593762  7855 raft_consensus.cc:385] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.616788  7855 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.616991  7856 tablet_bootstrap.cc:654] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.617765  7855 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.620167  7857 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.620762  7857 ts_tablet_manager.cc:1403] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.006s
I20260709 12:24:45.623890  7855 ts_tablet_manager.cc:1434] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.002s
I20260709 12:24:45.625162  7855 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.624821  7857 raft_consensus.cc:359] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.626073  7857 raft_consensus.cc:385] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.626492  7857 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.627709  7857 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.631395  7857 ts_tablet_manager.cc:1434] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.001s
I20260709 12:24:45.632961  7857 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.634440  7855 tablet_bootstrap.cc:654] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.643564  7857 tablet_bootstrap.cc:654] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.652634  7855 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.653245  7855 ts_tablet_manager.cc:1403] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.007s
I20260709 12:24:45.638239  7812 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:45.659539  7857 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.660419  7857 ts_tablet_manager.cc:1403] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.012s
I20260709 12:24:45.664544  7857 raft_consensus.cc:359] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.664994  7855 raft_consensus.cc:359] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.665699  7857 raft_consensus.cc:385] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.665882  7855 raft_consensus.cc:385] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.666378  7857 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.666464  7855 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.667941  7855 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.668309  7626 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:45.668804  7857 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.670228  7855 ts_tablet_manager.cc:1434] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:24:45.670290  7863 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.670998  7857 ts_tablet_manager.cc:1434] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:24:45.671552  7855 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.671693  7856 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.672180  7857 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.672304  7856 ts_tablet_manager.cc:1403] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.068s	user 0.046s	sys 0.014s
I20260709 12:24:45.671043  7863 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:45.673575  7863 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:45.675977  7856 raft_consensus.cc:359] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.677030  7856 raft_consensus.cc:385] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.677399  7856 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.678457  7855 tablet_bootstrap.cc:654] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.678185  7856 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.678457  7857 tablet_bootstrap.cc:654] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.680698  7856 ts_tablet_manager.cc:1434] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:45.681849  7856 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:45.688539  7856 tablet_bootstrap.cc:654] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.692788  7857 tablet_bootstrap.cc:492] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.693532  7857 ts_tablet_manager.cc:1403] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.007s
I20260709 12:24:45.697678  7857 raft_consensus.cc:359] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.698747  7857 raft_consensus.cc:385] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.699152  7857 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.700999  7857 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.703135  7855 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.703332  7857 ts_tablet_manager.cc:1434] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 12:24:45.703747  7855 ts_tablet_manager.cc:1403] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.008s
I20260709 12:24:45.704382  7857 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.707513  7855 raft_consensus.cc:359] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.708328  7855 raft_consensus.cc:385] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.708648  7855 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.709586  7855 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.712108  7857 tablet_bootstrap.cc:654] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.712157  7855 ts_tablet_manager.cc:1434] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:24:45.715473  7856 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.715327  7557 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:45.716266  7856 ts_tablet_manager.cc:1403] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.013s
I20260709 12:24:45.718415  7855 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.720614  7856 raft_consensus.cc:359] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.721889  7856 raft_consensus.cc:385] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.722470  7856 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.723539  7856 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.726287  7856 ts_tablet_manager.cc:1434] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:45.726531  7855 tablet_bootstrap.cc:654] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.748769  7856 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:45.758291  7856 tablet_bootstrap.cc:654] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.758309  7857 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.759089  7857 ts_tablet_manager.cc:1403] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.055s	user 0.023s	sys 0.028s
I20260709 12:24:45.760842  7855 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.761363  7855 ts_tablet_manager.cc:1403] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.012s
I20260709 12:24:45.762395  7857 raft_consensus.cc:359] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.763535  7857 raft_consensus.cc:385] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.763938  7857 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.764139  7855 raft_consensus.cc:359] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.764770  7855 raft_consensus.cc:385] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.765084  7855 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.764966  7857 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.765961  7855 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.768357  7857 ts_tablet_manager.cc:1434] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:24:45.770994  7855 ts_tablet_manager.cc:1434] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:24:45.771210  7857 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.772140  7855 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.773062  7856 tablet_bootstrap.cc:492] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.773661  7856 ts_tablet_manager.cc:1403] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.005s
I20260709 12:24:45.778053  7856 raft_consensus.cc:359] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.778954  7856 raft_consensus.cc:385] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.779302  7856 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.779843  7857 tablet_bootstrap.cc:654] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.780202  7855 tablet_bootstrap.cc:654] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.780159  7856 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.784373  7856 ts_tablet_manager.cc:1434] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 12:24:45.785423  7856 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:45.790427  7902 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.791143  7902 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.793073  7856 tablet_bootstrap.cc:654] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.793756  7902 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.795452  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.795588  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.796320  7812 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.796482  7661 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.797159  7857 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.797730  7857 ts_tablet_manager.cc:1403] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.003s
I20260709 12:24:45.800256  7902 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.800894  7902 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.801142  7857 raft_consensus.cc:359] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.801985  7857 raft_consensus.cc:385] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.802354  7857 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.802726  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.803610  7902 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.804294  7902 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.804688  7902 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.805055  7902 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.807488  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.808554  7812 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.811777  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.814348  7862 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.815197  7862 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.817015  7868 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.817646  7868 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.818111  7868 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.819314  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.819747  7902 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.822184  7662 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:45.822412  7902 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.823437  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.823880  7777 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:45.824189  7661 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.825596  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:45.826561  7862 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:45.827142  7862 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.827539  7862 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.827907  7862 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.828049  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:45.830159  7778 leader_election.cc:343] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.826282  7662 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.832000  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:45.833026  7812 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.833639  7902 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.834285  7902 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.834385  7857 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.844587  7857 ts_tablet_manager.cc:1434] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:24:45.847062  7868 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.849502  7662 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:45.851300  7810 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.852572  7868 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.859779  7902 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.864038  7855 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:45.864652  7857 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:45.864660  7855 ts_tablet_manager.cc:1403] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.093s	user 0.026s	sys 0.012s
I20260709 12:24:45.865557  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:45.865660  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:45.866914  7809 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.867072  7662 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.873651  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.874471  7811 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.874506  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.876367  7862 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.881307  7857 tablet_bootstrap.cc:654] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.878058  7809 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:45.878214  7855 raft_consensus.cc:359] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.878578  7902 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.857203  7812 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:45.881335  7856 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:45.885031  7855 raft_consensus.cc:385] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.885970  7855 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.886101  7856 ts_tablet_manager.cc:1403] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.101s	user 0.009s	sys 0.008s
I20260709 12:24:45.885195  7902 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.882277  7862 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:45.882747  7868 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.877554  7662 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:45.888724  7868 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.889825  7856 raft_consensus.cc:359] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.890795  7868 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.890623  7856 raft_consensus.cc:385] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.883526  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:45.883548  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:45.891312  7856 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.886971  7855 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.891695  7661 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.892272  7856 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.893689  7736 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.894393  7855 ts_tablet_manager.cc:1434] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 12:24:45.895934  7855 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:45.882813  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.900221  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.890867  7902 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.901527  7875 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.901971  7875 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.902333  7875 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.904086  7855 tablet_bootstrap.cc:654] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.907502  7868 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.910087  7661 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:45.913262  7777 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:45.914109  7736 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:45.914825  7862 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.915755  7862 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:45.916949  7862 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.920850  7856 ts_tablet_manager.cc:1434] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 12:24:45.922503  7856 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
W20260709 12:24:45.923696  7702 leader_election.cc:343] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.932324  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.933420  7902 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.934223  7902 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:45.938172  7857 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:45.942338  7856 tablet_bootstrap.cc:654] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.943099  7857 ts_tablet_manager.cc:1403] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.006s
I20260709 12:24:45.940457  7902 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.938948  7862 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.947757  7862 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.952546  7868 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:45.950801  7559 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:45.954382  7875 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.959218  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.961230  7863 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.961741  7863 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.962679  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.963506  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:45.963991  7875 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.964146  7863 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:45.964423  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.963442  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.967986  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
W20260709 12:24:45.967063  7702 leader_election.cc:343] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.972115  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:45.974014  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:45.974860  7810 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:45.975045  7702 leader_election.cc:343] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.975112  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:45.976227  7737 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:45.976039  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:45.977766  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:45.978551  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:45.980105  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:45.981432  7863 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.981897  7702 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:45.980271  7875 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:45.983820  7875 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.984511  7875 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.984550  7812 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.985169  7875 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.981925  7863 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.988600  7863 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.993072  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:45.995927  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:45.955180  7868 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:45.997623  7862 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:46.000799  7810 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:46.005648  7855 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.006352  7855 ts_tablet_manager.cc:1403] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.111s	user 0.022s	sys 0.027s
I20260709 12:24:46.002894  7863 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.008469  7661 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:46.007933  7857 raft_consensus.cc:359] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:45.983781  7702 leader_election.cc:343] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.011443  7857 raft_consensus.cc:385] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.012527  7857 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.002904  7875 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.014938  7737 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:46.015025  7863 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.027817  7896 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.038635  7896 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.020476  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.040069  7810 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.014431  7857 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.041692  7778 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.043220  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:46.044920  7626 leader_election.cc:343] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.045666  7896 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.026448  7559 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.018541  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:46.048606  7936 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.047643  7857 ts_tablet_manager.cc:1434] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.104s	user 0.009s	sys 0.002s
I20260709 12:24:46.017429  7855 raft_consensus.cc:359] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.050078  7855 raft_consensus.cc:385] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.050573  7855 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.054895  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:46.055918  7661 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:46.058207  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.059616  7857 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.051383  7855 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.016392  7875 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.063925  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:46.065213  7736 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:46.068110  7855 ts_tablet_manager.cc:1434] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.008s
I20260709 12:24:46.069059  7855 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.073453  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.071532  7936 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:46.077865  7702 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:46.096211  7856 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.109242  7855 tablet_bootstrap.cc:654] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.138473  7856 ts_tablet_manager.cc:1403] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.216s	user 0.049s	sys 0.073s
I20260709 12:24:46.107499  7857 tablet_bootstrap.cc:654] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.138978  7943 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:24:46.139248  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63968.2us
W20260709 12:24:46.140394  7627 leader_election.cc:343] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.140898  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.096774  7863 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260709 12:24:46.141776  7896 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.093005  7936 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.095412  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:46.145347  7862 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.146111  7856 raft_consensus.cc:359] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:46.147643  7856 raft_consensus.cc:385] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.147838  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.148067  7856 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.149725  7863 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.150473  7863 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:46.151830  7863 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.149120  7856 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:46.161515  7856 ts_tablet_manager.cc:1434] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:24:46.163486  7943 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.164106  7943 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } }
I20260709 12:24:46.165979  7857 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.167426  7857 ts_tablet_manager.cc:1403] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.016s
I20260709 12:24:46.175936  7943 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:24:46.177265  7856 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.177771  7855 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.177493  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.178555  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.178301  7855 ts_tablet_manager.cc:1403] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.109s	user 0.026s	sys 0.015s
W20260709 12:24:46.175010  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:46.180862  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:46.184751  7855 raft_consensus.cc:359] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.185587  7855 raft_consensus.cc:385] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.185932  7855 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.188366  7855 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.191287  7855 ts_tablet_manager.cc:1434] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:24:46.191145  7943 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.192755  7855 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.193712  7557 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.196043  7557 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.200100  7855 tablet_bootstrap.cc:654] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.196838  7857 raft_consensus.cc:359] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.214551  7857 raft_consensus.cc:385] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.214880  7857 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.225495  7857 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.237787  7857 ts_tablet_manager.cc:1434] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:24:46.239300  7857 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.241181  7736 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.241787  7856 tablet_bootstrap.cc:654] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.251008  7559 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.256878  7855 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.257536  7855 ts_tablet_manager.cc:1403] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.065s	user 0.033s	sys 0.016s
I20260709 12:24:46.259303  7857 tablet_bootstrap.cc:654] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.273613  7896 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.279893  7896 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.280548  7896 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.280992  7857 tablet_bootstrap.cc:492] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.281800  7857 ts_tablet_manager.cc:1403] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.008s
I20260709 12:24:46.283358  7896 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.285106  7855 raft_consensus.cc:359] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.285658  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.286577  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.287719  7857 raft_consensus.cc:359] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.288489  7857 raft_consensus.cc:385] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:46.288575  7627 leader_election.cc:343] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:46.289386  7626 leader_election.cc:343] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.288872  7857 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.290047  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.289559  7855 raft_consensus.cc:385] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.290781  7857 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.291867  7855 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.292347  7863 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.295331  7857 ts_tablet_manager.cc:1434] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:24:46.296698  7857 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.294926  7855 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.303748  7857 tablet_bootstrap.cc:654] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.309355  7856 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.310146  7856 ts_tablet_manager.cc:1403] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.019s
I20260709 12:24:46.315828  7855 ts_tablet_manager.cc:1434] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.058s	user 0.015s	sys 0.000s
I20260709 12:24:46.317880  7855 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.319108  7945 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.319993  7945 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.324343  7856 raft_consensus.cc:359] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.325058  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.327006  7856 raft_consensus.cc:385] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.327536  7856 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.328372  7856 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.329913  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:46.332732  7778 leader_election.cc:343] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.326663  7945 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:24:46.334616  7777 leader_election.cc:343] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.335397  7777 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.336812  7855 tablet_bootstrap.cc:654] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.337416  7945 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.338724  7856 ts_tablet_manager.cc:1434] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.006s
I20260709 12:24:46.340400  7856 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.357913  7857 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.358520  7857 ts_tablet_manager.cc:1403] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.028s
I20260709 12:24:46.359512  7856 tablet_bootstrap.cc:654] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.362331  7857 raft_consensus.cc:359] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.363255  7857 raft_consensus.cc:385] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.363601  7857 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.364440  7857 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.366923  7857 ts_tablet_manager.cc:1434] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:46.367808  7856 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.368532  7856 ts_tablet_manager.cc:1403] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.006s
I20260709 12:24:46.368880  7857 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.371964  7856 raft_consensus.cc:359] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.372777  7856 raft_consensus.cc:385] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.373067  7856 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.373798  7856 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.374967  7855 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.375811  7855 ts_tablet_manager.cc:1403] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.011s
I20260709 12:24:46.376814  7856 ts_tablet_manager.cc:1434] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:46.377465  7857 tablet_bootstrap.cc:654] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.378001  7856 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.379498  7855 raft_consensus.cc:359] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.380501  7855 raft_consensus.cc:385] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.380903  7855 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.382054  7855 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.385975  7856 tablet_bootstrap.cc:654] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.394138  7945 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.394706  7857 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.395546  7862 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.395784  7932 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.396202  7862 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.396512  7932 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.398012  7856 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.398569  7856 ts_tablet_manager.cc:1403] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.012s
I20260709 12:24:46.399324  7862 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:46.400385  7857 ts_tablet_manager.cc:1403] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.005s
I20260709 12:24:46.401329  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.401790  7856 raft_consensus.cc:359] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.402563  7856 raft_consensus.cc:385] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.402872  7856 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
W20260709 12:24:46.403306  7778 leader_election.cc:343] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.403666  7856 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.407395  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.408113  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.408393  7662 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:46.408586  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.409312  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:46.408957  7660 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:46.410445  7856 ts_tablet_manager.cc:1434] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 12:24:46.404182  7857 raft_consensus.cc:359] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.411128  7857 raft_consensus.cc:385] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.411301  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:46.411770  7856 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.411656  7857 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.412379  7862 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:46.412959  7862 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.412495  7932 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:46.413406  7862 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.416328  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.419876  7932 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:46.420472  7932 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.420883  7856 tablet_bootstrap.cc:654] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.421296  7932 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.423905  7857 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.426695  7857 ts_tablet_manager.cc:1434] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:24:46.428088  7857 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.429984  7863 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.430624  7863 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.431447  7855 ts_tablet_manager.cc:1434] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:24:46.432614  7943 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.433488  7856 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.434084  7856 ts_tablet_manager.cc:1403] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 12:24:46.435082  7855 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.435554  7863 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.429240  7862 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.437345  7856 raft_consensus.cc:359] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.436908  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.441320  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:46.446173  7856 raft_consensus.cc:385] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.446866  7660 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.441388  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:46.447480  7626 leader_election.cc:343] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:46.448621  7627 leader_election.cc:343] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.449290  7627 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.440187  7932 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.451093  7863 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.452829  7862 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:46.454924  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:46.455926  7737 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.457880  7660 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:46.446864  7856 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.459803  7777 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:46.460430  7856 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.461794  7862 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.461977  7943 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.462433  7862 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:46.462870  7856 ts_tablet_manager.cc:1434] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:24:46.463482  7862 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.464182  7856 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.462512  7943 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.459777  7857 tablet_bootstrap.cc:654] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.470422  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.471206  7662 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:46.472786  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:46.473013  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:46.473666  7660 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.473955  7936 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:46.474582  7936 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.475142  7936 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.477224  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:46.478912  7856 tablet_bootstrap.cc:654] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.479316  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.480190  7932 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:24:46.479365  7778 leader_election.cc:343] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.481923  7943 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:46.483225  7702 leader_election.cc:343] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.487077  7895 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.487870  7895 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.497856  7932 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.498052  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.501438  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:46.499663  7855 tablet_bootstrap.cc:654] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.501044  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.502745  7809 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:46.498934  7895 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.515724  7902 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.508288  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:46.520064  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:46.525393  7856 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.526055  7856 ts_tablet_manager.cc:1403] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.020s
I20260709 12:24:46.501439  7936 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.529531  7863 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:46.530138  7863 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.530529  7863 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.511919  7660 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:46.533363  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.538501  7932 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.568307  7868 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:24:46.580003  7936 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.578118  7932 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:46.597617  7855 tablet_bootstrap.cc:492] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.598091  7855 ts_tablet_manager.cc:1403] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.165s	user 0.038s	sys 0.013s
I20260709 12:24:46.600412  7856 raft_consensus.cc:359] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.601260  7856 raft_consensus.cc:385] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.601581  7856 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.603298  7856 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.606134  7857 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.606688  7857 ts_tablet_manager.cc:1403] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.179s	user 0.032s	sys 0.026s
I20260709 12:24:46.593906  7936 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.612673  7856 ts_tablet_manager.cc:1434] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.002s
I20260709 12:24:46.614490  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:46.614959  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.615386  7662 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:46.620868  7702 leader_election.cc:343] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.623431  7856 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.609565  7936 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.623698  7857 raft_consensus.cc:359] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.647644  7857 raft_consensus.cc:385] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.648064  7857 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.647006  7896 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.651072  7896 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.610055  7932 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.651187  7902 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:24:46.665349  7868 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.665306  7896 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:46.643055  7702 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:46.651083  7857 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.666066  7868 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.674067  7868 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.675305  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.680251  7855 raft_consensus.cc:359] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.681278  7855 raft_consensus.cc:385] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.681799  7855 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.683279  7857 ts_tablet_manager.cc:1434] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.076s	user 0.012s	sys 0.004s
I20260709 12:24:46.684430  7857 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.687012  7855 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.626931  7863 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.691660  7895 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:46.690321  7626 leader_election.cc:343] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.703011  7662 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:46.712920  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.713934  7809 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.723706  7855 ts_tablet_manager.cc:1434] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.125s	user 0.015s	sys 0.001s
I20260709 12:24:46.724330  7863 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.732486  7855 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.742148  7809 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:46.743350  7936 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.743863  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:46.744570  7856 tablet_bootstrap.cc:654] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.744818  7863 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.745342  7863 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:46.746845  7863 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:46.737510  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66039.5us
W20260709 12:24:46.758524  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 87223.3us
I20260709 12:24:46.772356  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:46.772104  7976 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:46.777688  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:46.778759  7737 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.779399  7857 tablet_bootstrap.cc:654] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.788128  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.793704  7661 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:46.797899  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.798662  7811 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
W20260709 12:24:46.801844  7627 leader_election.cc:343] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.802002  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:46.802480  7627 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:46.808287  7857 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.807374  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.799363  7558 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:46.809211  7857 ts_tablet_manager.cc:1403] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.125s	user 0.039s	sys 0.027s
I20260709 12:24:46.801883  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.811316  7856 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.809787  7863 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.811846  7856 ts_tablet_manager.cc:1403] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.189s	user 0.021s	sys 0.018s
I20260709 12:24:46.807307  7973 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:46.812552  7973 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:46.814167  7973 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.813742  7558 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:46.815395  7856 raft_consensus.cc:359] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.817685  7856 raft_consensus.cc:385] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.818159  7856 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.819157  7856 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.823048  7856 ts_tablet_manager.cc:1434] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:24:46.824693  7856 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.826875  7855 tablet_bootstrap.cc:654] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.829840  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:46.831403  7974 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:46.831466  7702 leader_election.cc:343] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.832195  7974 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.832793  7974 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.848647  7857 raft_consensus.cc:359] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.849643  7857 raft_consensus.cc:385] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.849977  7857 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
W20260709 12:24:46.852089  7702 leader_election.cc:343] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:46.851926  7973 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:46.852588  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:46.850787  7857 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.854530  7936 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:46.855967  7737 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:46.864934  7857 ts_tablet_manager.cc:1434] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.001s
I20260709 12:24:46.874025  7857 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:46.878364  7856 tablet_bootstrap.cc:654] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.886847  7855 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:46.887590  7855 ts_tablet_manager.cc:1403] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.019s
I20260709 12:24:46.912719  7855 raft_consensus.cc:359] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.913650  7855 raft_consensus.cc:385] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.913991  7855 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.914729  7855 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.918887  7855 ts_tablet_manager.cc:1434] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.005s
I20260709 12:24:46.918926  7936 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.919595  7936 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.920114  7855 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:46.920960  7974 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.921785  7936 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.933701  7974 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.935585  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:46.935585  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:46.936609  7809 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:46.936609  7661 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:46.939291  7856 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.939415  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:46.939996  7855 tablet_bootstrap.cc:654] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.940028  7856 ts_tablet_manager.cc:1403] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.116s	user 0.023s	sys 0.020s
I20260709 12:24:46.940423  7936 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:46.940850  7936 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.941220  7936 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.943784  7856 raft_consensus.cc:359] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.944545  7856 raft_consensus.cc:385] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.944845  7856 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.945662  7856 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.944981  7857 tablet_bootstrap.cc:654] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.948575  7856 ts_tablet_manager.cc:1434] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:46.949908  7856 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.953254  7559 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.959162  7856 tablet_bootstrap.cc:654] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:46.962924  7936 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.965778  7936 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.970795  7857 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:46.970348  7557 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.971438  7857 ts_tablet_manager.cc:1403] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.012s
I20260709 12:24:46.972084  7856 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:46.972606  7856 ts_tablet_manager.cc:1403] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
I20260709 12:24:46.974510  7857 raft_consensus.cc:359] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.975250  7856 raft_consensus.cc:359] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.975924  7857 raft_consensus.cc:385] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.976137  7856 raft_consensus.cc:385] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:46.976279  7857 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.976444  7856 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:46.978039  7857 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.978039  7856 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.980530  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.981292  7809 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.981590  7973 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.982174  7973 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:46.984611  7856 ts_tablet_manager.cc:1434] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:24:46.980628  7857 ts_tablet_manager.cc:1434] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:46.985091  7973 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:46.986389  7856 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:46.981467  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:46.987342  7812 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:46.995853  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:46.996811  7660 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.000797  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.002754  7812 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.002635  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.006127  7857 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.005928  7661 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.006454  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.009187  7810 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:46.996194  7856 tablet_bootstrap.cc:654] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.013291  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.023049  7857 tablet_bootstrap.cc:654] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:47.027642  7702 leader_election.cc:343] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.021765  7936 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.031651  7936 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:47.037600  7932 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:24:47.038476  7932 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.040755  7932 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.037788  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:47.042022  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.044222  7662 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:24:47.043213  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.045616  7855 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:47.045578  7777 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:47.046470  7855 ts_tablet_manager.cc:1403] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.038s
I20260709 12:24:47.045897  7735 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 2: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:47.048171  7862 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:47.048624  7809 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.048624  7862 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:24:47.050439  7862 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:47.050158  7855 raft_consensus.cc:359] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.053458  7855 raft_consensus.cc:385] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.054193  7855 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.055876  7855 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.059551  7855 ts_tablet_manager.cc:1434] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:24:47.052963  7857 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:47.060614  7857 ts_tablet_manager.cc:1403] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.000s
I20260709 12:24:47.060945  7945 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.051878  7660 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.068487  7855 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:47.049795  7936 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.066610  7857 raft_consensus.cc:359] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.072158  7857 raft_consensus.cc:385] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.072698  7857 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.073593  7973 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:47.074142  7973 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.074615  7973 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.073812  7857 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:47.083861  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46710.5us
I20260709 12:24:47.083866  7857 ts_tablet_manager.cc:1434] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 12:24:47.091792  7857 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.092121  7660 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.086720  7661 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.082841  7973 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.089360  7855 tablet_bootstrap.cc:654] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.066779  7862 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.104470  7862 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.109376  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.109958  7902 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.106125  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47462.9us
I20260709 12:24:47.112957  7902 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.113386  7974 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.116914  7974 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:47.117115  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:47.118882  7735 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 2: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:47.119658  7974 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.120602  7857 tablet_bootstrap.cc:654] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.119724  7902 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.134752  7856 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:47.136397  7856 ts_tablet_manager.cc:1403] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.150s	user 0.031s	sys 0.044s
I20260709 12:24:47.138062  7895 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.143893  7558 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.144730  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.165436  7857 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:47.166428  7857 ts_tablet_manager.cc:1403] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.005s
I20260709 12:24:47.140483  7973 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.176913  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.172672  7856 raft_consensus.cc:359] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.182114  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.183276  7809 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:24:47.182128  7856 raft_consensus.cc:385] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.184386  7856 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.186194  7856 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.188774  7857 raft_consensus.cc:359] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.189574  7857 raft_consensus.cc:385] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.189915  7857 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.186691  7945 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.188270  7895 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.191704  7895 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.194103  7895 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.197856  7943 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.198549  7943 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.200332  7902 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.200711  7943 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.200923  7902 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.190680  7857 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.207587  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.207893  7902 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.208160  7857 ts_tablet_manager.cc:1434] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.001s
I20260709 12:24:47.209285  7857 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.215988  7856 ts_tablet_manager.cc:1434] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.000s
I20260709 12:24:47.219067  7856 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:47.218881  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:47.223668  7626 leader_election.cc:343] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.224952  7945 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.220821  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:47.226943  7812 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.250545  7895 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.253279  7895 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.255479  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.256258  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
W20260709 12:24:47.258464  7702 leader_election.cc:343] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.262362  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.264984  7945 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.265720  7945 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.266161  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.271138  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.272714  7855 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:47.273363  7855 ts_tablet_manager.cc:1403] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.205s	user 0.019s	sys 0.002s
I20260709 12:24:47.274425  7895 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.289790  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.274885  7936 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:47.293334  7936 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.293769  7936 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.296083  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:47.297200  7626 leader_election.cc:343] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.291209  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.300778  7735 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.268093  7661 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
W20260709 12:24:47.303568  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:47.307883  7945 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.320487  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.323422  7734 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:47.325536  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.327221  7945 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:47.327852  7945 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.328272  7945 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.328830  7855 raft_consensus.cc:359] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.329947  7855 raft_consensus.cc:385] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.330291  7855 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.332062  7855 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.343714  7856 tablet_bootstrap.cc:654] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.330838  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:47.355044  7777 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:47.356316  7969 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:47.357018  7969 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:47.358174  7969 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.360993  7559 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.361083  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.364483  7902 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:47.365710  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45818.9us
W20260709 12:24:47.366439  7777 leader_election.cc:343] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:47.367931  7702 leader_election.cc:343] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.372236  7855 ts_tablet_manager.cc:1434] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.099s	user 0.010s	sys 0.008s
I20260709 12:24:47.373322  7855 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:47.374630  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.380710  7857 tablet_bootstrap.cc:654] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.390507  7863 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:47.393514  7627 leader_election.cc:343] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.386716  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.402153  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:47.417397  7863 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.418236  7863 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.418748  7863 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.418972  7969 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.421200  7863 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.428294  7559 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.414486  7936 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.426611  7902 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:24:47.437239  7856 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:47.438159  7856 ts_tablet_manager.cc:1403] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.219s	user 0.022s	sys 0.006s
I20260709 12:24:47.443677  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:47.445214  7902 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.445453  7736 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:47.444422  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.445869  7902 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:47.446887  7902 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.448571  7945 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.455598  7702 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:47.453207  7936 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.464412  7945 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.466172  7974 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.467415  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.468272  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:47.469097  7736 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.471115  7856 raft_consensus.cc:359] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.472239  7856 raft_consensus.cc:385] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.472553  7856 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.478843  7856 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:47.484659  7777 leader_election.cc:343] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.489259  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:47.491295  7976 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.491431  7856 ts_tablet_manager.cc:1434] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 12:24:47.492970  7856 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:47.496860  7857 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:47.497468  7857 ts_tablet_manager.cc:1403] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.288s	user 0.006s	sys 0.009s
W20260709 12:24:47.499753  7626 leader_election.cc:343] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.503398  7856 tablet_bootstrap.cc:654] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.510990  7855 tablet_bootstrap.cc:654] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.516770  7857 raft_consensus.cc:359] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.517452  7857 raft_consensus.cc:385] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.517767  7857 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.519209  7857 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.512005  7736 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:47.519575  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.519531  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.521909  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:47.522718  7857 ts_tablet_manager.cc:1434] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
W20260709 12:24:47.523517  7627 leader_election.cc:343] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.523831  7857 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.526039  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:24:47.526312  7702 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.529558  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:47.529841  7895 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.528226  7945 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.531162  7945 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:47.532167  7945 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.533942  7559 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.577524  7857 tablet_bootstrap.cc:654] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:47.543982  7702 leader_election.cc:343] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:47.582893  7976 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.583849  7856 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:47.584554  7856 ts_tablet_manager.cc:1403] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.092s	user 0.026s	sys 0.027s
I20260709 12:24:47.587947  7856 raft_consensus.cc:359] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.588792  7856 raft_consensus.cc:385] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.589093  7856 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.589947  7856 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.590181  7973 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:47.592517  7932 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.593926  7932 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.596623  7932 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.601357  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.603193  7856 ts_tablet_manager.cc:1434] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:24:47.604795  7856 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:47.601608  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.609675  7735 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:47.628708  7778 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.630357  7932 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:47.630818  7932 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.613626  7661 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:47.631170  7932 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.646775  7856 tablet_bootstrap.cc:654] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.652630  7559 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.658252  7857 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:47.659142  7857 ts_tablet_manager.cc:1403] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.136s	user 0.018s	sys 0.023s
I20260709 12:24:47.665777  7936 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.674559  7662 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:47.679575  7932 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.679582  7857 raft_consensus.cc:359] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.680632  7857 raft_consensus.cc:385] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.680974  7857 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.681710  7857 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.682152  7932 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.683502  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:47.684207  7735 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.685041  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:47.686132  7936 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.688159  7936 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.694526  7976 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.695307  7976 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.698698  7976 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.687111  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:47.700767  7662 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.702162  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.703327  7808 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:47.703794  7857 ts_tablet_manager.cc:1434] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:24:47.705132  7857 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.692510  7936 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:47.707463  7936 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.709049  7855 tablet_bootstrap.cc:492] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:47.709702  7855 ts_tablet_manager.cc:1403] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.337s	user 0.026s	sys 0.025s
I20260709 12:24:47.711642  7735 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:47.715019  7778 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:47.716315  7945 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:47.716876  7945 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:47.717777  7945 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.718324  7855 raft_consensus.cc:359] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.715101  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.719019  7855 raft_consensus.cc:385] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.720948  7855 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.722136  7855 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.737387  7627 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:47.738534  7976 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.739210  7943 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:47.742442  7855 ts_tablet_manager.cc:1434] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 12:24:47.743662  7855 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:47.760357  7976 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.770504  7857 tablet_bootstrap.cc:654] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.769963  7557 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.795090  7662 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:47.805543  7855 tablet_bootstrap.cc:654] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.822796  7856 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:47.823385  7856 ts_tablet_manager.cc:1403] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.219s	user 0.025s	sys 0.019s
I20260709 12:24:47.826795  7856 raft_consensus.cc:359] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.827527  7856 raft_consensus.cc:385] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.827826  7856 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.828580  7856 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.833680  7857 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:47.834513  7857 ts_tablet_manager.cc:1403] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.007s
I20260709 12:24:47.836925  7558 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.840433  7857 raft_consensus.cc:359] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.841316  7857 raft_consensus.cc:385] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:47.841858  7857 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.842218  7856 ts_tablet_manager.cc:1434] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:24:47.842644  7857 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.843326  7856 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:47.844987  7857 ts_tablet_manager.cc:1434] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:47.846009  7857 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:47.853323  7857 tablet_bootstrap.cc:654] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.853245  7856 tablet_bootstrap.cc:654] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:47.893757  8027 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.907738  7661 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:47.910995  7932 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.911890  7932 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.913111  7945 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.917984  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:47.918877  7932 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:47.920027  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.921367  7661 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:47.926783  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:47.933058  7932 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.951329  7661 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:47.952981  7932 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:47.955996  7902 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.956954  7902 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:47.960048  7902 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:47.979038  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:47.989017  7659 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:47.980747  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:47.987762  7735 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:47.994278  7945 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:48.044183  7855 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:48.044756  7855 ts_tablet_manager.cc:1403] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.301s	user 0.090s	sys 0.096s
I20260709 12:24:48.048529  7855 raft_consensus.cc:359] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.049542  7855 raft_consensus.cc:385] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.049875  7855 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.051095  7855 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.054234  7855 ts_tablet_manager.cc:1434] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:24:48.055547  7855 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:48.058717  7863 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.059486  7863 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.062497  7855 tablet_bootstrap.cc:654] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.072779  7855 tablet_bootstrap.cc:492] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:48.073426  7855 ts_tablet_manager.cc:1403] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 12:24:48.076334  7855 raft_consensus.cc:359] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.077212  7855 raft_consensus.cc:385] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.077529  7855 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.078293  7855 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.080713  7855 ts_tablet_manager.cc:1434] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:24:48.081676  8026 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.082089  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:48.082389  8026 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.084650  7855 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:48.085429  8027 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.086227  8027 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.086702  8027 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.086879  8026 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.093096  7855 tablet_bootstrap.cc:654] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.110759  7932 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:24:48.138238  7945 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:48.140332  7902 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.146759  7863 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.149221  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:48.171648  7702 leader_election.cc:343] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.179978  7932 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.180864  7932 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.201943  7997 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 12:24:48.258582  7856 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:48.259312  7856 ts_tablet_manager.cc:1403] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.416s	user 0.097s	sys 0.109s
I20260709 12:24:48.140935  7902 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.267026  7902 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.268582  7855 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:48.269145  7855 ts_tablet_manager.cc:1403] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.185s	user 0.051s	sys 0.059s
I20260709 12:24:48.274019  7932 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:48.277237  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.280297  8027 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:48.246865  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 75251.2us
I20260709 12:24:48.288852  7855 raft_consensus.cc:359] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.290000  7855 raft_consensus.cc:385] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.290411  7855 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.291527  7855 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.295955  7855 ts_tablet_manager.cc:1434] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 12:24:48.298331  7855 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:48.302125  7932 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:48.267076  7856 raft_consensus.cc:359] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.309463  7857 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:48.312526  7856 raft_consensus.cc:385] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.312916  7857 ts_tablet_manager.cc:1403] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.467s	user 0.123s	sys 0.113s
I20260709 12:24:48.312916  7856 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.315249  7856 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.313635  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.317616  7856 ts_tablet_manager.cc:1434] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.003s
I20260709 12:24:48.317560  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.318543  7661 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:48.319211  7856 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:48.320770  7855 tablet_bootstrap.cc:654] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.331054  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.341568  7863 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.346160  7863 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.339799  8027 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.343066  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:48.368752  7945 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.375550  7857 raft_consensus.cc:359] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.376912  7857 raft_consensus.cc:385] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.377290  7857 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.378134  7857 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.371408  7969 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:48.383404  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.388984  7863 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.393868  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.376739  7945 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.374635  8040 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.416730  7945 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:48.418160  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.424407  7856 tablet_bootstrap.cc:654] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.403496  8040 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.427811  7812 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:48.430110  8040 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:48.433025  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.440990  7735 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:48.441632  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.447033  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.447850  7737 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:48.425180  7857 ts_tablet_manager.cc:1434] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.112s	user 0.010s	sys 0.000s
I20260709 12:24:48.451210  7857 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:48.455319  7778 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:48.456269  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.419170  7662 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:48.423048  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:48.462471  7659 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:48.459664  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.465505  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.473528  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:48.475191  7945 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.475840  7945 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.476505  7945 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.467173  8040 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.479606  7734 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:48.480747  7658 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:48.478268  8040 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.483520  8040 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:48.493968  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:48.502043  7627 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:48.502303  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:48.504978  8043 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.505738  7627 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.506950  7976 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.507440  7976 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.507838  7976 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.511437  7855 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:48.513497  7855 ts_tablet_manager.cc:1403] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.215s	user 0.012s	sys 0.014s
I20260709 12:24:48.518357  7976 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.525283  7863 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.526037  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:48.526186  7863 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.527819  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:48.529244  7902 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.529472  7863 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.529855  7902 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.530257  7902 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.531728  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.532783  7945 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:48.533941  7626 leader_election.cc:343] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.535050  7812 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:48.524199  7855 raft_consensus.cc:359] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.540117  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:48.540391  8027 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:48.540963  7661 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.541836  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:48.540525  7855 raft_consensus.cc:385] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.542666  7855 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.543248  7856 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:48.540861  7945 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:48.543879  7856 ts_tablet_manager.cc:1403] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.225s	user 0.020s	sys 0.025s
I20260709 12:24:48.544438  7855 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.548528  8026 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.513357  8040 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.549170  8026 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.550241  7809 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:48.545962  7857 tablet_bootstrap.cc:654] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.551952  8026 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.553215  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:48.554288  7658 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:48.542908  7734 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:48.556739  8040 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:48.558164  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:48.556921  7855 ts_tablet_manager.cc:1434] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:24:48.558931  7734 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.559345  7976 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.562932  7855 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:48.564361  7661 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:48.562485  7856 raft_consensus.cc:359] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.565119  7856 raft_consensus.cc:385] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.565441  7856 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.566648  7856 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.578356  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:48.584038  7856 ts_tablet_manager.cc:1434] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 12:24:48.585300  7856 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:48.585467  7902 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.583197  7945 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:48.581533  7702 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:48.591606  7902 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.592792  7973 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:48.595031  7702 leader_election.cc:343] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.595958  7973 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:48.589200  7945 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:48.598568  7945 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.599933  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.600780  7973 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:48.603964  7734 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:48.604866  8026 raft_consensus.cc:2764] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:24:48.609129  7778 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:48.610841  7969 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:48.611501  7969 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:48.612879  7969 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.628682  7857 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:48.630041  7857 ts_tablet_manager.cc:1403] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.179s	user 0.019s	sys 0.009s
I20260709 12:24:48.632023  7812 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.634161  7857 raft_consensus.cc:359] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.635456  7857 raft_consensus.cc:385] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.635814  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.638546  7857 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.639875  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.640945  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:48.641599  7659 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.640674  7857 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.638789  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:48.639444  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:48.646147  7856 tablet_bootstrap.cc:654] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.646767  7734 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:48.644716  7857 ts_tablet_manager.cc:1434] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:24:48.645520  7855 tablet_bootstrap.cc:654] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.650319  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:48.650627  7658 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:48.640669  7737 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:48.661653  7627 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:24:48.664083  7702 leader_election.cc:343] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.663983  7976 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:48.651207  7808 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.670372  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.673699  7857 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:48.678601  8026 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.688261  8026 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.688791  8026 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.685436  7627 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.713811  8043 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:48.714409  8043 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.714792  8043 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.714766  7559 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.728024  7559 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.721982  7659 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:48.742606  7857 tablet_bootstrap.cc:654] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.749802  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:48.750916  8027 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:48.751559  8027 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:48.752651  8027 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.765756  7808 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:48.773212  7856 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:48.774813  7856 ts_tablet_manager.cc:1403] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.190s	user 0.021s	sys 0.029s
I20260709 12:24:48.780865  8026 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.781989  8043 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.791124  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:48.783852  8026 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.785055  8043 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.894625  7856 raft_consensus.cc:359] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:48.797442  7626 leader_election.cc:343] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:48.913182  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 115806us
I20260709 12:24:48.793123  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:48.793118  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:48.899712  7856 raft_consensus.cc:385] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.918292  7857 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:48.918370  7856 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.918815  7857 ts_tablet_manager.cc:1403] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.245s	user 0.024s	sys 0.023s
I20260709 12:24:48.919169  7856 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:48.922271  7702 leader_election.cc:343] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.928013  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:48.928898  7659 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.928249  7855 tablet_bootstrap.cc:492] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:48.923833  7734 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:48.931763  7855 ts_tablet_manager.cc:1403] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.372s	user 0.024s	sys 0.021s
I20260709 12:24:48.932799  7857 raft_consensus.cc:359] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.934213  7857 raft_consensus.cc:385] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.934645  7857 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.935652  7857 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.938894  7857 ts_tablet_manager.cc:1434] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 12:24:48.940474  7857 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:48.942085  7856 ts_tablet_manager.cc:1434] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.167s	user 0.010s	sys 0.004s
I20260709 12:24:48.943411  7856 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:48.951882  7855 raft_consensus.cc:359] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.953037  7855 raft_consensus.cc:385] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:48.953115  7863 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.953336  7659 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:48.953477  7855 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.953850  7863 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.954510  7855 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.957927  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:48.958184  7855 ts_tablet_manager.cc:1434] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:24:48.958434  7863 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:48.959764  7855 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
W20260709 12:24:48.960909  7627 leader_election.cc:343] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.964866  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:48.965272  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.966962  8027 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:48.967962  8027 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:24:48.969165  7626 leader_election.cc:343] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:48.970011  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:48.970997  8043 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:48.969123  8027 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.982239  7734 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:48.982616  7856 tablet_bootstrap.cc:654] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:48.985749  7627 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:48.987597  7863 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:48.988230  7863 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:48.989455  7863 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:48.991416  7857 tablet_bootstrap.cc:654] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.007952  7557 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.018114  7969 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.018998  7855 tablet_bootstrap.cc:654] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.019083  7857 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:49.019893  7857 ts_tablet_manager.cc:1403] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.080s	user 0.004s	sys 0.014s
I20260709 12:24:49.022820  7857 raft_consensus.cc:359] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.023561  7857 raft_consensus.cc:385] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.023885  7857 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.025027  7857 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.034642  7856 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:49.035339  7856 ts_tablet_manager.cc:1403] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.007s
I20260709 12:24:49.041596  7558 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.042636  7855 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:49.042420  7856 raft_consensus.cc:359] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.043179  7855 ts_tablet_manager.cc:1403] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.006s
I20260709 12:24:49.043289  7856 raft_consensus.cc:385] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.043706  7856 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.041636  7857 ts_tablet_manager.cc:1434] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 12:24:49.045378  7857 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:49.046422  7855 raft_consensus.cc:359] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.047164  7855 raft_consensus.cc:385] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.059974  7856 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.060818  7855 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.062008  7855 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.066833  7855 ts_tablet_manager.cc:1434] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:24:49.068115  7855 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:49.071342  7969 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.078146  7969 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.080761  7856 ts_tablet_manager.cc:1434] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.002s
I20260709 12:24:49.082261  7856 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:49.078748  7969 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.089937  8040 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.090600  8040 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.091116  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:49.091722  7969 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:49.091988  7734 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:49.092362  7969 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:49.093556  8040 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:49.093861  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:49.095325  8040 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:49.095683  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:49.095762  8040 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.099402  7857 tablet_bootstrap.cc:654] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.098357  8040 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.110895  7969 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.106562  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.113776  7661 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:49.107391  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.137703  7659 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:49.143991  7856 tablet_bootstrap.cc:654] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.164948  7855 tablet_bootstrap.cc:654] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:49.153157  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46867.9us
I20260709 12:24:49.174178  8040 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.177094  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:49.178355  7945 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.184614  8040 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:49.187958  8026 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:49.200331  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:49.201152  7734 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.208215  7945 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:49.228498  7734 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:49.233309  7863 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.232817  8026 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.234130  7863 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.246840  7855 tablet_bootstrap.cc:492] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:49.249195  7855 ts_tablet_manager.cc:1403] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.011s
I20260709 12:24:49.232842  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:49.246965  7863 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.252610  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:49.255113  7662 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.244110  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:49.256851  8071 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:49.257614  8071 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:49.258512  8071 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.242755  8026 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.261583  8026 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.264904  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:49.264744  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.266043  7661 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:49.266382  7855 raft_consensus.cc:359] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:49.266995  7702 leader_election.cc:343] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.267726  7855 raft_consensus.cc:385] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.268085  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:49.268410  7855 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.270275  7855 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.275732  7855 ts_tablet_manager.cc:1434] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.005s
I20260709 12:24:49.279479  7855 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:49.284289  8026 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:49.284258  7812 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:49.295898  8026 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.295709  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:49.301648  7856 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:49.326333  8026 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.326956  7856 ts_tablet_manager.cc:1403] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.245s	user 0.029s	sys 0.022s
I20260709 12:24:49.339670  7662 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
W20260709 12:24:49.307996  8078 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:24:49.354914  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:49.356196  7863 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:49.356642  7863 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.356993  7863 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.357004  7855 tablet_bootstrap.cc:654] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.364979  7557 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:49.366043  7863 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.377841  7857 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:49.378635  7857 ts_tablet_manager.cc:1403] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.333s	user 0.032s	sys 0.028s
I20260709 12:24:49.381745  7857 raft_consensus.cc:359] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.382890  7857 raft_consensus.cc:385] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.383323  7857 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.389796  7856 raft_consensus.cc:359] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.390666  7856 raft_consensus.cc:385] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.390970  7856 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.391500  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:49.388633  7943 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:49.393956  7856 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.394325  7857 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.396605  7856 ts_tablet_manager.cc:1434] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 12:24:49.397877  7856 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:49.407101  7855 tablet_bootstrap.cc:492] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:49.408281  7943 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:49.409633  7855 ts_tablet_manager.cc:1403] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.002s
I20260709 12:24:49.413000  7855 raft_consensus.cc:359] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.413803  7855 raft_consensus.cc:385] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.414083  7855 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.415512  7855 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.416697  8026 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.418509  7855 ts_tablet_manager.cc:1434] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:49.427862  7734 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:49.428128  7855 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:49.421001  7856 tablet_bootstrap.cc:654] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.421381  7863 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.422058  7857 ts_tablet_manager.cc:1434] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 12:24:49.424780  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:49.434939  7857 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:49.441201  7810 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.447520  8026 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.466671  7810 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:49.468649  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:49.469401  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:49.470374  7659 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:49.471295  7895 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:49.472203  7895 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:49.473390  7895 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:49.480161  7626 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.480654  7557 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.487407  8026 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:49.486850  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:49.493302  7812 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:49.500445  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:49.501812  8026 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:49.507895  7857 tablet_bootstrap.cc:654] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.512604  7856 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:49.514246  7856 ts_tablet_manager.cc:1403] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.117s	user 0.030s	sys 0.029s
I20260709 12:24:49.516382  7855 tablet_bootstrap.cc:654] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.518612  7559 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.522531  8026 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.523145  8026 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.523376  7856 raft_consensus.cc:359] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.524206  7856 raft_consensus.cc:385] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.524497  7856 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.525276  7856 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.527925  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.528271  7856 ts_tablet_manager.cc:1434] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:24:49.537736  7856 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:49.542595  8026 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.538322  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:49.544250  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:49.544451  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:49.545173  7812 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:49.545755  8026 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:49.546927  8026 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.548130  8026 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.570277  7856 tablet_bootstrap.cc:654] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.574958  7973 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.575840  7973 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.578322  7973 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.580433  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.583086  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:49.583379  7702 leader_election.cc:343] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.583525  8026 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:49.585095  7702 leader_election.cc:343] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.585559  7702 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:49.586046  8026 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.589037  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:49.589099  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:49.589767  7812 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.590152  7662 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.595449  7973 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:49.601243  7662 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:49.601918  7812 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:49.602917  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:49.604324  7973 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:49.604950  7973 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:49.606004  7973 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.610044  7856 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:49.610713  7856 ts_tablet_manager.cc:1403] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.016s
I20260709 12:24:49.620890  7895 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:24:49.644712  7856 raft_consensus.cc:359] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.645542  7856 raft_consensus.cc:385] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.645872  7856 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.647019  7856 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.650835  7856 ts_tablet_manager.cc:1434] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.006s
I20260709 12:24:49.652112  7856 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
W20260709 12:24:49.629132  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:49.639549  7855 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:49.715075  7855 ts_tablet_manager.cc:1403] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.287s	user 0.013s	sys 0.020s
I20260709 12:24:49.617501  7857 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:49.711866  7558 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.736140  7857 ts_tablet_manager.cc:1403] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.301s	user 0.012s	sys 0.005s
I20260709 12:24:49.739084  7855 raft_consensus.cc:359] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.741406  7855 raft_consensus.cc:385] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.741863  7855 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.742924  7855 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.777405  7857 raft_consensus.cc:359] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.779662  7857 raft_consensus.cc:385] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.780104  7857 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.787573  7855 ts_tablet_manager.cc:1434] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.010s
I20260709 12:24:49.793042  7857 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.801159  7976 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.802289  7976 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.811779  7855 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:49.813426  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:49.816111  7626 leader_election.cc:343] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.817093  7976 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.819545  7857 ts_tablet_manager.cc:1434] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.083s	user 0.013s	sys 0.003s
I20260709 12:24:49.820552  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:49.824621  7945 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.829691  7857 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:49.824705  7856 tablet_bootstrap.cc:654] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:49.839066  7627 leader_election.cc:343] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.840161  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:49.842514  8094 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:49.844127  8026 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.844774  8026 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.848466  8026 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.856696  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:49.859020  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:49.861536  7702 leader_election.cc:343] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:49.862722  7702 leader_election.cc:343] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:49.863394  7702 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:49.865010  8026 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:49.874662  7857 tablet_bootstrap.cc:654] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.897964  7855 tablet_bootstrap.cc:654] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:49.900272  8094 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.901202  8096 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:49.970738  7857 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:49.971487  7857 ts_tablet_manager.cc:1403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.142s	user 0.029s	sys 0.011s
I20260709 12:24:49.987068  7857 raft_consensus.cc:359] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.987972  7857 raft_consensus.cc:385] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:49.988312  7857 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.989459  7863 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.989144  7857 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.989960  7863 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:49.991056  7857 ts_tablet_manager.cc:1434] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 12:24:49.992380  7863 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:49.993299  7857 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:49.991015  7809 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:49.997502  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.000505  8094 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:50.004228  7855 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.004897  7855 ts_tablet_manager.cc:1403] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.194s	user 0.020s	sys 0.014s
I20260709 12:24:50.005834  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:50.004794  7626 leader_election.cc:343] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.009328  7855 raft_consensus.cc:359] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.011004  7855 raft_consensus.cc:385] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.011546  7855 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.012465  7855 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.013782  7856 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.014523  7856 ts_tablet_manager.cc:1403] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.363s	user 0.021s	sys 0.013s
I20260709 12:24:50.015237  7855 ts_tablet_manager.cc:1434] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:50.016641  7855 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
W20260709 12:24:50.016958  7627 leader_election.cc:343] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.017616  7627 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:50.016780  7857 tablet_bootstrap.cc:654] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.022773  8094 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:50.031121  7855 tablet_bootstrap.cc:654] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.050391  7856 raft_consensus.cc:359] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.055972  7856 raft_consensus.cc:385] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.056649  7856 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.068789  7856 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.081249  7857 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.081933  7857 ts_tablet_manager.cc:1403] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.089s	user 0.021s	sys 0.003s
I20260709 12:24:50.080947  7973 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:50.084998  7857 raft_consensus.cc:359] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.085819  7857 raft_consensus.cc:385] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.086113  7857 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.088864  8096 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:24:50.089463  8096 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.091068  7857 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.092818  7855 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.093366  7855 ts_tablet_manager.cc:1403] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.077s	user 0.025s	sys 0.008s
I20260709 12:24:50.093753  7857 ts_tablet_manager.cc:1434] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:24:50.095013  7857 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.096454  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.096521  7855 raft_consensus.cc:359] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.097406  7855 raft_consensus.cc:385] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.097729  7855 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.102296  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.105890  7857 tablet_bootstrap.cc:654] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.107487  7778 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:50.108997  8099 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.098798  7855 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.113734  7856 ts_tablet_manager.cc:1434] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.099s	user 0.004s	sys 0.007s
I20260709 12:24:50.118670  7856 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.112589  7855 ts_tablet_manager.cc:1434] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:24:50.103358  8096 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.125334  7855 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.133944  7857 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.134528  7857 ts_tablet_manager.cc:1403] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.012s
I20260709 12:24:50.138197  7857 raft_consensus.cc:359] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.139506  7857 raft_consensus.cc:385] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.139860  7857 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.140601  7857 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.143769  7857 ts_tablet_manager.cc:1434] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:24:50.145187  7857 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.148729  7856 tablet_bootstrap.cc:654] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.152468  7857 tablet_bootstrap.cc:654] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.156956  7973 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:50.154157  7855 tablet_bootstrap.cc:654] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.159802  7976 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.160485  7976 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.167805  7976 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.168852  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.170192  7809 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:50.171939  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.173557  7976 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.174274  7976 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.174767  7976 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.182230  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:50.184405  7627 leader_election.cc:343] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.215415  7976 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.220132  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:50.221189  7809 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.227098  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.245709  7809 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:50.247326  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.248808  8094 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.249560  8094 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:24:50.248826  7627 leader_election.cc:343] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.250579  8094 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.255254  7857 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.256278  7857 ts_tablet_manager.cc:1403] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.111s	user 0.036s	sys 0.010s
I20260709 12:24:50.259552  7976 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.259778  7857 raft_consensus.cc:359] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.260529  7857 raft_consensus.cc:385] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.260839  7857 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.264245  7857 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.269682  7857 ts_tablet_manager.cc:1434] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:24:50.271056  7857 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.274533  7855 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.275743  7855 ts_tablet_manager.cc:1403] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.151s	user 0.026s	sys 0.025s
I20260709 12:24:50.281082  7857 tablet_bootstrap.cc:654] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.292919  7856 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.293530  7856 ts_tablet_manager.cc:1403] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.175s	user 0.050s	sys 0.088s
I20260709 12:24:50.297221  7856 raft_consensus.cc:359] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.294970  7855 raft_consensus.cc:359] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.298200  7856 raft_consensus.cc:385] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.298534  7855 raft_consensus.cc:385] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.298715  7856 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.298908  7855 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.298220  7557 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.299672  7856 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.302515  7856 ts_tablet_manager.cc:1434] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:50.305052  7856 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.299747  7855 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.312341  7855 ts_tablet_manager.cc:1434] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:24:50.313647  7855 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.320634  7855 tablet_bootstrap.cc:654] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.322222  7856 tablet_bootstrap.cc:654] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.343513  7855 tablet_bootstrap.cc:492] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.344550  7855 ts_tablet_manager.cc:1403] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.013s
I20260709 12:24:50.347128  7857 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.347815  7857 ts_tablet_manager.cc:1403] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.009s
I20260709 12:24:50.348606  7855 raft_consensus.cc:359] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.349766  7855 raft_consensus.cc:385] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.350064  7855 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.350713  7855 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.352254  7855 ts_tablet_manager.cc:1434] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:50.353372  7855 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.359789  7856 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.361182  7855 tablet_bootstrap.cc:654] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.361182  7856 ts_tablet_manager.cc:1403] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.004s
I20260709 12:24:50.367892  7856 raft_consensus.cc:359] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.368721  7856 raft_consensus.cc:385] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.369279  7856 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.368630  7857 raft_consensus.cc:359] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.370325  7857 raft_consensus.cc:385] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.370682  7857 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.371315  7856 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.374107  8096 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.375164  8096 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.381454  7856 ts_tablet_manager.cc:1434] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:24:50.379699  7857 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.382731  7856 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.386750  7857 ts_tablet_manager.cc:1434] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 12:24:50.388145  7857 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.392356  8096 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.394443  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.395296  7662 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.397015  7856 tablet_bootstrap.cc:654] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.396962  7777 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.400738  7857 tablet_bootstrap.cc:654] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.400575  8099 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.405539  8099 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.419744  8026 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.420492  8026 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.421288  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.426288  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.415892  8099 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.433104  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.427847  8094 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.434244  8094 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.436820  8094 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.438269  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.427099  8026 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.439602  7973 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.440183  7973 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.441643  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.433377  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.448583  7855 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
W20260709 12:24:50.444285  7627 leader_election.cc:343] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.449954  7856 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.452033  7856 ts_tablet_manager.cc:1403] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.070s	user 0.026s	sys 0.007s
I20260709 12:24:50.452829  7855 ts_tablet_manager.cc:1403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.012s
W20260709 12:24:50.455283  7626 leader_election.cc:343] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.460084  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.460558  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:50.453685  7661 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:50.460906  7662 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:50.465459  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.469625  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.468727  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.468217  8026 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.470820  7943 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.471048  8026 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.474324  7943 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.466109  8094 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:50.475451  7943 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.475453  8026 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.465363  7973 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.480957  7856 raft_consensus.cc:359] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.481863  7856 raft_consensus.cc:385] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.482235  7856 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.478948  8099 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.484543  7807 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:50.487643  7857 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.488243  7857 ts_tablet_manager.cc:1403] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.100s	user 0.020s	sys 0.010s
I20260709 12:24:50.490474  7856 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:50.497336  7702 leader_election.cc:343] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.500488  8099 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.501766  7856 ts_tablet_manager.cc:1434] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 12:24:50.503130  7856 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.507499  7857 raft_consensus.cc:359] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.508199  7855 raft_consensus.cc:359] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.510501  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.511684  7855 raft_consensus.cc:385] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.511958  7662 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.512049  7855 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.508988  7857 raft_consensus.cc:385] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.512567  7857 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.512861  7855 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.525311  7662 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.526659  7777 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.517191  7857 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.527712  8099 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.528980  8099 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:50.529119  7855 ts_tablet_manager.cc:1434] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.005s
I20260709 12:24:50.529762  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.530608  7855 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.530246  8099 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.535200  8122 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.535815  8122 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.538038  7943 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.539287  8122 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.539577  7856 tablet_bootstrap.cc:654] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.540782  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.541574  7661 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.542601  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.543422  7737 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.543994  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.545570  7943 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.546270  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.547009  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:50.547402  7661 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:50.549206  7702 leader_election.cc:343] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.545409  7855 tablet_bootstrap.cc:654] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.551591  8122 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.552156  8122 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.552563  8122 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.554823  8094 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.555470  8094 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.559010  8094 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.561138  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.562093  8094 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.562446  7737 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:50.562995  8094 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.561148  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.564832  7627 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.566025  8094 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.566612  7976 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:50.566428  7626 leader_election.cc:343] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.567306  7976 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.567764  7976 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.570005  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.572296  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:50.574505  7627 leader_election.cc:343] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.578445  7661 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:50.578325  7976 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.579742  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:50.539322  8026 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.580830  7943 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.581104  7733 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:50.581506  7943 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:50.582531  7943 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:50.583918  7626 leader_election.cc:343] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.584203  7976 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.584450  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:50.586323  7895 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:50.587296  8026 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.583163  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.588024  7733 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.589762  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:50.585318  7857 ts_tablet_manager.cc:1434] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.001s
I20260709 12:24:50.591545  7895 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.592119  7895 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:50.591727  7626 leader_election.cc:343] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:50.595082  7559 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.595063  7895 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.596220  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:50.605965  7809 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:50.598043  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.607846  7737 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:50.599427  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:50.595758  7857 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.610074  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.611505  7895 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.611941  7895 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.612370  7895 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.617107  7856 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.621193  7856 ts_tablet_manager.cc:1403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.118s	user 0.022s	sys 0.022s
I20260709 12:24:50.626338  7856 raft_consensus.cc:359] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.631094  7895 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.632052  7733 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:50.634865  7627 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:50.634951  7856 raft_consensus.cc:385] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.636482  7976 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.635962  7856 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.638928  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.639839  7661 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:50.638232  7856 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.642895  7856 ts_tablet_manager.cc:1434] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:24:50.658474  7557 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.660053  7856 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.642550  8122 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.644714  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:50.666936  7855 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.648921  7976 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:50.668609  7976 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.673333  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.674495  7734 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.675027  7943 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:50.675338  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.676283  7661 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.650911  7857 tablet_bootstrap.cc:654] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.679409  8122 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.680692  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.688257  7737 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.681231  7895 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:50.667692  7855 ts_tablet_manager.cc:1403] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.137s	user 0.013s	sys 0.014s
I20260709 12:24:50.700826  7734 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:50.682241  7856 tablet_bootstrap.cc:654] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.702505  7558 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.701363  7855 raft_consensus.cc:359] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.713197  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.713505  7855 raft_consensus.cc:385] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.714288  7855 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.714805  7895 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.715449  7895 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:50.707880  7857 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.716239  7857 ts_tablet_manager.cc:1403] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.125s	user 0.020s	sys 0.010s
I20260709 12:24:50.716394  7895 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.716552  7855 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.722316  7855 ts_tablet_manager.cc:1434] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:24:50.718046  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:50.725111  7807 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.722731  7661 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.732321  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.733929  7945 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.734228  7855 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.734552  7945 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:50.723992  7857 raft_consensus.cc:359] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.735127  7857 raft_consensus.cc:385] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.743361  7857 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.735591  7945 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.744342  7857 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.746884  7857 ts_tablet_manager.cc:1434] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:24:50.711325  7737 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.750576  7857 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.746583  7559 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.747912  7856 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.742992  7855 tablet_bootstrap.cc:654] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.754374  7856 ts_tablet_manager.cc:1403] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.098s	user 0.020s	sys 0.004s
I20260709 12:24:50.760627  7807 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:50.765391  7857 tablet_bootstrap.cc:654] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:50.769560  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:50.771068  7856 raft_consensus.cc:359] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.771880  7856 raft_consensus.cc:385] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.772217  7856 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.773701  7856 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.792652  7855 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.793438  7855 ts_tablet_manager.cc:1403] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.059s	user 0.008s	sys 0.024s
I20260709 12:24:50.797097  7855 raft_consensus.cc:359] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.797896  7855 raft_consensus.cc:385] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.798187  7855 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.799952  7856 ts_tablet_manager.cc:1434] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.007s
I20260709 12:24:50.801175  7856 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.807332  7737 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.808574  7856 tablet_bootstrap.cc:654] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.798893  7855 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.815061  7855 ts_tablet_manager.cc:1434] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:24:50.816691  7855 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.818086  7559 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.833017  7856 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.833614  7856 ts_tablet_manager.cc:1403] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.000s
I20260709 12:24:50.837169  7895 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:50.838109  7856 raft_consensus.cc:359] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.839078  7856 raft_consensus.cc:385] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.839437  7856 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.843910  7856 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.848240  7856 ts_tablet_manager.cc:1434] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:24:50.849454  7856 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:50.851029  8122 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.852167  8122 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.845638  7855 tablet_bootstrap.cc:654] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.854605  7857 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:50.855489  7857 ts_tablet_manager.cc:1403] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.105s	user 0.036s	sys 0.027s
I20260709 12:24:50.857165  8122 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.861099  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.862003  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.864005  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.867247  7857 raft_consensus.cc:359] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.868341  7857 raft_consensus.cc:385] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.868722  7857 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.873088  7976 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:50.866829  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:50.866681  7734 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.873591  7856 tablet_bootstrap.cc:654] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:50.877378  7857 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.880656  8122 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.881430  8122 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.881934  8122 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.882421  7857 ts_tablet_manager.cc:1434] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:24:50.904721  7857 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:50.906737  7945 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.920697  7945 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.901753  8122 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.924827  7855 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:50.926118  7855 ts_tablet_manager.cc:1403] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.110s	user 0.041s	sys 0.026s
I20260709 12:24:50.927023  8122 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.929989  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.930248  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.930785  7733 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.930943  7662 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.941606  7733 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.943626  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.945127  8099 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:50.945087  7855 raft_consensus.cc:359] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.946245  8099 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:50.946826  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.947640  7734 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.947366  8099 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.949115  7945 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.950186  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.950860  7661 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:50.946219  7855 raft_consensus.cc:385] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:50.951453  7855 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.952431  7778 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:50.952212  7855 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.954082  8071 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:50.954522  8071 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.954885  8071 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.956907  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.957078  8122 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:50.960105  8122 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.960813  8122 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.963191  8122 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:50.964453  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:50.965421  7734 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:50.965574  8071 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.967275  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:50.968118  7855 ts_tablet_manager.cc:1434] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:24:50.972626  7856 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:50.975551  7856 ts_tablet_manager.cc:1403] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.008s
I20260709 12:24:50.974268  7855 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:50.983687  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:50.984701  8099 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.986059  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:50.986680  7661 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.987186  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:50.990200  7737 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:50.997584  7661 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:51.003137  7737 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:50.978854  7856 raft_consensus.cc:359] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:50.993213  8071 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:51.004483  7856 raft_consensus.cc:385] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.005829  7856 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.006737  7857 tablet_bootstrap.cc:654] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.005448  7778 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:51.017118  8122 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:51.018047  8122 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:51.019419  8122 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.017385  7855 tablet_bootstrap.cc:654] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.007997  7856 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.021987  8130 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.022578  8130 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.024909  8130 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.027055  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:51.001914  8099 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:51.029513  7626 leader_election.cc:343] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:51.058837  7856 ts_tablet_manager.cc:1434] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:24:51.040338  7558 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.038830  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:51.107074  7856 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:51.104717  7734 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:51.126183  7627 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:51.128093  8130 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:51.128748  8130 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.129153  8130 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.141891  7863 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.143736  8122 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.163044  7856 tablet_bootstrap.cc:654] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.164384  7857 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:51.165252  7857 ts_tablet_manager.cc:1403] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.261s	user 0.019s	sys 0.028s
I20260709 12:24:51.169401  7857 raft_consensus.cc:359] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.170296  7857 raft_consensus.cc:385] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.170619  7857 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.178284  7857 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.180706  7857 ts_tablet_manager.cc:1434] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 12:24:51.181833  7857 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:51.178269  7855 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:51.202508  7857 tablet_bootstrap.cc:654] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.202417  8094 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.205452  8094 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.198195  7855 ts_tablet_manager.cc:1403] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.224s	user 0.009s	sys 0.037s
I20260709 12:24:51.208531  8094 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.209631  7559 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.217398  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:51.218484  7810 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:51.216033  7973 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:51.245204  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:51.246362  7976 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:51.246788  7976 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.247170  7976 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.224815  7855 raft_consensus.cc:359] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.225603  7863 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:51.230224  8122 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.248328  8094 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.251526  7895 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.252183  7895 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:51.231707  8152 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:24:51.256570  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:51.265782  8130 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.255060  7895 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.270265  7855 raft_consensus.cc:385] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.270817  7855 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.271719  7855 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:51.276176  7702 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:51.278815  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:51.279652  7736 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
W20260709 12:24:51.281069  7626 leader_election.cc:343] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:51.281286  7855 ts_tablet_manager.cc:1434] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.004s
I20260709 12:24:51.277899  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:51.302186  7856 tablet_bootstrap.cc:492] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:51.302897  7856 ts_tablet_manager.cc:1403] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.201s	user 0.026s	sys 0.040s
I20260709 12:24:51.309882  7857 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:51.311286  7857 ts_tablet_manager.cc:1403] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.130s	user 0.021s	sys 0.001s
I20260709 12:24:51.312258  7855 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:51.319767  8130 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.344110  8099 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:51.344763  7855 tablet_bootstrap.cc:654] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.358160  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:51.358909  7976 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.359426  7857 raft_consensus.cc:359] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.360174  7857 raft_consensus.cc:385] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.360491  7857 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.361544  7976 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.361793  7856 raft_consensus.cc:359] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.362547  7856 raft_consensus.cc:385] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.362852  7856 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.363420  7857 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.364130  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:51.367118  7856 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.374985  7857 ts_tablet_manager.cc:1434] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:24:51.358935  7811 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.380081  7857 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
W20260709 12:24:51.366415  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72545.7us
W20260709 12:24:51.366101  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45750.4us
I20260709 12:24:51.372198  7809 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.386133  7856 ts_tablet_manager.cc:1434] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.000s
I20260709 12:24:51.401676  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:51.407322  7856 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
W20260709 12:24:51.427043  7627 leader_election.cc:343] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:51.427547  8157 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:51.420126  7809 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.432991  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:51.433833  7734 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.430841  8157 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.436671  8157 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.437304  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:51.430919  7811 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.439342  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:51.440137  7737 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.440515  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:51.444187  7855 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:51.444860  7855 ts_tablet_manager.cc:1403] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.143s	user 0.042s	sys 0.026s
I20260709 12:24:51.446267  8161 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:51.450580  7855 raft_consensus.cc:359] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.451529  7855 raft_consensus.cc:385] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.451817  7855 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.452610  7855 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.453722  7863 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:51.460741  7855 ts_tablet_manager.cc:1434] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 12:24:51.463558  7857 tablet_bootstrap.cc:654] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.469036  7855 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:51.490820  7945 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.492054  7737 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
W20260709 12:24:51.513521  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53880.4us
I20260709 12:24:51.513679  7863 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:51.516628  7863 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.520828  7856 tablet_bootstrap.cc:654] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.521938  8167 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260709 12:24:51.532105  8150 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.532737  8150 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.520897  8161 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:51.533387  8157 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.553625  8122 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.545967  8157 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.562780  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:51.566931  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:51.567823  7737 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:51.566622  7626 leader_election.cc:343] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:51.574218  8161 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.576922  7855 tablet_bootstrap.cc:654] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.607321  7857 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:51.608101  7857 ts_tablet_manager.cc:1403] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.228s	user 0.021s	sys 0.017s
I20260709 12:24:51.604987  8150 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:51.612596  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:51.630163  7737 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.625324  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:51.644897  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:51.668108  8099 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:51.703843  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:51.713178  7855 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:51.713207  7734 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.713845  7855 ts_tablet_manager.cc:1403] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.252s	user 0.011s	sys 0.014s
I20260709 12:24:51.714882  7857 raft_consensus.cc:359] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.715682  7857 raft_consensus.cc:385] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.716081  7857 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.727612  7857 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.707289  8161 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:51.748240  7857 ts_tablet_manager.cc:1434] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.140s	user 0.024s	sys 0.013s
I20260709 12:24:51.741220  8161 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:51.747813  7558 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.753394  7558 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.758798  8161 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.765098  7807 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.767712  7976 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.768507  7976 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.769307  7856 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:51.772572  7856 ts_tablet_manager.cc:1403] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.366s	user 0.027s	sys 0.027s
I20260709 12:24:51.774366  8177 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.781347  7857 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:51.782814  7976 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.782776  8178 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:51.786350  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:51.787500  7856 raft_consensus.cc:359] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.788460  7856 raft_consensus.cc:385] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.788800  7856 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.790064  7856 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:51.780215  8174 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:24:51.798345  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:51.799052  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:51.808668  7855 raft_consensus.cc:359] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.809235  7856 ts_tablet_manager.cc:1434] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 12:24:51.809508  7855 raft_consensus.cc:385] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.809900  7855 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.810670  7855 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.800989  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:51.811770  7856 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:51.812625  8161 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:51.813067  8161 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.813441  8161 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.813699  7809 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:51.821316  7855 ts_tablet_manager.cc:1434] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.004s
W20260709 12:24:51.833832  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:51.841343  7855 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:51.842120  7856 tablet_bootstrap.cc:654] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.858847  8161 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.874334  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:51.874981  7809 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.887528  7809 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.889881  7626 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:51.892365  8094 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:51.895514  8094 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:51.889341  8161 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:51.897455  8094 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.919256  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:51.920238  7731 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:51.928261  7857 tablet_bootstrap.cc:654] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.932385  7855 tablet_bootstrap.cc:654] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.948508  7856 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:51.949280  7856 ts_tablet_manager.cc:1403] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.138s	user 0.022s	sys 0.005s
I20260709 12:24:51.952294  7856 raft_consensus.cc:359] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.956620  7856 raft_consensus.cc:385] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.957213  7856 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.958667  7731 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:51.961573  7856 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.968416  7856 ts_tablet_manager.cc:1434] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:24:51.972026  7856 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:51.978016  7857 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:51.978595  7857 ts_tablet_manager.cc:1403] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.221s	user 0.014s	sys 0.012s
I20260709 12:24:51.981370  7856 tablet_bootstrap.cc:654] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:51.982254  8094 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:51.996728  7857 raft_consensus.cc:359] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:51.997625  7857 raft_consensus.cc:385] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:51.997942  7857 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.003403  7856 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.004201  7856 ts_tablet_manager.cc:1403] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.011s
I20260709 12:24:52.013408  7856 raft_consensus.cc:359] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.014601  7856 raft_consensus.cc:385] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.015054  7856 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.999708  7857 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.016510  7856 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.027405  7857 ts_tablet_manager.cc:1434] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 12:24:52.028673  7857 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:52.032213  7856 ts_tablet_manager.cc:1434] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.028s	user 0.013s	sys 0.000s
I20260709 12:24:52.037839  7856 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:52.037547  7556 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:52.051693  7855 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:52.052350  7855 ts_tablet_manager.cc:1403] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.218s	user 0.019s	sys 0.037s
I20260709 12:24:52.054020  7556 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:52.055178  7857 tablet_bootstrap.cc:654] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.081975  7855 raft_consensus.cc:359] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.082898  7855 raft_consensus.cc:385] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.083302  7855 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.084128  7855 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.086879  7856 tablet_bootstrap.cc:654] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.093114  7855 ts_tablet_manager.cc:1434] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.011s
I20260709 12:24:52.095641  8094 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.105024  7855 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:52.104882  8169 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:52.110764  8094 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.117774  7855 tablet_bootstrap.cc:654] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.151805  8168 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.152532  8168 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.154314  7857 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:52.154908  7857 ts_tablet_manager.cc:1403] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.126s	user 0.007s	sys 0.018s
I20260709 12:24:52.156184  7973 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.156728  7973 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.157300  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.158969  7973 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:52.159312  7626 leader_election.cc:343] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.159946  8168 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.174464  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.177708  7659 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:52.183354  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.184437  7734 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
W20260709 12:24:52.185177  7626 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:52.185822  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.196069  7857 raft_consensus.cc:359] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.197024  7857 raft_consensus.cc:385] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.197312  7857 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.198058  7857 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.202668  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:52.203754  7855 tablet_bootstrap.cc:492] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:52.204346  7855 ts_tablet_manager.cc:1403] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.100s	user 0.012s	sys 0.018s
I20260709 12:24:52.204685  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:24:52.205660  7702 leader_election.cc:343] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.208456  8161 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.209829  8161 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.207736  8026 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.210348  8161 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.210386  8026 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.210934  8026 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.220968  7857 ts_tablet_manager.cc:1434] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.066s	user 0.011s	sys 0.000s
I20260709 12:24:52.222311  7857 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:52.232581  7855 raft_consensus.cc:359] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.238507  7857 tablet_bootstrap.cc:654] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.233387  7855 raft_consensus.cc:385] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.239768  7855 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.240836  7855 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.234781  8161 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.248006  8026 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.252169  7855 ts_tablet_manager.cc:1434] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.003s
I20260709 12:24:52.254386  8026 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.257402  7856 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.258495  7856 ts_tablet_manager.cc:1403] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.221s	user 0.020s	sys 0.047s
I20260709 12:24:52.263754  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:52.260785  8161 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:52.266189  7626 leader_election.cc:343] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.266379  7856 raft_consensus.cc:359] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.267148  7856 raft_consensus.cc:385] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.267498  7856 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.269008  7855 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:52.269295  7856 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.271147  7856 ts_tablet_manager.cc:1434] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:24:52.272357  7856 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:52.282387  7856 tablet_bootstrap.cc:654] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.286034  7857 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:52.286654  7857 ts_tablet_manager.cc:1403] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.007s
I20260709 12:24:52.289675  7857 raft_consensus.cc:359] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.290462  7857 raft_consensus.cc:385] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.290741  7857 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.295066  8099 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.295687  8099 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.298108  8099 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:52.297940  7857 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.301704  7857 ts_tablet_manager.cc:1434] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:24:52.301411  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.302335  7737 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:52.304466  7855 tablet_bootstrap.cc:654] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.307796  7857 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:52.308019  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.309468  7659 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:52.307799  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:52.315822  8122 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.316527  8122 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.316990  8122 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.322176  7857 tablet_bootstrap.cc:654] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:52.338332  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 119495us
I20260709 12:24:52.343178  7857 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:52.343776  7857 ts_tablet_manager.cc:1403] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.002s
I20260709 12:24:52.348806  7857 raft_consensus.cc:359] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.349591  7857 raft_consensus.cc:385] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.349897  7857 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.350181  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:52.351156  7659 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:52.350678  7857 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.353467  7857 ts_tablet_manager.cc:1434] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:52.354993  7857 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:52.358157  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:52.362291  7857 tablet_bootstrap.cc:654] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.374967  7857 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:52.376433  7857 ts_tablet_manager.cc:1403] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:24:52.378371  8122 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.379736  7857 raft_consensus.cc:359] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.380297  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:52.380618  7857 raft_consensus.cc:385] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.381065  7857 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.381357  7734 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:52.381781  7856 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.382289  7856 ts_tablet_manager.cc:1403] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.110s	user 0.030s	sys 0.055s
I20260709 12:24:52.390822  8161 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.391261  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:52.392659  8161 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:24:52.394439  7702 leader_election.cc:343] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.395361  7856 raft_consensus.cc:359] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.396202  7856 raft_consensus.cc:385] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.396513  7856 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.397280  7856 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.401304  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:52.401700  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:52.402153  7973 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.402281  7737 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.402812  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:52.386626  7857 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.406423  8094 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.411427  7857 ts_tablet_manager.cc:1434] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:24:52.411509  7737 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:52.412573  7659 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.412663  8122 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:52.415212  7856 ts_tablet_manager.cc:1434] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:24:52.416877  7856 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:52.418140  7857 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:52.425467  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:52.426816  8122 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:52.427258  8094 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.427938  8094 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.427875  8122 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:24:52.431140  7856 tablet_bootstrap.cc:654] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.431322  8122 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.437722  7855 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:52.438380  7855 ts_tablet_manager.cc:1403] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.170s	user 0.013s	sys 0.021s
I20260709 12:24:52.441720  7855 raft_consensus.cc:359] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.442512  7855 raft_consensus.cc:385] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.442798  7855 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.479417  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:52.481168  7626 leader_election.cc:343] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.484125  7857 tablet_bootstrap.cc:654] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.484572  8094 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.487515  7855 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.489457  7855 ts_tablet_manager.cc:1434] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.001s
I20260709 12:24:52.493746  7559 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:52.501672  7659 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:52.502075  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.599982  8177 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.600644  8177 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.603427  8177 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:52.605791  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.606730  7737 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:52.609265  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:52.610711  8177 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.611359  8177 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.611946  8177 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.617797  7855 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:52.622021  7856 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.622623  7856 ts_tablet_manager.cc:1403] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.206s	user 0.009s	sys 0.016s
I20260709 12:24:52.626061  7856 raft_consensus.cc:359] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.626888  7856 raft_consensus.cc:385] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.627242  7856 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.628229  7856 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.630141  7856 ts_tablet_manager.cc:1434] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:24:52.631512  7856 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:52.640631  7856 tablet_bootstrap.cc:654] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.649127  8213 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.649793  8213 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.652196  8213 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:52.658556  8161 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.660240  8161 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.661429  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.684481  8177 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.703672  8130 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.662735  8161 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.687855  8164 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.705461  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:52.705268  8164 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.706440  7734 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.708994  8164 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.691823  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.704344  8130 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.710269  7657 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 0.
I20260709 12:24:52.711732  8130 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.685721  7856 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.712545  7856 ts_tablet_manager.cc:1403] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.082s	user 0.017s	sys 0.000s
I20260709 12:24:52.713660  8177 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:52.715696  7856 raft_consensus.cc:359] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.700558  7855 tablet_bootstrap.cc:654] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.719611  7856 raft_consensus.cc:385] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.720002  7856 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.720755  7856 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.723027  7856 ts_tablet_manager.cc:1434] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:24:52.724277  7856 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:52.724414  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:52.715621  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.725090  7659 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.727025  7734 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:52.728912  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:52.733577  7976 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.732098  8177 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:52.720383  7857 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
W20260709 12:24:52.737057  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75314.5us
I20260709 12:24:52.737123  8177 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:24:52.737833  7627 leader_election.cc:343] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.739779  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:52.740165  7857 ts_tablet_manager.cc:1403] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.322s	user 0.011s	sys 0.003s
I20260709 12:24:52.740558  8130 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.743460  7857 raft_consensus.cc:359] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.744274  7857 raft_consensus.cc:385] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.744642  7857 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.742879  8177 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.761402  8130 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.762324  8130 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.765995  8130 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.778352  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.779156  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.781903  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.784494  7777 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:52.784969  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.785833  7734 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 1: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:24:52.757272  7659 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:24:52.787048  8122 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.788764  7855 tablet_bootstrap.cc:492] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:52.789342  7855 ts_tablet_manager.cc:1403] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.172s	user 0.033s	sys 0.033s
I20260709 12:24:52.789836  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.791282  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:52.792021  7734 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
W20260709 12:24:52.793682  7626 leader_election.cc:343] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.745366  7857 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.746927  7856 tablet_bootstrap.cc:654] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.805754  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.818404  7857 ts_tablet_manager.cc:1434] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.000s
I20260709 12:24:52.819803  7857 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
W20260709 12:24:52.824468  7702 leader_election.cc:343] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.824673  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:52.834197  7702 leader_election.cc:343] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.834801  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:52.835637  7856 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:52.835875  8201 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.836342  7856 ts_tablet_manager.cc:1403] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.006s
W20260709 12:24:52.840421  7626 leader_election.cc:343] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.840726  7627 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:52.843835  7855 raft_consensus.cc:359] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.844609  8219 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.846204  7855 raft_consensus.cc:385] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:52.848493  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64063.5us
I20260709 12:24:52.849331  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:52.848627  7855 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.863209  7856 raft_consensus.cc:359] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.867022  7856 raft_consensus.cc:385] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:52.867501  7856 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
W20260709 12:24:52.868726  7627 leader_election.cc:343] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.869477  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:52.870734  7856 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.870260  7855 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.883296  7856 ts_tablet_manager.cc:1434] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.000s
I20260709 12:24:52.877017  8094 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:52.869673  8219 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.885125  8219 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.885555  8219 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.889353  7808 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.902666  8164 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.903340  8164 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.906049  7855 ts_tablet_manager.cc:1434] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.116s	user 0.014s	sys 0.002s
I20260709 12:24:52.907416  7855 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:52.914283  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:52.915274  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:52.915971  7657 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:52.907264  8164 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:52.928972  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:52.936277  7857 tablet_bootstrap.cc:654] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:52.958678  8219 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:52.962414  8099 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:52.962815  8219 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:52.968897  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:24:52.970924  7626 leader_election.cc:343] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:52.983517  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:52.984174  7737 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:52.983868  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:52.988333  7973 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:52.992660  7973 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.998077  7856 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
W20260709 12:24:53.003878  7702 leader_election.cc:343] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:53.000797  7973 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:53.042672  7737 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:53.055401  7855 tablet_bootstrap.cc:654] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:53.066056  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:53.066627  7856 tablet_bootstrap.cc:654] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:53.067334  8219 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:53.067930  8219 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:53.070886  7857 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:53.071525  7857 ts_tablet_manager.cc:1403] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.252s	user 0.044s	sys 0.007s
W20260709 12:24:53.080807  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45675.2us
I20260709 12:24:53.070393  8219 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.195998  7973 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.163491  8177 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.242455  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 86751.3us
I20260709 12:24:53.248814  7973 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:53.259682  8177 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.288013  7857 raft_consensus.cc:359] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.288888  7857 raft_consensus.cc:385] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:53.289197  7857 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.301328  7857 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.303332  7857 ts_tablet_manager.cc:1434] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.232s	user 0.007s	sys 0.003s
I20260709 12:24:53.304587  7857 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
W20260709 12:24:53.310549  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 162852us
I20260709 12:24:53.316854  7857 tablet_bootstrap.cc:654] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:53.333603  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:24:53.311519  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157188us
I20260709 12:24:53.336707  8177 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:53.321219  8229 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.342249  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:53.342983  7662 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:53.345765  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:53.349453  7660 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:53.340303  8218 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.356751  8229 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.376904  8229 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:53.378394  8218 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.403947  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:53.404691  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:53.410780  7662 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:53.404801  7856 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:53.412642  7856 ts_tablet_manager.cc:1403] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.415s	user 0.018s	sys 0.008s
W20260709 12:24:53.417888  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70221.2us
I20260709 12:24:53.428148  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:53.429167  7855 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:53.430125  7855 ts_tablet_manager.cc:1403] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.523s	user 0.015s	sys 0.008s
I20260709 12:24:53.432933  7973 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:53.433414  7973 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.434064  7973 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:53.437732  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:53.437255  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:53.440618  8026 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:53.449369  7973 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.484390  7973 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:53.488380  8026 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:24:53.490626  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57286.6us
I20260709 12:24:53.490185  8026 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.522473  7855 raft_consensus.cc:359] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.523392  7855 raft_consensus.cc:385] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:53.523797  7855 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.525923  8218 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:53.536459  7855 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:53.548193  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85169us
I20260709 12:24:53.586175  7857 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:53.586759  7857 ts_tablet_manager.cc:1403] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.283s	user 0.024s	sys 0.016s
I20260709 12:24:53.588919  7856 raft_consensus.cc:359] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.590126  7856 raft_consensus.cc:385] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:53.590484  7856 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.527575  8213 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260709 12:24:53.592695  7856 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.597422  7856 ts_tablet_manager.cc:1434] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.184s	user 0.007s	sys 0.003s
W20260709 12:24:53.597621  8258 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:53.607324  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:53.650379  7737 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:53.665954  7857 raft_consensus.cc:359] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.603267  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:53.667476  7857 raft_consensus.cc:385] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:53.671766  8213 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:53.669775  7857 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.680126  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:24:53.689507  7626 leader_election.cc:343] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:53.690287  7857 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.694255  7855 ts_tablet_manager.cc:1434] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.263s	user 0.010s	sys 0.031s
I20260709 12:24:53.694242  8243 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:53.695845  7855 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:53.719580  7856 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:53.727182  7857 ts_tablet_manager.cc:1434] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.140s	user 0.002s	sys 0.009s
I20260709 12:24:53.740921  7857 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:53.747763  8264 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:53.791386  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53761.8us
I20260709 12:24:53.837625  7558 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:53.841856  7856 tablet_bootstrap.cc:654] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:53.868629  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:53.869488  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:53.981040  7855 tablet_bootstrap.cc:654] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:53.985142  7626 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:53.995870  8243 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.987818  7857 tablet_bootstrap.cc:654] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:53.989215  7973 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.996511  8243 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.997422  7973 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:53.999586  8243 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:53.999737  7973 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:53.994719  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:54.000669  7657 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.017587  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.017571  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.019264  7662 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:24:54.027014  7702 leader_election.cc:343] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:54.031963  7702 leader_election.cc:343] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:54.055898  7702 leader_election.cc:343] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.052326  8264 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.060853  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.066001  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.066004  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.066689  8169 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.073289  8169 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.073714  8169 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.080587  7657 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
W20260709 12:24:54.084473  7702 leader_election.cc:343] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.088119  8251 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:54.089200  7702 leader_election.cc:343] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.090591  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.128979  7855 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.081394  8169 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.129842  7855 ts_tablet_manager.cc:1403] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.434s	user 0.036s	sys 0.014s
I20260709 12:24:54.128751  8164 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.140849  7856 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.141651  7856 ts_tablet_manager.cc:1403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.422s	user 0.023s	sys 0.024s
I20260709 12:24:54.142609  8252 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.148608  8252 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.146764  7855 raft_consensus.cc:359] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.151137  7855 raft_consensus.cc:385] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.151679  7855 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.140681  8251 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.156462  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.157089  8252 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.158600  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.159461  7812 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.161018  8251 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.161417  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.161976  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:54.163797  7973 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:54.164295  7973 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:54.165097  7973 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.168309  7855 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.167146  8253 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:54.173314  8251 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.178504  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.179818  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:24:54.182042  7627 leader_election.cc:343] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.176402  8253 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.185355  7855 ts_tablet_manager.cc:1434] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.002s
I20260709 12:24:54.180831  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.193279  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.185964  8253 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:54.197078  7856 raft_consensus.cc:359] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.197868  7856 raft_consensus.cc:385] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.198148  7856 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.198825  7856 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.201076  7856 ts_tablet_manager.cc:1434] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.001s
I20260709 12:24:54.202131  7856 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.204481  7855 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:54.205900  7857 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.206480  7857 ts_tablet_manager.cc:1403] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.466s	user 0.015s	sys 0.008s
I20260709 12:24:54.206773  8169 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.209438  7857 raft_consensus.cc:359] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.210287  7857 raft_consensus.cc:385] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.210623  7857 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.211643  7857 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.214690  7857 ts_tablet_manager.cc:1434] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:24:54.216144  7857 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.218626  7856 tablet_bootstrap.cc:654] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.218195  8253 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:54.230957  7626 leader_election.cc:343] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.231501  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:54.233460  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:54.234257  7733 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:54.234755  8253 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.236763  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.239675  8094 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.240721  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.241423  7810 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:54.241463  7626 leader_election.cc:343] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.242866  7855 tablet_bootstrap.cc:654] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.243590  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:54.244261  7731 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.249980  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:24:54.253646  7731 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.253755  7810 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:24:54.256533  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:54.258383  8217 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:54.258970  8217 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:54.259943  8217 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.261857  7778 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:54.263753  7809 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.264032  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.267197  8094 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:54.267772  8094 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:54.268976  7857 tablet_bootstrap.cc:654] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.271970  7973 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:54.268648  8094 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.275460  8264 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.288599  8164 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:54.295395  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:54.326370  7856 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.333248  7856 ts_tablet_manager.cc:1403] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.012s
I20260709 12:24:54.359052  7559 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 0 to 1, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:54.363404  7856 raft_consensus.cc:359] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.364691  7856 raft_consensus.cc:385] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.365479  7856 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.372429  7857 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.372991  7857 ts_tablet_manager.cc:1403] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.020s
I20260709 12:24:54.376605  7857 raft_consensus.cc:359] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.377401  7857 raft_consensus.cc:385] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.377669  7857 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.378793  8217 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.379390  8217 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.382253  7556 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 92) took 1287 ms. Trace:
I20260709 12:24:54.392364  8217 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.393620  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:54.395422  7626 leader_election.cc:343] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.400596  7856 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.405807  7857 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.408006  7857 ts_tablet_manager.cc:1434] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:24:54.382859  7556 rpcz_store.cc:276] 0709 12:24:53.094508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.101358 (+  6850us) service_pool.cc:224] Handling call
0709 12:24:54.382160 (+1280802us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35,"reported_tablets":12}
I20260709 12:24:54.409407  7857 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.413781  7558 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:24:54.417718  7856 ts_tablet_manager.cc:1434] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.079s	user 0.001s	sys 0.008s
I20260709 12:24:54.417295  7558 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:54.418967  7856 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.416167  8094 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.419718  8094 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.426229  7856 tablet_bootstrap.cc:654] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.433394  7855 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.435950  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.439325  8094 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.443111  7855 ts_tablet_manager.cc:1403] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.247s	user 0.016s	sys 0.009s
I20260709 12:24:54.446324  7857 tablet_bootstrap.cc:654] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:54.440084  7626 leader_election.cc:343] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.443720  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.456763  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.460223  7728 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:54.447479  8237 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.462069  8237 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.464581  8237 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:54.468449  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.470952  7809 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.445628  7556 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:54.478557  7855 raft_consensus.cc:359] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.488684  7855 raft_consensus.cc:385] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.489188  7855 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.497901  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.503463  7857 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.504343  7778 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:24:54.507663  7627 leader_election.cc:343] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.509908  8026 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.511940  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:54.512310  8026 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.513464  8094 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.513892  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.514562  8026 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.515159  8094 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.515656  8094 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.516127  8094 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.516328  7857 ts_tablet_manager.cc:1403] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.107s	user 0.013s	sys 0.006s
I20260709 12:24:54.522547  7855 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.494076  7556 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 2, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:54.526860  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.527640  7855 ts_tablet_manager.cc:1434] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.000s
I20260709 12:24:54.528539  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:54.529269  7702 leader_election.cc:343] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.530050  7857 raft_consensus.cc:359] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.530916  7857 raft_consensus.cc:385] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.531263  7857 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.531805  7856 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.532341  7856 ts_tablet_manager.cc:1403] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.114s	user 0.038s	sys 0.061s
I20260709 12:24:54.532356  8094 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:54.533174  7702 leader_election.cc:343] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.534475  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.535409  8026 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.535925  7856 raft_consensus.cc:359] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.536900  7856 raft_consensus.cc:385] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.537184  7856 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.537678  8217 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.537891  7856 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.538395  8217 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.540418  7856 ts_tablet_manager.cc:1434] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:24:54.540987  8217 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.541574  7856 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.540493  8094 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.547986  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.550717  7856 tablet_bootstrap.cc:654] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.552474  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.553798  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:54.554819  7733 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.555814  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.556583  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:54.559191  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:54.560442  8252 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:54.560699  7626 leader_election.cc:343] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.561028  8252 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.561611  8252 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.568648  7856 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.569211  7856 ts_tablet_manager.cc:1403] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.009s
I20260709 12:24:54.570168  7855 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:54.571540  7857 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.580997  7857 ts_tablet_manager.cc:1434] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
W20260709 12:24:54.582928  7702 consensus_peers.cc:597] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:54.584656  7856 raft_consensus.cc:359] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.585481  7856 raft_consensus.cc:385] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.585798  7856 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
W20260709 12:24:54.588835  7626 leader_election.cc:343] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.591934  7856 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.595817  8122 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:54.596174  7856 ts_tablet_manager.cc:1434] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 12:24:54.606792  8252 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.609095  7733 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.608474  7856 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.610145  7857 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.615543  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.617473  8217 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:54.618153  8217 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:54.618579  7855 tablet_bootstrap.cc:654] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.618356  8252 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.619143  8217 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.617661  7857 tablet_bootstrap.cc:654] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.622502  7856 tablet_bootstrap.cc:654] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.623741  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:54.624449  7737 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.634496  7855 tablet_bootstrap.cc:492] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.635529  7855 ts_tablet_manager.cc:1403] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.006s
I20260709 12:24:54.637797  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:24:54.639134  7626 leader_election.cc:343] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.639461  7855 raft_consensus.cc:359] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.640523  7855 raft_consensus.cc:385] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.640527  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.641085  7855 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.641952  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.644523  7857 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.645138  7857 ts_tablet_manager.cc:1403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.002s
I20260709 12:24:54.645807  8252 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:54.648075  8252 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:54.648442  7857 raft_consensus.cc:359] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.649192  7857 raft_consensus.cc:385] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.649475  7857 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.649024  8252 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.650295  7855 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.653507  7857 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.660061  7857 ts_tablet_manager.cc:1434] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 12:24:54.661375  7857 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.662181  7856 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.663012  7856 ts_tablet_manager.cc:1403] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.055s	user 0.006s	sys 0.016s
I20260709 12:24:54.668640  7857 tablet_bootstrap.cc:654] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.666668  7856 raft_consensus.cc:359] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.671890  7856 raft_consensus.cc:385] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.672235  7856 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.672439  7855 ts_tablet_manager.cc:1434] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 12:24:54.673018  7856 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.676295  7856 ts_tablet_manager.cc:1434] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:24:54.673525  7855 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:54.678333  7856 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.684975  7856 tablet_bootstrap.cc:654] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.684975  7855 tablet_bootstrap.cc:654] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.696319  7556 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:54.698411  7856 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.700129  7856 ts_tablet_manager.cc:1403] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.008s
I20260709 12:24:54.698391  7556 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:54.706223  7857 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.707453  7857 ts_tablet_manager.cc:1403] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.002s
I20260709 12:24:54.710587  7857 raft_consensus.cc:359] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.711620  7857 raft_consensus.cc:385] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.711992  7857 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.711711  8202 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:54.712874  7857 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.703951  7856 raft_consensus.cc:359] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.714509  7856 raft_consensus.cc:385] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.714821  7856 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.715734  7856 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.718444  7857 ts_tablet_manager.cc:1434] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:24:54.719826  7857 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.729955  7856 ts_tablet_manager.cc:1434] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:24:54.731706  7856 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.734639  7809 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:54.738807  8094 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:54.739625  7857 tablet_bootstrap.cc:654] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.741667  8094 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.742224  8094 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.738237  7856 tablet_bootstrap.cc:654] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.744725  8094 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.747591  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:54.749444  7626 leader_election.cc:343] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.759001  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:54.759711  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:54.767205  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.769058  8218 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.769639  8218 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.770084  8218 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.791162  7855 tablet_bootstrap.cc:492] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.791854  7855 ts_tablet_manager.cc:1403] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.119s	user 0.031s	sys 0.039s
I20260709 12:24:54.796089  7855 raft_consensus.cc:359] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.797102  7855 raft_consensus.cc:385] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.797480  7855 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.800453  8094 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:54.798545  7855 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:54.804543  7626 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:54.805522  7855 ts_tablet_manager.cc:1434] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:24:54.807595  7855 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:54.809799  7856 tablet_bootstrap.cc:492] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.810415  7856 ts_tablet_manager.cc:1403] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.020s
I20260709 12:24:54.815492  7855 tablet_bootstrap.cc:654] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.817490  7856 raft_consensus.cc:359] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.818356  7856 raft_consensus.cc:385] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.818672  7856 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.822767  7856 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.829917  7855 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.830878  7855 ts_tablet_manager.cc:1403] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 12:24:54.832297  7856 ts_tablet_manager.cc:1434] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:24:54.833766  7856 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.837819  7855 raft_consensus.cc:359] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.802412  8218 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.838889  7855 raft_consensus.cc:385] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.839458  7855 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.840348  7855 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.842986  7855 ts_tablet_manager.cc:1434] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 12:24:54.841321  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:54.838744  8164 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.848129  8164 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.850095  7857 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:54.844278  7855 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:54.850669  8164 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.850672  7857 ts_tablet_manager.cc:1403] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.131s	user 0.025s	sys 0.022s
I20260709 12:24:54.844187  7728 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:54.844709  8218 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.856366  8094 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:54.856570  7857 raft_consensus.cc:359] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.857431  7857 raft_consensus.cc:385] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.857652  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.857780  7857 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.847242  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.858893  7857 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.848237  7737 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.861102  7857 ts_tablet_manager.cc:1434] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:24:54.862493  7857 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:54.870292  7857 tablet_bootstrap.cc:654] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.875625  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.876170  8164 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.876533  7657 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:54.876719  8164 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.877341  8202 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.857926  7855 tablet_bootstrap.cc:654] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:54.877863  8202 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.880190  8202 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.857993  7856 tablet_bootstrap.cc:654] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:54.881978  7702 leader_election.cc:343] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.882650  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:54.886318  7702 leader_election.cc:343] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.887085  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:54.887728  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.890249  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.891621  7662 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:54.890983  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.894033  8026 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.894403  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.894536  8026 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.895416  8202 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.895777  8202 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.896114  8202 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.896543  8026 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.897545  8026 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.897876  8026 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.898190  8026 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.898885  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.899883  7657 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:54.879014  8164 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.900946  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:24:54.907375  7626 leader_election.cc:343] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.907523  8161 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:24:54.912236  7702 leader_election.cc:343] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:54.912613  8161 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:54.914352  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.916368  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:54.926313  7806 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:54.932353  8164 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.932802  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:54.933079  8164 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.933327  7856 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:54.933516  8164 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.934006  7856 ts_tablet_manager.cc:1403] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.101s	user 0.015s	sys 0.015s
I20260709 12:24:54.936391  7973 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:54.936859  7973 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.937162  7973 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.938486  7856 raft_consensus.cc:359] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.941629  7856 raft_consensus.cc:385] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.941937  7856 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.942580  7856 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.945044  7856 ts_tablet_manager.cc:1434] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:24:54.946617  7856 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:54.957185  7973 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.959977  8026 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.966737  7855 tablet_bootstrap.cc:492] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:54.967664  7855 ts_tablet_manager.cc:1403] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.033s
I20260709 12:24:54.975018  8026 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.929941  8161 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.985440  7855 raft_consensus.cc:359] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.972479  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:54.987018  7855 raft_consensus.cc:385] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:54.987629  7855 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.988889  7973 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.989533  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:54.990139  7662 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:54.990761  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.929913  8202 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.991313  7855 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:54.996402  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:54.996784  7855 ts_tablet_manager.cc:1434] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 12:24:54.997213  7805 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.003168  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:24:55.007983  7702 leader_election.cc:343] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.010560  7662 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:55.007149  7805 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:55.014292  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:54.992990  7657 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:55.017206  8312 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.017985  7857 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:55.018525  7857 ts_tablet_manager.cc:1403] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.156s	user 0.027s	sys 0.021s
I20260709 12:24:55.018576  8312 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:55.014505  8202 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:54.978296  8164 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.021004  8312 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.022909  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:55.023530  7662 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.026286  8164 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.027827  7855 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:55.027968  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:24:55.034765  7662 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:55.036831  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.022702  8218 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.038519  7806 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.040426  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:55.041767  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.042613  7662 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.049976  7662 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:55.038602  8202 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.051213  8202 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:55.053198  8202 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.048450  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:55.060999  7857 raft_consensus.cc:359] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.061893  7857 raft_consensus.cc:385] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.062388  7857 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.063696  8164 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.064312  8164 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:55.063290  7857 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.065215  8164 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.075541  7857 ts_tablet_manager.cc:1434] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.004s
I20260709 12:24:55.076649  7857 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:55.082530  7856 tablet_bootstrap.cc:654] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.078325  7557 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.084167  7805 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.081207  7556 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.091461  7855 tablet_bootstrap.cc:654] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:55.097765  7702 leader_election.cc:343] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.098896  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.110909  8314 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.111640  8314 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.107064  8312 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.114177  8314 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.114729  8312 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:55.116191  8312 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.121717  7809 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.124153  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:24:55.136159  7702 leader_election.cc:343] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.141104  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:55.144649  8238 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.145334  8238 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:55.156113  7702 leader_election.cc:343] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.125844  8251 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:24:55.169783  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.171123  8237 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.190622  8094 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.202450  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:55.188145  8237 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.188391  8218 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.222313  7857 tablet_bootstrap.cc:654] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.201560  8311 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:55.211372  8218 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.225826  8218 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.252305  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:24:55.254408  7626 leader_election.cc:343] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.160074  7806 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:55.271750  7856 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:55.272418  7856 ts_tablet_manager.cc:1403] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.326s	user 0.044s	sys 0.037s
W20260709 12:24:55.272511  8315 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:24:55.262279  8218 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.276885  8237 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:55.280807  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:55.273747  8218 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.288419  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:55.289194  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:55.297290  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:55.348132  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:55.305444  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:24:55.443264  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 137882us
I20260709 12:24:55.446511  8252 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:55.447052  8252 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.447484  8252 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.290741  8314 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.314749  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:55.451319  7556 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:55.337545  7626 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:55.455076  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 117639us
I20260709 12:24:55.455829  7556 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.458317  7556 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.299787  7855 tablet_bootstrap.cc:492] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:55.461020  7855 ts_tablet_manager.cc:1403] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.434s	user 0.022s	sys 0.011s
I20260709 12:24:55.461690  7805 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.304991  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:55.348928  8238 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:55.455199  8314 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.467178  8314 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.468719  7855 raft_consensus.cc:359] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.469655  7855 raft_consensus.cc:385] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.469982  7855 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.426726  7856 raft_consensus.cc:359] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.431969  7857 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:55.310708  8218 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:55.472154  7626 leader_election.cc:343] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.480872  7857 ts_tablet_manager.cc:1403] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.404s	user 0.015s	sys 0.024s
I20260709 12:24:55.484702  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:55.484702  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:55.470780  7855 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.487375  7857 raft_consensus.cc:359] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.488478  7857 raft_consensus.cc:385] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.489094  7857 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.485586  7856 raft_consensus.cc:385] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.489789  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:55.490038  7856 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.491043  7856 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.477890  8252 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.494705  7658 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 1: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:24:55.494556  7733 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:55.495102  7856 ts_tablet_manager.cc:1434] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.222s	user 0.011s	sys 0.000s
I20260709 12:24:55.505239  7856 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Bootstrap starting.
I20260709 12:24:55.508020  7857 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:55.512435  7607 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:55.502746  8252 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.514392  7857 ts_tablet_manager.cc:1434] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:24:55.515543  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:55.517988  7855 ts_tablet_manager.cc:1434] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:24:55.520406  7855 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:55.526651  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.528632  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:24:55.529256  7626 leader_election.cc:343] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.530409  8251 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:55.530846  8251 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.531188  8251 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:55.533739  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 236556us
I20260709 12:24:55.549391  7733 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.546052  8251 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.550300  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:55.552469  8237 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.554190  7857 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Bootstrap starting.
I20260709 12:24:55.564580  8260 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:55.564911  7856 tablet_bootstrap.cc:654] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.568279  7733 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:55.571480  8314 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.578274  8314 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:55.580916  8251 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:24:55.583586  7702 leader_election.cc:343] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.583283  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.584146  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.584960  8312 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:55.585877  7626 leader_election.cc:343] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.594302  7857 tablet_bootstrap.cc:654] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.597644  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.601044  8330 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.601624  8330 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:55.602522  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:55.603338  7731 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.606271  7855 tablet_bootstrap.cc:654] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:55.609339  7626 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:55.602486  8330 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.615622  7805 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.621884  7856 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: No bootstrap required, opened a new log
I20260709 12:24:55.622474  7856 ts_tablet_manager.cc:1403] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent bootstrapping tablet: real 0.118s	user 0.022s	sys 0.014s
I20260709 12:24:55.623684  7731 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:55.642338  7856 raft_consensus.cc:359] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.644135  7856 raft_consensus.cc:385] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.644455  7856 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.644563  8314 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.645391  7856 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.652356  7856 ts_tablet_manager.cc:1434] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.001s
I20260709 12:24:55.681835  7556 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.683830  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.684715  8094 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.685500  8094 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:55.686314  8094 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.688422  7857 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: No bootstrap required, opened a new log
I20260709 12:24:55.689009  7857 ts_tablet_manager.cc:1403] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent bootstrapping tablet: real 0.135s	user 0.010s	sys 0.003s
I20260709 12:24:55.693753  7657 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.694527  8253 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:55.700208  7857 raft_consensus.cc:359] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.711779  7857 raft_consensus.cc:385] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.712105  7857 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.723590  7855 tablet_bootstrap.cc:492] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:55.724272  7855 ts_tablet_manager.cc:1403] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.205s	user 0.012s	sys 0.031s
I20260709 12:24:55.730484  7855 raft_consensus.cc:359] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.720150  7857 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.736531  7855 raft_consensus.cc:385] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.736927  7855 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.737778  7855 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.741740  8314 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.739043  7857 ts_tablet_manager.cc:1434] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:24:55.760582  7855 ts_tablet_manager.cc:1434] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.034s	user 0.000s	sys 0.007s
I20260709 12:24:55.763087  7805 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.762360  7559 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.767933  7855 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:55.774612  8312 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.788496  7855 tablet_bootstrap.cc:654] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.802860  7855 tablet_bootstrap.cc:492] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:55.803440  7855 ts_tablet_manager.cc:1403] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.007s
I20260709 12:24:55.812381  8312 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.813772  8217 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.814358  8217 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.815384  7855 raft_consensus.cc:359] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.816418  7855 raft_consensus.cc:385] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.816756  8217 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.817111  8312 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:55.816771  7855 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.818220  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:55.819265  7855 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:55.820021  7626 leader_election.cc:343] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:55.821781  7855 ts_tablet_manager.cc:1434] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:24:55.823312  7855 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:55.828979  8312 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.830595  8312 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.830097  7855 tablet_bootstrap.cc:654] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.846705  8312 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.858059  8314 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:55.867065  7855 tablet_bootstrap.cc:492] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:55.867733  7855 ts_tablet_manager.cc:1403] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.024s
I20260709 12:24:55.868958  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:55.869748  7736 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 0.
I20260709 12:24:55.870829  7855 raft_consensus.cc:359] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.872311  7855 raft_consensus.cc:385] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:55.872608  7855 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:55.873477  7855 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.876253  7855 ts_tablet_manager.cc:1434] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:24:55.877269  7855 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:55.897234  7855 tablet_bootstrap.cc:654] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:55.898715  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:55.899546  7657 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 0.
I20260709 12:24:55.903198  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:55.905622  8330 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:55.906123  8330 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.906510  8330 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.917786  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:24:55.919454  8314 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:55.919941  8314 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.920339  8314 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.964641  8314 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.967458  8314 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.968775  8330 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:55.970538  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:24:55.971350  7657 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.972627  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.973300  7805 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:55.979094  8330 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:55.980528  7805 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:55.981993  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:24:55.983035  8330 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:55.983803  8330 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:24:55.985410  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:24:55.985610  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:24:55.986102  7736 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:55.989283  7702 leader_election.cc:343] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:55.999150  7626 consensus_peers.cc:597] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:55.995677  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:24:55.984719  8330 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.016037  7657 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:24:56.016923  7736 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:24:56.021458  7855 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:56.022172  7855 ts_tablet_manager.cc:1403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.145s	user 0.032s	sys 0.021s
I20260709 12:24:56.026108  7855 raft_consensus.cc:359] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.027171  7855 raft_consensus.cc:385] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:56.027570  7855 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:56.028545  7855 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:24:56.037509  7702 leader_election.cc:343] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:56.039969  7855 ts_tablet_manager.cc:1434] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 12:24:56.041170  7855 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:56.040165  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:24:56.042279  8314 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:56.043061  8314 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:24:56.043993  8314 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.058926  7855 tablet_bootstrap.cc:654] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:56.085683  7557 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.116407  7855 tablet_bootstrap.cc:492] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:56.116972  7855 ts_tablet_manager.cc:1403] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.076s	user 0.029s	sys 0.031s
I20260709 12:24:56.119334  7559 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.120280  7855 raft_consensus.cc:359] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.121239  7855 raft_consensus.cc:385] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:56.121502  7855 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:56.122174  7855 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.124385  7855 ts_tablet_manager.cc:1434] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:56.125356  7855 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:56.131464  7855 tablet_bootstrap.cc:654] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:56.138561  8330 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:56.157223  7855 tablet_bootstrap.cc:492] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:56.157938  7855 ts_tablet_manager.cc:1403] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.003s
I20260709 12:24:56.162669  7855 raft_consensus.cc:359] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.163475  7855 raft_consensus.cc:385] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:56.163789  7855 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:56.164734  7855 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.168231  7855 ts_tablet_manager.cc:1434] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:24:56.171592  7855 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Bootstrap starting.
I20260709 12:24:56.181622  7855 tablet_bootstrap.cc:654] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:56.186074  8330 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:56.190210  7855 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: No bootstrap required, opened a new log
I20260709 12:24:56.191051  7855 ts_tablet_manager.cc:1403] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.001s
I20260709 12:24:56.193434  7855 raft_consensus.cc:359] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.194016  7855 raft_consensus.cc:385] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:56.194350  7855 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Initialized, Role: FOLLOWER
I20260709 12:24:56.195076  7855 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.195461  8122 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.196120  8122 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.197710  7855 ts_tablet_manager.cc:1434] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:24:56.198045  8122 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:56.199770  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:56.208389  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:56.213704  7811 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:56.215842  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:56.224526  8326 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:56.226404  8330 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:56.308709  8122 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.309275  8122 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.311103  8122 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:56.312383  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:56.316748  7658 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 1: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:24:56.314086  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:56.319878  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:56.321187  8321 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:56.327600  8122 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.328475  8122 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.331079  8122 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:56.333446  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:56.333909  7805 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:56.338958  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:56.340433  8321 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:56.341883  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:56.342820  8321 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:56.373397  7811 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:56.423162  8161 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:56.437456  8252 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:56.439669  8321 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.440286  8321 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:24:56.443660  8321 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:56.444479  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:24:56.452898  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:24:56.454878  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:24:56.456081  8321 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:56.477660  7805 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:56.501277  8312 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:56.521174  8164 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:57.354789  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:57.385023  8368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:57.388931  8369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:57.389621  8371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:57.399855  6451 server_base.cc:1061] running on GCE node
I20260709 12:24:57.403120  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:57.403539  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:57.403784  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783599897403762 us; error 0 us; skew 500 ppm
I20260709 12:24:57.404604  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:57.409915  6451 webserver.cc:533] Webserver started at http://127.6.76.196:44599/ using document root <none> and password file <none>
I20260709 12:24:57.412770  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:57.413245  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:57.413707  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:57.415174  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root/instance:
uuid: "992d6df95ef74f7f8b12534534a45501"
format_stamp: "Formatted at 2026-07-09 12:24:57 on dist-test-slave-n447"
I20260709 12:24:57.422953  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:57.428762  8376 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:57.429970  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:24:57.430449  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
uuid: "992d6df95ef74f7f8b12534534a45501"
format_stamp: "Formatted at 2026-07-09 12:24:57 on dist-test-slave-n447"
I20260709 12:24:57.430778  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:57.467469  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:57.469321  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:57.471491  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:57.475031  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:57.475546  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.001s	sys 0.000s
I20260709 12:24:57.475862  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:57.476102  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:57.622916  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.196:36487
I20260709 12:24:57.623068  8439 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.196:36487 every 8 connection(s)
I20260709 12:24:57.670122  6451 auto_leader_rebalancer-test.cc:439] add a tserver: 992d6df95ef74f7f8b12534534a45501
I20260709 12:24:57.721396  8440 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33941
I20260709 12:24:57.722007  8440 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:57.723208  8440 heartbeater.cc:507] Master 127.6.76.254:33941 requested a full tablet report, sending...
I20260709 12:24:57.726760  7556 ts_manager.cc:194] Registered new tserver with Master: 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:24:57.729116  7556 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35204
I20260709 12:24:58.733019  8440 heartbeater.cc:499] Master 127.6.76.254:33941 was elected leader, sending a full tablet report...
I20260709 12:24:58.825681  7810 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:58.846222  7731 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:58.847882  7659 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:58.854406  8321 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:58.856877  8410 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:58.880016  7775 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ddca3b06c0a84859adac6776a38c3518. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:58.887261  8449 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:58.890828  7731 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:58.893637  7659 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:58.916864  7556 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:58.917115  7811 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:24:58.935933  7775 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8d96c92c58a4ad683334c5afc8e9a67. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:58.937513  7735 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:58.939410  8454 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:58.939846  7658 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:58.942217  8448 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:58.980481  8449 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:24:58.982908  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48639.9us
I20260709 12:24:58.985865  7735 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:58.986683  7658 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.037364  7557 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.045689  7658 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.060819  7811 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.066282  7728 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.069051  8457 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.070904  8330 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.114603  8252 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:24:59.122373  7624 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e34ef3d78ea4d53847d21b92b07fab2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:59.123178  7810 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.130641  7735 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.159080  7557 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.183410  7728 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.199510  7811 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.200946  7657 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.203480  8164 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.205557  8312 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:59.215586  7700 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73bea8a36a5046b5b9fb3b3c3a5386f9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:59.220477  8164 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.222810  7811 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.223989  7658 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.241210  7559 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:24:59.481853  8474 ts_tablet_manager.cc:933] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:59.484773  8474 tablet_copy_client.cc:323] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:24:59.502573  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet ddca3b06c0a84859adac6776a38c3518 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:24:59.503401  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet ddca3b06c0a84859adac6776a38c3518 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-ddca3b06c0a84859adac6776a38c3518
I20260709 12:24:59.511940  7822 tablet_copy_source_session.cc:215] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:59.517120  8474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddca3b06c0a84859adac6776a38c3518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:59.533572  8474 tablet_copy_client.cc:806] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:59.534374  8474 tablet_copy_client.cc:670] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:59.539955  8474 tablet_copy_client.cc:538] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:59.548867  8474 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:24:59.556804  8478 ts_tablet_manager.cc:933] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:24:59.559034  8478 tablet_copy_client.cc:323] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:24:59.561169  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet e8d96c92c58a4ad683334c5afc8e9a67 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:24:59.561775  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet e8d96c92c58a4ad683334c5afc8e9a67 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-e8d96c92c58a4ad683334c5afc8e9a67
I20260709 12:24:59.578843  7822 tablet_copy_source_session.cc:215] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:59.581076  8474 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:59.582679  8474 tablet_bootstrap.cc:492] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:24:59.583940  8474 ts_tablet_manager.cc:1403] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.012s
I20260709 12:24:59.586778  8478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d96c92c58a4ad683334c5afc8e9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:59.589397  8474 raft_consensus.cc:359] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.590535  8474 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:24:59.603869  8474 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.632341  8474 ts_tablet_manager.cc:1434] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.048s	user 0.012s	sys 0.008s
I20260709 12:24:59.636861  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-ddca3b06c0a84859adac6776a38c3518 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:24:59.637470  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-ddca3b06c0a84859adac6776a38c3518 on tablet ddca3b06c0a84859adac6776a38c3518 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:24:59.653492  8478 tablet_copy_client.cc:806] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:59.654313  8478 tablet_copy_client.cc:670] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:59.660250  8478 tablet_copy_client.cc:538] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:59.674618  8474 ts_tablet_manager.cc:933] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:24:59.681075  8478 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:24:59.700045  8474 tablet_copy_client.cc:323] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:24:59.729125  8478 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:59.732914  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 7e34ef3d78ea4d53847d21b92b07fab2 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:24:59.733727  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 7e34ef3d78ea4d53847d21b92b07fab2 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-7e34ef3d78ea4d53847d21b92b07fab2
I20260709 12:24:59.733799  8478 tablet_bootstrap.cc:492] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:24:59.735208  8478 ts_tablet_manager.cc:1403] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.008s
I20260709 12:24:59.739342  8478 raft_consensus.cc:359] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.740384  8478 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:24:59.741160  8478 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:24:59.745748  8478 ts_tablet_manager.cc:1434] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:24:59.748528  8485 ts_tablet_manager.cc:933] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:24:59.749511  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-e8d96c92c58a4ad683334c5afc8e9a67 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:24:59.750083  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-e8d96c92c58a4ad683334c5afc8e9a67 on tablet e8d96c92c58a4ad683334c5afc8e9a67 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:24:59.761791  7672 tablet_copy_source_session.cc:215] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:59.767851  8474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e34ef3d78ea4d53847d21b92b07fab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:59.771713  8485 tablet_copy_client.cc:323] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:24:59.898272  8474 tablet_copy_client.cc:806] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:59.900007  8474 tablet_copy_client.cc:670] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:59.960193  8474 tablet_copy_client.cc:538] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:59.960898  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:24:59.961905  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-73bea8a36a5046b5b9fb3b3c3a5386f9
I20260709 12:25:00.013975  8415 raft_consensus.cc:1217] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:00.025904  7747 tablet_copy_source_session.cc:215] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:00.081583  8485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:00.102355  8415 raft_consensus.cc:1217] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:00.105904  8474 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:00.172838  8485 tablet_copy_client.cc:806] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:00.173700  8485 tablet_copy_client.cc:670] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:00.184083  8485 tablet_copy_client.cc:538] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:00.200201  8485 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:00.270210  8474 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.271533  8474 tablet_bootstrap.cc:492] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:00.272445  8474 ts_tablet_manager.cc:1403] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.167s	user 0.032s	sys 0.026s
I20260709 12:25:00.290709  8474 raft_consensus.cc:359] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:00.295401  8474 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:00.306669  8474 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:00.319410  8485 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.321939  8485 tablet_bootstrap.cc:492] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:00.323062  8485 ts_tablet_manager.cc:1403] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.123s	user 0.023s	sys 0.014s
I20260709 12:25:00.337070  8474 ts_tablet_manager.cc:1434] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.000s
I20260709 12:25:00.339548  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-7e34ef3d78ea4d53847d21b92b07fab2 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:00.340109  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-7e34ef3d78ea4d53847d21b92b07fab2 on tablet 7e34ef3d78ea4d53847d21b92b07fab2 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:00.344265  8485 raft_consensus.cc:359] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:00.345717  8485 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:00.346642  8485 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:00.354593  8485 ts_tablet_manager.cc:1434] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:25:00.359159  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-73bea8a36a5046b5b9fb3b3c3a5386f9 received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:00.359728  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-73bea8a36a5046b5b9fb3b3c3a5386f9 on tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:00.542274  8498 raft_consensus.cc:1064] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:00.545593  8498 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:00.556890  8415 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.564867  7658 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.566107  7735 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.567164  8449 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:00.568938  8490 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:00.570928  8498 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:00.592254  8414 raft_consensus.cc:1217] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:00.593463  8490 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.604462  7736 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.608417  8414 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.624225  7659 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.631266  7559 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:00.676294  8490 raft_consensus.cc:1064] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:00.679591  8490 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:00.691797  8414 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.693604  7736 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.696043  8454 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:00.699028  8492 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:00.705017  7659 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:00.707654  8449 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:25:00.725749  8415 raft_consensus.cc:1217] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:00.748338  8498 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.750226  8414 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.752707  7658 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.765849  7736 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.791605  7805 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:00.794452  7556 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.801784  8414 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:00.803558  8449 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:00.805332  7658 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:00.807742  8449 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:00.833397  8454 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.845844  7657 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.847451  8414 raft_consensus.cc:2955] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.851512  7805 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:00.876353  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ddca3b06c0a84859adac6776a38c3518 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:00.892198  7557 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.908317  8414 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:00.912564  8454 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:00.924187  8490 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:00.935134  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54248.4us
I20260709 12:25:00.939426  7736 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:00.940052  8414 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.944001  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet ddca3b06c0a84859adac6776a38c3518 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:00.945657  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8d96c92c58a4ad683334c5afc8e9a67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:00.957743  8490 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:00.961617  7556 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.968215  7731 raft_consensus.cc:2955] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:00.976187  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet e8d96c92c58a4ad683334c5afc8e9a67 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:01.019873  8519 tablet_replica.cc:333] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:01.021227  8519 raft_consensus.cc:2243] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.022185  8519 raft_consensus.cc:2272] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.028676  8519 ts_tablet_manager.cc:1916] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.030586  8518 tablet_replica.cc:333] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:01.032075  8518 raft_consensus.cc:2243] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.033072  8518 raft_consensus.cc:2272] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.038156  8518 ts_tablet_manager.cc:1916] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.061827  8519 ts_tablet_manager.cc:1929] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:01.061827  8518 ts_tablet_manager.cc:1929] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:01.062674  8519 log.cc:1199] T e8d96c92c58a4ad683334c5afc8e9a67 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/e8d96c92c58a4ad683334c5afc8e9a67
I20260709 12:25:01.062718  8518 log.cc:1199] T ddca3b06c0a84859adac6776a38c3518 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/ddca3b06c0a84859adac6776a38c3518
I20260709 12:25:01.065330  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet ddca3b06c0a84859adac6776a38c3518 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:01.066406  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet e8d96c92c58a4ad683334c5afc8e9a67 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:01.178637  8495 raft_consensus.cc:1064] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:01.182408  8495 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:01.193084  7810 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.194880  8499 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.196298  8414 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.201179  8252 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.227600  8444 raft_consensus.cc:1064] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:01.232230  8444 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:01.241204  7736 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.248648  8499 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.258162  8414 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.262764  8444 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.279451  7805 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.295090  7657 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.296041  8444 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:25:01.346051  8495 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.348959  8444 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.359771  7735 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.382378  7810 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.405295  8444 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.417490  7557 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.428009  8414 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.434423  7811 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.453204  8415 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.459372  7658 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.487411  7557 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.516903  7659 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:01.529338  7733 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:01.541757  8414 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:01.544534  8531 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.546697  7659 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:01.549274  8497 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.562922  8496 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.565678  7658 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.566191  8415 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:25:01.570178  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77860.2us
I20260709 12:25:01.568909  8414 raft_consensus.cc:2955] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.578675  7731 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:01.595284  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:01.597273  8252 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.597376  7557 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.608412  8495 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.625779  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:01.631245  8527 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.639559  7733 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.645016  8414 raft_consensus.cc:2955] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.671814  7557 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:01.678982  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7e34ef3d78ea4d53847d21b92b07fab2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:01.682788  8534 tablet_replica.cc:333] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:01.698812  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e34ef3d78ea4d53847d21b92b07fab2 with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:01.699774  8534 raft_consensus.cc:2243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.700608  8534 raft_consensus.cc:2272] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.711668  8534 ts_tablet_manager.cc:1916] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.731000  8534 ts_tablet_manager.cc:1929] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:01.731555  8534 log.cc:1199] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/73bea8a36a5046b5b9fb3b3c3a5386f9
I20260709 12:25:01.733716  8534 tablet_replica.cc:333] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:01.733963  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 73bea8a36a5046b5b9fb3b3c3a5386f9 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:01.735201  8534 raft_consensus.cc:2243] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.736250  8534 raft_consensus.cc:2272] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.740023  8534 ts_tablet_manager.cc:1916] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.754148  8534 ts_tablet_manager.cc:1929] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:01.754932  8534 log.cc:1199] T 7e34ef3d78ea4d53847d21b92b07fab2 P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/7e34ef3d78ea4d53847d21b92b07fab2
I20260709 12:25:01.756799  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 7e34ef3d78ea4d53847d21b92b07fab2 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:02.801594  7805 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:02.810392  7733 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:02.811064  7658 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:02.811946  7775 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ccd837ed3845462283f72c3010de59c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.812119  8492 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:02.813575  8490 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:02.824667  8492 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:02.826876  7733 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:02.828681  7658 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:02.854028  7659 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:02.857230  7559 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:02.871897  7624 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b71af551cbdf4c63a9a0da6165082874. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.881381  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:02.885022  7733 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:02.890589  8535 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:02.892923  8535 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:02.922390  8495 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:02.926855  7811 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:02.937216  7728 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.008780  7557 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:03.022310  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71290.3us
I20260709 12:25:03.030121  7733 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.055411  7811 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:03.055464  7659 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:03.055999  7700 consensus_peers.cc:597] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7112151d65374f6690a117439cc2dc33. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.057559  8539 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:03.060726  8539 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:03.075927  8497 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.104791  7659 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.105558  7810 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.117167  7809 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.132539  7733 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:03.134047  7775 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35f6a2ef5ca4422b882b27d3d2ad1f36. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.134336  8490 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:03.144340  7659 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:03.145622  8490 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.149353  8492 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:03.149425  7557 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.148701  7728 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.166000  7659 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:03.188596  7557 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.383965  8558 ts_tablet_manager.cc:933] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:03.387537  8558 tablet_copy_client.cc:323] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:03.390067  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet b71af551cbdf4c63a9a0da6165082874 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:03.390784  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet b71af551cbdf4c63a9a0da6165082874 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-b71af551cbdf4c63a9a0da6165082874
I20260709 12:25:03.402906  7672 tablet_copy_source_session.cc:215] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.412431  8558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b71af551cbdf4c63a9a0da6165082874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:03.425251  8560 ts_tablet_manager.cc:933] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:03.439327  8558 tablet_copy_client.cc:806] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:03.441226  8558 tablet_copy_client.cc:670] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:03.442091  8560 tablet_copy_client.cc:323] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:03.446462  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet ccd837ed3845462283f72c3010de59c3 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:03.447506  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet ccd837ed3845462283f72c3010de59c3 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-ccd837ed3845462283f72c3010de59c3
I20260709 12:25:03.450945  8558 tablet_copy_client.cc:538] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.477015  8558 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:03.482009  7822 tablet_copy_source_session.cc:215] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.499400  8560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd837ed3845462283f72c3010de59c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:03.533135  8560 tablet_copy_client.cc:806] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:03.536593  8560 tablet_copy_client.cc:670] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:03.542356  8560 tablet_copy_client.cc:538] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.556768  8560 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:03.566583  8558 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.568071  8558 tablet_bootstrap.cc:492] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:03.569201  8558 ts_tablet_manager.cc:1403] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.092s	user 0.022s	sys 0.015s
I20260709 12:25:03.573153  8558 raft_consensus.cc:359] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.574311  8558 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:03.575117  8558 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.598280  8558 ts_tablet_manager.cc:1434] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.029s	user 0.011s	sys 0.017s
I20260709 12:25:03.603856  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-b71af551cbdf4c63a9a0da6165082874 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:03.604423  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-b71af551cbdf4c63a9a0da6165082874 on tablet b71af551cbdf4c63a9a0da6165082874 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:03.613898  8560 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.615363  8560 tablet_bootstrap.cc:492] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:03.616448  8560 ts_tablet_manager.cc:1403] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.060s	user 0.019s	sys 0.018s
I20260709 12:25:03.620437  8560 raft_consensus.cc:359] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.621687  8560 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:03.622582  8560 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.626307  8560 ts_tablet_manager.cc:1434] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:25:03.636495  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-ccd837ed3845462283f72c3010de59c3 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:03.637202  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-ccd837ed3845462283f72c3010de59c3 on tablet ccd837ed3845462283f72c3010de59c3 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:03.660974  8558 ts_tablet_manager.cc:933] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:03.662654  8560 ts_tablet_manager.cc:933] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:03.677731  8560 tablet_copy_client.cc:323] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:03.695202  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:03.695945  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-35f6a2ef5ca4422b882b27d3d2ad1f36
I20260709 12:25:03.712246  7822 tablet_copy_source_session.cc:215] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.718364  8558 tablet_copy_client.cc:323] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:25:03.718546  8560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:03.721269  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 7112151d65374f6690a117439cc2dc33 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:03.721907  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 7112151d65374f6690a117439cc2dc33 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-7112151d65374f6690a117439cc2dc33
I20260709 12:25:03.735728  7747 tablet_copy_source_session.cc:215] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.740885  8558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7112151d65374f6690a117439cc2dc33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:03.743317  8560 tablet_copy_client.cc:806] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:03.744102  8560 tablet_copy_client.cc:670] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:03.748165  8560 tablet_copy_client.cc:538] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.760484  8560 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:03.768163  8558 tablet_copy_client.cc:806] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:03.768970  8558 tablet_copy_client.cc:670] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:03.779366  8558 tablet_copy_client.cc:538] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.785404  8560 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.786701  8560 tablet_bootstrap.cc:492] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:03.787626  8560 ts_tablet_manager.cc:1403] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.008s
I20260709 12:25:03.791141  8560 raft_consensus.cc:359] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.792135  8560 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:03.793040  8560 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.798431  8560 ts_tablet_manager.cc:1434] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 12:25:03.801790  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-35f6a2ef5ca4422b882b27d3d2ad1f36 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:03.802312  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-35f6a2ef5ca4422b882b27d3d2ad1f36 on tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:03.802423  8414 raft_consensus.cc:1217] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:03.802475  8558 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:03.890477  8558 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.891659  8558 tablet_bootstrap.cc:492] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:03.892488  8558 ts_tablet_manager.cc:1403] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.090s	user 0.037s	sys 0.021s
I20260709 12:25:03.895876  8558 raft_consensus.cc:359] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.896994  8558 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:03.898200  8558 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:03.904997  8558 ts_tablet_manager.cc:1434] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 12:25:03.905129  8414 raft_consensus.cc:1217] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:03.907265  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-7112151d65374f6690a117439cc2dc33 received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:03.907812  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-7112151d65374f6690a117439cc2dc33 on tablet 7112151d65374f6690a117439cc2dc33 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:04.174907  8414 raft_consensus.cc:1217] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:04.249866  8414 raft_consensus.cc:1217] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:04.382216  8540 raft_consensus.cc:1064] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:04.386570  8540 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.399669  8527 raft_consensus.cc:1064] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:04.402397  8527 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.410487  8414 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.412523  8540 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.418026  7658 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.418795  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.419677  8415 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.421181  8490 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.421195  8563 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.424135  8563 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.436070  7728 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.449479  8540 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:04.470587  8574 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:04.487241  8562 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.490048  8540 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.498085  8415 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.498595  8414 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.505120  7733 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.513329  8563 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.531021  7728 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.549787  7731 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.552129  7658 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.570026  7559 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.566489  7811 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.635509  7557 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:04.669167  7657 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.689072  8414 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:04.691614  8542 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.701689  8573 raft_consensus.cc:1064] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:04.703601  8563 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.708658  8573 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.712649  7809 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:04.722596  8414 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.728447  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b71af551cbdf4c63a9a0da6165082874 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:04.733878  8580 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.740108  7556 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.747838  8567 raft_consensus.cc:1064] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:04.757094  7811 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.757519  8567 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:04.757324  7810 raft_consensus.cc:2955] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.765623  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet b71af551cbdf4c63a9a0da6165082874 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:04.774964  8414 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:04.776981  8492 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.781168  7733 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:04.785866  8492 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.795948  8540 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.801940  8588 tablet_replica.cc:333] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:04.806934  8588 raft_consensus.cc:2243] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:04.807863  8588 raft_consensus.cc:2272] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:04.811403  8415 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.811132  8414 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.817806  7658 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.822230  8492 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.824407  7811 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.824664  8415 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.829715  7658 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.843688  8588 ts_tablet_manager.cc:1916] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:04.843318  7728 raft_consensus.cc:2955] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.850308  7558 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:04.845120  8497 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.860126  8572 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.864979  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet ccd837ed3845462283f72c3010de59c3 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:04.866091  8490 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.874640  7733 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:04.896504  8593 tablet_replica.cc:333] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:04.900563  8588 ts_tablet_manager.cc:1929] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:04.907755  8531 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.935776  8588 log.cc:1199] T b71af551cbdf4c63a9a0da6165082874 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/b71af551cbdf4c63a9a0da6165082874
I20260709 12:25:04.938943  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet b71af551cbdf4c63a9a0da6165082874 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:04.947557  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ccd837ed3845462283f72c3010de59c3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:04.903095  8596 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:25:04.992450  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71095.1us
I20260709 12:25:05.006170  8591 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.008829  7658 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.021943  8497 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.027034  8413 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.052569  7556 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.057025  8414 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.059954  7733 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.082419  8593 raft_consensus.cc:2243] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.090687  8593 raft_consensus.cc:2272] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.118618  8593 ts_tablet_manager.cc:1916] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.092908  7558 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:05.143172  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72597.7us
I20260709 12:25:05.155483  7809 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.169070  7658 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.182592  7737 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:05.197679  8413 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:05.203958  8602 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.211987  8593 ts_tablet_manager.cc:1929] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:05.212682  8593 log.cc:1199] T ccd837ed3845462283f72c3010de59c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/ccd837ed3845462283f72c3010de59c3
I20260709 12:25:05.214576  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet ccd837ed3845462283f72c3010de59c3 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:05.217298  8567 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.233848  7809 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:05.229180  8413 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.240873  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7112151d65374f6690a117439cc2dc33 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.243420  7659 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:05.249681  8497 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.253968  8414 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:05.258610  8591 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:05.261634  7557 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:05.267125  7731 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:05.273378  7658 raft_consensus.cc:2955] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.277046  8596 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.296327  8592 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.299435  8414 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.304175  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 7112151d65374f6690a117439cc2dc33 with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:05.312824  7731 raft_consensus.cc:2955] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.336620  8604 tablet_replica.cc:333] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:05.356431  8604 raft_consensus.cc:2243] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.352384  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.346191  7559 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:05.364375  8604 raft_consensus.cc:2272] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.368633  8604 ts_tablet_manager.cc:1916] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.371114  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:05.372387  8593 tablet_replica.cc:333] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:05.386551  8593 raft_consensus.cc:2243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.387624  8593 raft_consensus.cc:2272] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.392480  8593 ts_tablet_manager.cc:1916] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.393328  8604 ts_tablet_manager.cc:1929] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:05.393842  8604 log.cc:1199] T 7112151d65374f6690a117439cc2dc33 P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/7112151d65374f6690a117439cc2dc33
I20260709 12:25:05.395910  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 7112151d65374f6690a117439cc2dc33 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:05.413353  8593 ts_tablet_manager.cc:1929] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:05.414149  8593 log.cc:1199] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/35f6a2ef5ca4422b882b27d3d2ad1f36
I20260709 12:25:05.418874  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 35f6a2ef5ca4422b882b27d3d2ad1f36 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:06.471984  7658 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:06.484246  7735 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.485116  7809 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.486009  8599 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:06.487434  7624 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 033af988cc8f496ca24e4443c21ea5d3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:06.489342  8527 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:06.502952  8599 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.506476  7737 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.509140  7809 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.523123  7658 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:06.528504  7559 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:06.545564  7624 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cbbae2856e41420d91e3ccfbf507eacf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:06.548485  7737 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.550380  8542 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.554620  7809 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.556752  8610 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:06.571780  8542 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.573442  7737 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.639861  7557 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:06.651114  7806 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.658363  7737 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:06.672629  7809 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.674846  8619 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:06.680462  7700 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0741ccb73f0c4316be6682db332fd98e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:06.687273  7658 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.689440  8567 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.694236  8619 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.696718  7809 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.720364  7811 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:06.746798  7558 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.747918  7737 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:06.777112  7775 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0bbccb309817428d8d84eeecd4ac4410. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:06.786015  8618 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.776842  7658 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.780339  7657 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:06.792835  8617 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.817377  8617 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.828536  7659 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.856822  7737 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:06.864092  7558 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:06.920279  8627 ts_tablet_manager.cc:933] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:06.922773  8627 tablet_copy_client.cc:323] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:06.924978  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 033af988cc8f496ca24e4443c21ea5d3 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:06.925601  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 033af988cc8f496ca24e4443c21ea5d3 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-033af988cc8f496ca24e4443c21ea5d3
I20260709 12:25:06.932420  7672 tablet_copy_source_session.cc:215] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:06.936704  8627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033af988cc8f496ca24e4443c21ea5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:06.955843  8627 tablet_copy_client.cc:806] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:06.957288  8627 tablet_copy_client.cc:670] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:06.958220  8629 ts_tablet_manager.cc:933] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:06.962483  8627 tablet_copy_client.cc:538] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:06.987574  8627 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:06.988793  8629 tablet_copy_client.cc:323] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:06.992201  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet cbbae2856e41420d91e3ccfbf507eacf from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:06.992834  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet cbbae2856e41420d91e3ccfbf507eacf from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-cbbae2856e41420d91e3ccfbf507eacf
I20260709 12:25:07.006083  7672 tablet_copy_source_session.cc:215] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:07.012328  8629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbbae2856e41420d91e3ccfbf507eacf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:07.028687  8627 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:07.029920  8627 tablet_bootstrap.cc:492] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:07.030725  8627 ts_tablet_manager.cc:1403] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.006s
I20260709 12:25:07.033980  8629 tablet_copy_client.cc:806] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:07.034731  8629 tablet_copy_client.cc:670] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:07.034412  8627 raft_consensus.cc:359] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.035878  8627 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:07.036662  8627 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.040308  8629 tablet_copy_client.cc:538] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:07.041519  8627 ts_tablet_manager.cc:1434] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.009s
I20260709 12:25:07.044857  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-033af988cc8f496ca24e4443c21ea5d3 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:07.045643  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-033af988cc8f496ca24e4443c21ea5d3 on tablet 033af988cc8f496ca24e4443c21ea5d3 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:07.065873  8629 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:07.095723  8629 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:07.096814  8629 tablet_bootstrap.cc:492] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:07.097633  8629 ts_tablet_manager.cc:1403] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 12:25:07.101516  8629 raft_consensus.cc:359] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.102653  8629 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:07.103826  8629 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.110858  8629 ts_tablet_manager.cc:1434] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:25:07.113834  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-cbbae2856e41420d91e3ccfbf507eacf received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:07.114383  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-cbbae2856e41420d91e3ccfbf507eacf on tablet cbbae2856e41420d91e3ccfbf507eacf with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:07.188412  8629 ts_tablet_manager.cc:933] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:07.191718  8629 tablet_copy_client.cc:323] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:07.194267  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 0bbccb309817428d8d84eeecd4ac4410 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:07.194887  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 0bbccb309817428d8d84eeecd4ac4410 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-0bbccb309817428d8d84eeecd4ac4410
I20260709 12:25:07.204324  7822 tablet_copy_source_session.cc:215] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:07.211061  8629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbccb309817428d8d84eeecd4ac4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:07.229487  8629 tablet_copy_client.cc:806] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:07.230509  8629 tablet_copy_client.cc:670] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:07.236276  8629 tablet_copy_client.cc:538] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:07.250728  8629 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:07.262792  8627 ts_tablet_manager.cc:933] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:07.290401  8629 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:07.291967  8629 tablet_bootstrap.cc:492] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:07.292827  8627 tablet_copy_client.cc:323] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:25:07.293088  8629 ts_tablet_manager.cc:1403] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.043s	user 0.034s	sys 0.001s
I20260709 12:25:07.295270  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 0741ccb73f0c4316be6682db332fd98e from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:07.295881  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 0741ccb73f0c4316be6682db332fd98e from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-0741ccb73f0c4316be6682db332fd98e
I20260709 12:25:07.297616  8629 raft_consensus.cc:359] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.298602  8629 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:07.299369  8629 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.304783  8629 ts_tablet_manager.cc:1434] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:25:07.306681  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-0bbccb309817428d8d84eeecd4ac4410 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:07.307183  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-0bbccb309817428d8d84eeecd4ac4410 on tablet 0bbccb309817428d8d84eeecd4ac4410 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:07.320145  7747 tablet_copy_source_session.cc:215] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:07.330150  8627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741ccb73f0c4316be6682db332fd98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:07.359755  8627 tablet_copy_client.cc:806] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:07.360471  8627 tablet_copy_client.cc:670] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:07.365065  8627 tablet_copy_client.cc:538] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:07.385517  8413 raft_consensus.cc:1217] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:07.392021  8627 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:07.425621  8413 raft_consensus.cc:1217] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:07.433888  8627 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:07.435158  8627 tablet_bootstrap.cc:492] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:07.435976  8627 ts_tablet_manager.cc:1403] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.044s	user 0.027s	sys 0.012s
I20260709 12:25:07.441848  8627 raft_consensus.cc:359] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.443492  8627 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:07.444219  8627 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:07.446825  8627 ts_tablet_manager.cc:1434] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:25:07.451107  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-0741ccb73f0c4316be6682db332fd98e received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:07.451645  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-0741ccb73f0c4316be6682db332fd98e on tablet 0741ccb73f0c4316be6682db332fd98e with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:07.572247  8413 raft_consensus.cc:1217] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:07.754355  8413 raft_consensus.cc:1217] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:07.885391  8527 raft_consensus.cc:1064] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:07.888891  8527 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:07.898466  7731 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:07.898826  8413 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:07.900884  8599 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:07.901836  8638 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:07.902945  7809 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:07.904774  8638 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:07.923554  8638 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:07.933895  7737 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:07.939275  8413 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:07.950474  7809 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:07.957351  7557 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:07.981925  8527 raft_consensus.cc:1064] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:07.987025  8527 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.005031  7806 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.006325  8413 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.006675  8527 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.007189  7737 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.008044  8599 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.009007  8645 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.036656  8599 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.049662  7809 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.064348  8414 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.070231  7658 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.095173  8415 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.095785  7731 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.097543  8527 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:08.102897  8527 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:08.100675  8596 raft_consensus.cc:1064] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:08.112560  8596 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.119971  7557 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.137219  7737 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.139199  7661 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.138206  8599 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.142247  8653 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.149636  8653 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.153123  8415 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.150408  8414 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.160364  7658 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.161378  8596 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.170068  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cbbae2856e41420d91e3ccfbf507eacf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:08.178905  7737 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.194389  7811 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.196669  8413 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.198770  8599 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:08.200476  8599 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:08.209154  8527 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.212195  7811 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.218698  7661 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.222133  8413 raft_consensus.cc:2955] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.207758  8653 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.283713  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 033af988cc8f496ca24e4443c21ea5d3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:08.293583  7557 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.307138  7735 raft_consensus.cc:2955] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.322082  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet cbbae2856e41420d91e3ccfbf507eacf with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:08.322810  7556 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.326471  8661 tablet_replica.cc:333] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:08.330390  8413 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.348243  7728 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.352042  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 033af988cc8f496ca24e4443c21ea5d3 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:08.373561  8619 raft_consensus.cc:1064] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:08.389194  8619 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.432379  8662 tablet_replica.cc:333] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:08.434477  8662 raft_consensus.cc:2243] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:08.437242  8662 raft_consensus.cc:2272] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:08.445215  8662 ts_tablet_manager.cc:1916] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:08.480031  7556 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.492401  8661 raft_consensus.cc:2243] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:08.493432  8661 raft_consensus.cc:2272] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:08.508806  8661 ts_tablet_manager.cc:1916] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:08.548224  7809 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.584260  8662 ts_tablet_manager.cc:1929] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:08.584800  8662 log.cc:1199] T 033af988cc8f496ca24e4443c21ea5d3 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/033af988cc8f496ca24e4443c21ea5d3
I20260709 12:25:08.586746  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 033af988cc8f496ca24e4443c21ea5d3 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:08.589072  8413 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.593918  8660 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:08.627629  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144747us
W20260709 12:25:08.634486  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 102788us
W20260709 12:25:08.641793  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58072.8us
I20260709 12:25:08.645570  7658 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.653084  8659 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.659197  8413 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.672322  8660 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:08.671965  8661 ts_tablet_manager.cc:1929] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:08.675292  8661 log.cc:1199] T cbbae2856e41420d91e3ccfbf507eacf P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/cbbae2856e41420d91e3ccfbf507eacf
I20260709 12:25:08.684129  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet cbbae2856e41420d91e3ccfbf507eacf (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:08.692337  7557 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.694103  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0bbccb309817428d8d84eeecd4ac4410 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:08.697863  7809 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.698475  7658 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.706365  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 0bbccb309817428d8d84eeecd4ac4410 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:08.707495  8662 tablet_replica.cc:333] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:08.708700  8662 raft_consensus.cc:2243] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:08.709796  8662 raft_consensus.cc:2272] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:08.717072  8413 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.720352  7658 raft_consensus.cc:2955] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.721778  8662 ts_tablet_manager.cc:1916] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:08.738794  8531 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.742590  8567 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.797531  8667 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.799295  8662 ts_tablet_manager.cc:1929] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:08.799935  8662 log.cc:1199] T 0bbccb309817428d8d84eeecd4ac4410 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/0bbccb309817428d8d84eeecd4ac4410
I20260709 12:25:08.801882  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 0bbccb309817428d8d84eeecd4ac4410 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:08.820564  8668 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.822594  7809 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.849285  8413 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.851887  7658 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.857573  7559 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.903630  7735 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:08.909616  8413 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.909616  7811 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:08.912350  8667 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:08.913805  8668 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:08.921326  8666 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.923885  7809 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.923885  8413 raft_consensus.cc:2955] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:08.935397  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0741ccb73f0c4316be6682db332fd98e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:08.946830  7558 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.980394  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 0741ccb73f0c4316be6682db332fd98e with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:08.982786  8672 tablet_replica.cc:333] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:08.984651  8672 raft_consensus.cc:2243] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:08.986349  8672 raft_consensus.cc:2272] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:08.997592  8672 ts_tablet_manager.cc:1916] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:09.016582  8672 ts_tablet_manager.cc:1929] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:09.017019  8672 log.cc:1199] T 0741ccb73f0c4316be6682db332fd98e P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/0741ccb73f0c4316be6682db332fd98e
I20260709 12:25:09.018852  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 0741ccb73f0c4316be6682db332fd98e (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
W20260709 12:25:10.073410  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51006.8us
I20260709 12:25:10.210210  7658 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:10.226423  7624 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 31230d31601445449fab8f0f9943db57. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:10.227041  7810 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.229707  8677 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.231412  7730 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.233516  8676 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.244953  8676 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.247834  7809 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.251001  7730 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.266433  7810 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.278050  7558 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:10.284145  7775 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 463898dea61f4d7ab42c74b60beb15e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:10.288103  7730 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.293365  8660 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.310571  7658 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.311693  8659 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.313141  8660 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.316190  7737 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.335058  7658 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.336753  7730 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:10.369300  7700 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e78ad8f196c145088ee9ca42482e70ce. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:10.370097  7806 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.361917  7557 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.373490  8682 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.375489  7658 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.377574  8666 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.403548  8668 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.404857  7806 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.405709  7658 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.435894  7730 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.437820  7557 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:10.472608  7700 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 116499cb3f0a4b4eba7b2ace2817bb25. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:10.473193  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.474888  8668 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.476408  7658 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:10.478209  8668 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.512616  8668 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.514966  7658 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.519413  7806 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.537593  7556 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:10.778405  8696 ts_tablet_manager.cc:933] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:10.787489  8696 tablet_copy_client.cc:323] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:10.790920  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 463898dea61f4d7ab42c74b60beb15e6 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:10.792008  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 463898dea61f4d7ab42c74b60beb15e6 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-463898dea61f4d7ab42c74b60beb15e6
I20260709 12:25:10.805632  7822 tablet_copy_source_session.cc:215] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:10.809602  8696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463898dea61f4d7ab42c74b60beb15e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:10.831539  8696 tablet_copy_client.cc:806] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:10.832824  8696 tablet_copy_client.cc:670] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:10.834597  8698 ts_tablet_manager.cc:933] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:10.837145  8698 tablet_copy_client.cc:323] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:10.840323  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 31230d31601445449fab8f0f9943db57 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:10.841012  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 31230d31601445449fab8f0f9943db57 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-31230d31601445449fab8f0f9943db57
I20260709 12:25:10.855948  8696 tablet_copy_client.cc:538] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:10.857067  7672 tablet_copy_source_session.cc:215] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:10.860998  8698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31230d31601445449fab8f0f9943db57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:10.870119  8696 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:10.886255  8698 tablet_copy_client.cc:806] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:10.887022  8698 tablet_copy_client.cc:670] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:10.892199  8698 tablet_copy_client.cc:538] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:10.906426  8698 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:10.910794  8696 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:10.912106  8696 tablet_bootstrap.cc:492] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:10.913034  8696 ts_tablet_manager.cc:1403] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.023s
I20260709 12:25:10.917088  8696 raft_consensus.cc:359] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.918809  8696 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:10.919683  8696 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.930235  8696 ts_tablet_manager.cc:1434] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:25:10.935451  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-463898dea61f4d7ab42c74b60beb15e6 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:10.936110  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-463898dea61f4d7ab42c74b60beb15e6 on tablet 463898dea61f4d7ab42c74b60beb15e6 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:10.950240  8696 ts_tablet_manager.cc:933] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:10.964458  8698 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:10.965687  8698 tablet_bootstrap.cc:492] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:10.966554  8698 ts_tablet_manager.cc:1403] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.009s
I20260709 12:25:10.970192  8698 raft_consensus.cc:359] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.980053  8698 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:10.980981  8698 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:10.982877  8696 tablet_copy_client.cc:323] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:25:10.986442  8698 ts_tablet_manager.cc:1434] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:25:10.990959  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet e78ad8f196c145088ee9ca42482e70ce from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:10.991667  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet e78ad8f196c145088ee9ca42482e70ce from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-e78ad8f196c145088ee9ca42482e70ce
I20260709 12:25:10.992224  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-31230d31601445449fab8f0f9943db57 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:10.992750  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-31230d31601445449fab8f0f9943db57 on tablet 31230d31601445449fab8f0f9943db57 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:11.005323  7747 tablet_copy_source_session.cc:215] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:11.010421  8696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ad8f196c145088ee9ca42482e70ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:11.073920  8696 tablet_copy_client.cc:806] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:11.074955  8696 tablet_copy_client.cc:670] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:11.080799  8696 tablet_copy_client.cc:538] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:11.102103  8696 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:11.122164  8698 ts_tablet_manager.cc:933] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:11.127507  8698 tablet_copy_client.cc:323] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:25:11.129639  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 116499cb3f0a4b4eba7b2ace2817bb25 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:11.130270  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 116499cb3f0a4b4eba7b2ace2817bb25 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-116499cb3f0a4b4eba7b2ace2817bb25
I20260709 12:25:11.145887  7747 tablet_copy_source_session.cc:215] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:11.154984  8696 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:11.155928  8698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116499cb3f0a4b4eba7b2ace2817bb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:11.156277  8696 tablet_bootstrap.cc:492] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:11.159379  8696 ts_tablet_manager.cc:1403] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.006s
I20260709 12:25:11.163000  8696 raft_consensus.cc:359] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:11.164296  8696 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:11.167133  8696 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:11.172734  8696 ts_tablet_manager.cc:1434] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:25:11.176465  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-e78ad8f196c145088ee9ca42482e70ce received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:11.177031  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-e78ad8f196c145088ee9ca42482e70ce on tablet e78ad8f196c145088ee9ca42482e70ce with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:11.202757  8698 tablet_copy_client.cc:806] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:11.203768  8698 tablet_copy_client.cc:670] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:11.209169  8698 tablet_copy_client.cc:538] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:11.221426  8413 raft_consensus.cc:1217] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:11.227685  8698 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:11.258652  8413 raft_consensus.cc:1217] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:11.303469  8698 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:11.304831  8698 tablet_bootstrap.cc:492] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:11.305648  8698 ts_tablet_manager.cc:1403] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.002s
I20260709 12:25:11.309257  8698 raft_consensus.cc:359] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:11.310406  8698 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:11.311297  8698 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:11.314165  8698 ts_tablet_manager.cc:1434] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:25:11.316102  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-116499cb3f0a4b4eba7b2ace2817bb25 received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:11.316700  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-116499cb3f0a4b4eba7b2ace2817bb25 on tablet 116499cb3f0a4b4eba7b2ace2817bb25 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:11.341554  8414 raft_consensus.cc:1217] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:11.886723  8668 raft_consensus.cc:1064] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:11.889328  8668 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:11.912391  8413 raft_consensus.cc:1217] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:11.950510  8715 raft_consensus.cc:1064] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:11.951553  8660 raft_consensus.cc:1064] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:11.957139  8660 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:11.977084  8715 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:11.990612  8414 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:25:12.001555  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60310.4us
W20260709 12:25:12.013839  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69690.3us
I20260709 12:25:12.017108  8706 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.018877  8414 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.028074  8712 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:12.049394  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49039us
W20260709 12:25:12.050035  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56600.6us
I20260709 12:25:12.058178  7735 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.058485  8413 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.060220  8660 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.064276  8721 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:25:12.086400  7806 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.092970  7658 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.096807  8722 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.100001  8721 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.099783  7735 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.106070  8726 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.120954  7658 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.126108  7730 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.129031  7809 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.135335  7661 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.141232  8726 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.142459  8414 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.153180  8666 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.153384  7558 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.165324  8707 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.165345  8727 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.183614  7812 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.199597  7734 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.202073  8666 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.216655  8414 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.225598  7662 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.226807  7812 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.231621  8413 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.288679  7556 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.291983  7556 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.297240  7812 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:12.321851  8414 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.323093  7734 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.323952  8714 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:12.324843  8660 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:12.332396  8660 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.334658  8414 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.336912  7734 raft_consensus.cc:2955] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.348809  7735 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:12.355957  7559 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.359299  8414 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.360153  7662 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.362133  8712 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:12.369004  7662 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:12.372304  8707 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:12.375612  8718 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.380471  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 463898dea61f4d7ab42c74b60beb15e6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:12.381345  7812 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.381716  8414 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.384161  8726 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:12.386206  8726 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:12.387326  8413 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.403913  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 463898dea61f4d7ab42c74b60beb15e6 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:12.406445  7557 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.428296  8713 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.449976  8742 tablet_replica.cc:333] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:12.451642  8742 raft_consensus.cc:2243] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:12.430500  8415 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.453437  8742 raft_consensus.cc:2272] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:12.453671  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e78ad8f196c145088ee9ca42482e70ce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:12.458896  8742 ts_tablet_manager.cc:1916] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:12.465831  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 31230d31601445449fab8f0f9943db57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:12.442453  7662 raft_consensus.cc:2955] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.469761  7812 raft_consensus.cc:2955] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.496567  8666 raft_consensus.cc:1064] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:12.505151  8666 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:12.515919  8742 ts_tablet_manager.cc:1929] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:12.518713  8742 log.cc:1199] T 463898dea61f4d7ab42c74b60beb15e6 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/463898dea61f4d7ab42c74b60beb15e6
I20260709 12:25:12.520165  7559 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.521449  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 463898dea61f4d7ab42c74b60beb15e6 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:12.529825  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet e78ad8f196c145088ee9ca42482e70ce with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:12.541651  8744 tablet_replica.cc:333] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:12.552610  8744 raft_consensus.cc:2243] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:12.553646  8744 raft_consensus.cc:2272] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:12.560467  8413 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.566125  8707 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.568975  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 31230d31601445449fab8f0f9943db57 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:12.573365  8745 tablet_replica.cc:333] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:12.574774  8745 raft_consensus.cc:2243] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:12.575747  8745 raft_consensus.cc:2272] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:12.579708  8745 ts_tablet_manager.cc:1916] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:12.570216  7662 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.586455  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:12.588270  8707 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:12.591588  8707 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:12.625872  8744 ts_tablet_manager.cc:1916] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:12.635406  8745 ts_tablet_manager.cc:1929] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:12.635942  8745 log.cc:1199] T 31230d31601445449fab8f0f9943db57 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/31230d31601445449fab8f0f9943db57
I20260709 12:25:12.634035  8707 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.638034  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 31230d31601445449fab8f0f9943db57 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:12.643445  7662 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.646392  7812 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.663341  8744 ts_tablet_manager.cc:1929] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:12.663935  8744 log.cc:1199] T e78ad8f196c145088ee9ca42482e70ce P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/e78ad8f196c145088ee9ca42482e70ce
I20260709 12:25:12.671144  7557 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.675621  8413 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.683691  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet e78ad8f196c145088ee9ca42482e70ce (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:12.705816  7734 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:12.712657  8413 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.714694  8717 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:12.718743  7812 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:12.721349  8717 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:12.723881  8717 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.726518  8415 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.740366  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 116499cb3f0a4b4eba7b2ace2817bb25 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:12.738544  7811 raft_consensus.cc:2955] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:12.754966  7557 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:12.768514  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08000a64e0 after lost signal to thread 6454
W20260709 12:25:12.769937  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800129e00 after lost signal to thread 7593
W20260709 12:25:12.771497  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08002b3fa0 after lost signal to thread 7686
I20260709 12:25:12.772032  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 116499cb3f0a4b4eba7b2ace2817bb25 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
W20260709 12:25:12.772925  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800327be0 after lost signal to thread 7761
I20260709 12:25:12.773348  8742 tablet_replica.cc:333] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
W20260709 12:25:12.774277  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080031ea80 after lost signal to thread 7836
I20260709 12:25:12.774535  8742 raft_consensus.cc:2243] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:12.775349  8742 raft_consensus.cc:2272] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:12.779496  8742 ts_tablet_manager.cc:1916] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:12.787791  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08002eef00 after lost signal to thread 8439
I20260709 12:25:12.839859  8742 ts_tablet_manager.cc:1929] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:12.840499  8742 log.cc:1199] T 116499cb3f0a4b4eba7b2ace2817bb25 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/116499cb3f0a4b4eba7b2ace2817bb25
I20260709 12:25:12.844836  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 116499cb3f0a4b4eba7b2ace2817bb25 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
W20260709 12:25:13.758049  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53784.5us
W20260709 12:25:13.871403  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55825.1us
I20260709 12:25:13.946934  7812 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:13.986100  7775 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bae510633fdb47c1a109530c37a265b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:13.989007  7661 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:13.991011  8721 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:13.989007  7728 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:13.996204  8721 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:14.018579  8721 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.024353  7661 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.049016  7658 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.050617  7557 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.035567  7728 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.066912  7812 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:14.085956  8706 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:14.089723  7624 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e2e2ecc360e4304b9a841fbd6b8287e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:14.090149  7728 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:14.091794  8725 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:14.136499  8725 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.140560  7728 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.199268  7556 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.218513  7811 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.235540  7661 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.253778  7812 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:14.258525  8763 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:14.261392  7728 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:14.264810  7624 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db9652df6fb248539296e0b903cc4d1c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:14.266773  8762 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:14.289021  8725 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.342557  7811 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.350221  7734 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.386919  7556 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.404246  7809 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:14.434001  7775 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6563fdb577f64184ac4e28f7c43e8c30. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:14.435209  7662 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:14.437179  7734 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:14.437899  8750 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.439769  8751 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:14.451653  8721 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.458350  7662 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.474489  7734 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:14.482664  7556 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.483070  8771 ts_tablet_manager.cc:933] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:14.579015  8771 tablet_copy_client.cc:323] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:14.581192  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet bae510633fdb47c1a109530c37a265b0 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:14.581737  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet bae510633fdb47c1a109530c37a265b0 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-bae510633fdb47c1a109530c37a265b0
I20260709 12:25:14.591830  7822 tablet_copy_source_session.cc:215] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:14.601989  8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae510633fdb47c1a109530c37a265b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:14.621011  8773 ts_tablet_manager.cc:933] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:14.629328  8771 tablet_copy_client.cc:806] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:14.630605  8771 tablet_copy_client.cc:670] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:14.636407  8771 tablet_copy_client.cc:538] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.637477  8773 tablet_copy_client.cc:323] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:14.640128  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 1e2e2ecc360e4304b9a841fbd6b8287e from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:14.640791  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 1e2e2ecc360e4304b9a841fbd6b8287e from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-1e2e2ecc360e4304b9a841fbd6b8287e
I20260709 12:25:14.651005  8771 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:14.657845  7672 tablet_copy_source_session.cc:215] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:14.662077  8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2e2ecc360e4304b9a841fbd6b8287e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:14.710633  8773 tablet_copy_client.cc:806] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:14.711542  8773 tablet_copy_client.cc:670] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:14.711993  8771 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:14.713104  8771 tablet_bootstrap.cc:492] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:14.713946  8771 ts_tablet_manager.cc:1403] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.063s	user 0.026s	sys 0.009s
I20260709 12:25:14.718134  8771 raft_consensus.cc:359] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.719341  8773 tablet_copy_client.cc:538] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.719271  8771 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:14.720297  8771 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.731829  8771 ts_tablet_manager.cc:1434] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.018s	user 0.015s	sys 0.003s
I20260709 12:25:14.734354  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-bae510633fdb47c1a109530c37a265b0 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:14.735013  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-bae510633fdb47c1a109530c37a265b0 on tablet bae510633fdb47c1a109530c37a265b0 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:14.738735  8773 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:14.782778  8771 ts_tablet_manager.cc:933] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:14.795158  8771 tablet_copy_client.cc:323] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:14.797240  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet db9652df6fb248539296e0b903cc4d1c from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:14.797881  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet db9652df6fb248539296e0b903cc4d1c from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-db9652df6fb248539296e0b903cc4d1c
I20260709 12:25:14.810871  7672 tablet_copy_source_session.cc:215] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:14.815853  8773 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:14.822526  8773 tablet_bootstrap.cc:492] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:14.823551  8773 ts_tablet_manager.cc:1403] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.085s	user 0.032s	sys 0.033s
I20260709 12:25:14.828102  8773 raft_consensus.cc:359] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.835275  8773 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:14.836522  8773 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.861646  8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9652df6fb248539296e0b903cc4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:14.871040  8773 ts_tablet_manager.cc:1434] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.001s
I20260709 12:25:14.881814  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-1e2e2ecc360e4304b9a841fbd6b8287e received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:14.882653  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-1e2e2ecc360e4304b9a841fbd6b8287e on tablet 1e2e2ecc360e4304b9a841fbd6b8287e with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:14.893711  8771 tablet_copy_client.cc:806] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:14.895063  8771 tablet_copy_client.cc:670] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:14.900473  8771 tablet_copy_client.cc:538] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.915303  8415 raft_consensus.cc:1217] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:14.930436  8771 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:14.979871  8771 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:14.980965  8771 tablet_bootstrap.cc:492] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:14.981766  8771 ts_tablet_manager.cc:1403] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.062s	user 0.022s	sys 0.007s
I20260709 12:25:14.985064  8771 raft_consensus.cc:359] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.986418  8771 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:14.987306  8771 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:14.992971  8771 ts_tablet_manager.cc:1434] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:25:14.996464  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-db9652df6fb248539296e0b903cc4d1c received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:14.997012  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-db9652df6fb248539296e0b903cc4d1c on tablet db9652df6fb248539296e0b903cc4d1c with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:15.039846  8771 ts_tablet_manager.cc:933] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:15.068691  8771 tablet_copy_client.cc:323] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:15.071038  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 6563fdb577f64184ac4e28f7c43e8c30 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:15.071753  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 6563fdb577f64184ac4e28f7c43e8c30 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-6563fdb577f64184ac4e28f7c43e8c30
I20260709 12:25:15.084708  8415 raft_consensus.cc:1217] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:15.087172  7822 tablet_copy_source_session.cc:215] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:15.091301  8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6563fdb577f64184ac4e28f7c43e8c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:15.127821  8771 tablet_copy_client.cc:806] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:15.128566  8771 tablet_copy_client.cc:670] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:15.137272  8771 tablet_copy_client.cc:538] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:15.165551  8771 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:15.195751  8771 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.197125  8771 tablet_bootstrap.cc:492] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:15.197994  8771 ts_tablet_manager.cc:1403] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.033s	user 0.027s	sys 0.004s
I20260709 12:25:15.201244  8771 raft_consensus.cc:359] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:15.202276  8771 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:15.204278  8771 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:15.207561  8771 ts_tablet_manager.cc:1434] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:25:15.210565  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-6563fdb577f64184ac4e28f7c43e8c30 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:15.211117  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-6563fdb577f64184ac4e28f7c43e8c30 on tablet 6563fdb577f64184ac4e28f7c43e8c30 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:15.318706  8415 raft_consensus.cc:1217] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:15.481089  8751 raft_consensus.cc:1064] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:15.485172  8751 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:15.502177  8415 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.503211  7734 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.505493  8721 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:15.506935  8787 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:15.510036  7662 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.516466  8786 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:15.541169  8413 raft_consensus.cc:1217] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:15.578541  8786 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.580098  7734 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.596822  8413 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.617717  7556 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.635496  7662 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.689551  8706 raft_consensus.cc:1064] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:15.692423  8706 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:15.695904  7811 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:15.717671  8413 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:15.720913  8721 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:15.785077  8786 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:15.850775  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 68652.5us
I20260709 12:25:15.854521  7661 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:15.862010  8725 raft_consensus.cc:1064] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:15.869141  8725 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:15.879940  8721 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:15.891443  7661 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.898501  8413 raft_consensus.cc:2955] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.916901  8415 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.919409  8763 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:15.926559  7558 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:15.961817  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet bae510633fdb47c1a109530c37a265b0 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:15.975620  8413 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.977355  8762 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:15.986177  7734 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.995422  7811 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.997123  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bae510633fdb47c1a109530c37a265b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:16.005398  8800 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:16.006064  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43838.9us
I20260709 12:25:16.017755  8706 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:16.056964  8803 tablet_replica.cc:333] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:16.063038  7735 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:25:16.075631  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50604us
I20260709 12:25:16.078538  7812 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:16.088126  8802 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:16.101104  8762 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:16.104022  8803 raft_consensus.cc:2243] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:16.108510  8803 raft_consensus.cc:2272] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:16.118100  8786 raft_consensus.cc:1064] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:16.125895  8800 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.138281  8803 ts_tablet_manager.cc:1916] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:16.121500  8786 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:16.147333  7812 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.170176  8762 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.171446  7811 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.175860  7559 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.183682  7658 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:16.185436  7735 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:16.185667  8413 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:16.188730  8807 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.190514  8751 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.193117  8751 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.211253  8415 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.233489  8413 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.234820  8787 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.248507  7735 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.264325  8414 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.265738  7558 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.267748  7658 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.282853  7737 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.304044  7556 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.429211  8803 ts_tablet_manager.cc:1929] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:16.429876  8803 log.cc:1199] T bae510633fdb47c1a109530c37a265b0 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/bae510633fdb47c1a109530c37a265b0
I20260709 12:25:16.432035  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet bae510633fdb47c1a109530c37a265b0 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:16.435321  7662 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:16.443188  8415 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:16.446599  8763 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.456061  7812 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:16.469214  8725 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.470772  8413 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.485836  7662 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:16.498368  8415 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:16.500842  8762 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:16.510586  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1e2e2ecc360e4304b9a841fbd6b8287e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:16.517231  8414 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.513767  8798 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.525478  8412 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:16.537700  8787 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:16.545413  7737 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:16.551761  8807 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:16.560408  8786 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.565760  7556 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.580336  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet db9652df6fb248539296e0b903cc4d1c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:16.579802  7728 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.611991  7556 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.621004  7730 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.633234  7735 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:16.644023  7558 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.646134  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6563fdb577f64184ac4e28f7c43e8c30 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:16.660346  7737 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:16.683880  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 6563fdb577f64184ac4e28f7c43e8c30 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:16.690572  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e2e2ecc360e4304b9a841fbd6b8287e with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:16.697367  8816 tablet_replica.cc:333] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:16.706616  8725 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:16.705984  8414 raft_consensus.cc:2955] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.707787  8816 raft_consensus.cc:2243] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:16.708611  8816 raft_consensus.cc:2272] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:16.706234  8815 tablet_replica.cc:333] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:16.710232  8815 raft_consensus.cc:2243] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:16.717934  8815 raft_consensus.cc:2272] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:16.723496  8816 ts_tablet_manager.cc:1916] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:16.742111  8815 ts_tablet_manager.cc:1916] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:16.751452  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet db9652df6fb248539296e0b903cc4d1c with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:16.774925  8816 ts_tablet_manager.cc:1929] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:16.775686  8816 log.cc:1199] T 1e2e2ecc360e4304b9a841fbd6b8287e P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/1e2e2ecc360e4304b9a841fbd6b8287e
I20260709 12:25:16.777835  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 1e2e2ecc360e4304b9a841fbd6b8287e (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:16.778119  8816 tablet_replica.cc:333] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:16.779362  8816 raft_consensus.cc:2243] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:16.780350  8816 raft_consensus.cc:2272] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:16.784945  8815 ts_tablet_manager.cc:1929] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:16.785777  8815 log.cc:1199] T 6563fdb577f64184ac4e28f7c43e8c30 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/6563fdb577f64184ac4e28f7c43e8c30
I20260709 12:25:16.788973  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 6563fdb577f64184ac4e28f7c43e8c30 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:16.794196  8816 ts_tablet_manager.cc:1916] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:16.811408  8725 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.823114  8816 ts_tablet_manager.cc:1929] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:16.823879  8816 log.cc:1199] T db9652df6fb248539296e0b903cc4d1c P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/db9652df6fb248539296e0b903cc4d1c
I20260709 12:25:16.826308  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet db9652df6fb248539296e0b903cc4d1c (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:16.837018  7734 raft_consensus.cc:2955] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:16.836645  7730 raft_consensus.cc:2955] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:17.731065  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:17.903187  7662 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:17.921113  7624 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79849a645ae64c3787630b03e7da6307. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:17.923514  7730 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:17.925309  8762 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:17.926600  7811 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:17.928968  8725 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:17.926630  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:17.930392  7662 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:17.951210  8725 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:17.955399  7806 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:17.974932  7737 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.003438  7556 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:17.981153  7658 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:18.024652  7730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:18.025339  7730 raft_consensus.cc:606] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:18.031142  7661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:18.031854  7661 raft_consensus.cc:606] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:18.036128  7806 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:18.041013  7624 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cee2ab66e8604e5bbd1f08acbef110d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:18.043017  7661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:18.045948  7661 raft_consensus.cc:606] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:18.046063  8762 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:18.057521  7661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:18.058322  7661 raft_consensus.cc:606] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:18.061627  7661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:18.062248  7661 raft_consensus.cc:606] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:18.063994  7730 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:18.067301  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:18.069180  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:18.070232  8828 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.077792  8827 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.081758  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:18.089762  7810 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.117493  7559 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.127730  7661 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:18.119592  7737 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.162302  8827 raft_consensus.cc:993] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.162958  8763 raft_consensus.cc:1081] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.165333  7806 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:18.166188  7806 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.166623  7806 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.170011  7810 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:18.172472  8763 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:18.173817  7624 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 674d58a219c247d091f8a10ca4b75af1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:18.173939  7737 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:18.175494  8763 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:18.182806  7806 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.192989  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.193212  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:18.193840  7661 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:18.193953  7734 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.194298  7661 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:18.192998  7806 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:18.196434  7661 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.197778  7661 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.200629  8762 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.202873  7810 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.209136  7734 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:18.215287  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:18.216543  8786 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:18.217058  8786 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:18.218089  8786 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.221963  7661 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:18.224560  7811 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:18.226496  7559 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:18.226588  7737 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.261426  8682 raft_consensus.cc:993] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
W20260709 12:25:18.261906  7775 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b5cde7dcc814e5d90f0b8e433f8f452. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:18.261193  7559 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.262084  8682 raft_consensus.cc:1081] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.262703  7661 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:18.264212  7810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:18.264478  7737 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:18.264874  7810 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.264488  8787 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.265337  7810 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.267391  8787 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.280143  7810 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.280561  8787 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.283296  7810 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:18.288471  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.289139  7658 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.289381  7661 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.296391  7658 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.298425  7777 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:18.300045  8786 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.300644  8786 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:18.300660  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:18.304466  7737 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.304785  7737 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:18.305540  7737 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.307422  7737 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.312510  8828 raft_consensus.cc:993] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:18.313090  8828 raft_consensus.cc:1081] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:18.302182  8786 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.315972  8414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:18.316877  8414 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.315865  7734 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.317402  8414 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.327863  7737 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.329097  7559 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:18.369405  8414 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.374377  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.375298  7661 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.375834  7661 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:18.376969  7661 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.378224  7661 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.379285  8414 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:18.382443  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 84127.5us
I20260709 12:25:18.385246  8763 raft_consensus.cc:993] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.385881  8763 raft_consensus.cc:1081] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.393572  8847 ts_tablet_manager.cc:933] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:18.398803  7811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:18.399837  7811 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.400327  7811 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.410971  8828 raft_consensus.cc:993] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.417482  8828 raft_consensus.cc:1081] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:18.419628  7810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:18.420259  7810 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.420632  7810 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.437198  7810 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.440766  7811 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.446235  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:18.457638  7734 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.461639  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.462225  7811 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:18.462543  7658 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.464010  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:18.464627  7737 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.469045  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.469700  7662 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.470023  7662 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:18.470829  7662 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.472105  7662 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.473043  7810 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:18.454416  7661 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:18.476789  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:18.477413  7730 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.481159  7662 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.482554  8847 tablet_copy_client.cc:323] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:18.485277  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:18.463099  7658 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:18.487249  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 79849a645ae64c3787630b03e7da6307 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:18.488497  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 79849a645ae64c3787630b03e7da6307 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-79849a645ae64c3787630b03e7da6307
I20260709 12:25:18.489540  8849 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.482671  7734 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:18.491240  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:18.496966  8843 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.498271  8843 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:18.501585  8828 raft_consensus.cc:993] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:18.501714  8843 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.506986  8828 raft_consensus.cc:1081] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:18.509274  8415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:18.509884  8415 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.510284  8415 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.519521  7730 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.521502  7672 tablet_copy_source_session.cc:215] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:18.525306  8847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79849a645ae64c3787630b03e7da6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:18.527473  8849 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:18.528659  8849 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.518391  7558 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.558756  8847 tablet_copy_client.cc:806] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:18.502421  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.564585  8415 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.565150  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:18.566236  8843 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.566834  8843 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:18.567271  8847 tablet_copy_client.cc:670] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:18.568131  8843 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.568842  7559 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.497287  7658 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:18.576661  7658 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.581933  8847 tablet_copy_client.cc:538] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:18.606096  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:18.607322  7657 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.607702  7657 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:18.608748  7657 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.609807  8847 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:18.610066  7657 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.617911  8854 ts_tablet_manager.cc:933] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:18.625598  8415 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:18.627703  8856 ts_tablet_manager.cc:933] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:18.628990  7556 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.630259  7657 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:18.632367  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:18.635052  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:18.636050  7810 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.639001  8854 tablet_copy_client.cc:323] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:18.641034  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet cee2ab66e8604e5bbd1f08acbef110d2 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:18.641839  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet cee2ab66e8604e5bbd1f08acbef110d2 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-cee2ab66e8604e5bbd1f08acbef110d2
I20260709 12:25:18.643680  7556 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.651870  8849 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.652858  8849 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:18.654603  7657 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:18.655158  8856 tablet_copy_client.cc:323] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:18.656548  8843 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.659006  7671 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 674d58a219c247d091f8a10ca4b75af1 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:18.659678  7671 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 674d58a219c247d091f8a10ca4b75af1 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-674d58a219c247d091f8a10ca4b75af1
W20260709 12:25:18.660038  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50515.4us
I20260709 12:25:18.674899  7671 tablet_copy_source_session.cc:215] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:18.679626  8856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674d58a219c247d091f8a10ca4b75af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:18.683177  7810 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:18.654078  8849 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:18.688143  7658 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:18.691949  7672 tablet_copy_source_session.cc:215] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:18.696288  7730 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:18.698328  8843 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.700976  8854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee2ab66e8604e5bbd1f08acbef110d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:18.701975  8847 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:18.702981  8847 tablet_bootstrap.cc:492] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:18.703990  8847 ts_tablet_manager.cc:1403] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.094s	user 0.025s	sys 0.020s
I20260709 12:25:18.754436  8856 tablet_copy_client.cc:806] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:18.755287  8856 tablet_copy_client.cc:670] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:18.794983  7730 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:18.797904  8843 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.825937  8856 tablet_copy_client.cc:538] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:18.814302  8847 raft_consensus.cc:359] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:18.886409  8847 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:18.923663  7556 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:18.956588  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 182917us
I20260709 12:25:18.941845  8847 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:18.997608  8863 ts_tablet_manager.cc:933] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:19.010262  8856 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:19.011066  7657 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.018541  8847 ts_tablet_manager.cc:1434] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.314s	user 0.006s	sys 0.007s
I20260709 12:25:19.019454  8849 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.027516  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-79849a645ae64c3787630b03e7da6307 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:19.028049  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-79849a645ae64c3787630b03e7da6307 on tablet 79849a645ae64c3787630b03e7da6307 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.033676  7662 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.043280  8854 tablet_copy_client.cc:806] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:19.054781  8854 tablet_copy_client.cc:670] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:19.061080  8854 tablet_copy_client.cc:538] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:19.072520  8870 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:19.092684  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 268800us
I20260709 12:25:19.129493  7658 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.132079  7732 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.132835  8872 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:19.144232  7733 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.149142  8856 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:19.156916  8856 tablet_bootstrap.cc:492] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:19.157932  8856 ts_tablet_manager.cc:1403] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.148s	user 0.023s	sys 0.009s
I20260709 12:25:19.163537  8849 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.171082  7661 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.167706  8863 tablet_copy_client.cc:323] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:19.174572  8854 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:19.187327  8856 raft_consensus.cc:359] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:19.187760  8872 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:19.188309  8856 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:19.199604  8856 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:19.200045  8787 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:19.195833  7657 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.212098  7737 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.227296  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:19.229704  8850 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.235697  8856 ts_tablet_manager.cc:1434] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.002s
I20260709 12:25:19.241158  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-674d58a219c247d091f8a10ca4b75af1 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:19.241730  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-674d58a219c247d091f8a10ca4b75af1 on tablet 674d58a219c247d091f8a10ca4b75af1 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.270685  8854 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:19.271835  8854 tablet_bootstrap.cc:492] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:19.272672  8854 ts_tablet_manager.cc:1403] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.005s
I20260709 12:25:19.287662  8854 raft_consensus.cc:359] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:19.289106  8854 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:19.290061  8854 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:19.329981  8854 ts_tablet_manager.cc:1434] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.001s
I20260709 12:25:19.345184  8873 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:19.348865  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-cee2ab66e8604e5bbd1f08acbef110d2 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:19.349457  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-cee2ab66e8604e5bbd1f08acbef110d2 on tablet cee2ab66e8604e5bbd1f08acbef110d2 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.412204  8413 raft_consensus.cc:1217] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:25:19.415380  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61984.5us
I20260709 12:25:19.421054  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:19.421865  7659 raft_consensus.cc:606] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.430716  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:19.431380  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 6b5cde7dcc814e5d90f0b8e433f8f452 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-6b5cde7dcc814e5d90f0b8e433f8f452
I20260709 12:25:19.431602  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31230d31601445449fab8f0f9943db57"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:19.434173  7659 raft_consensus.cc:606] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.440606  7822 tablet_copy_source_session.cc:215] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:19.445585  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.470127  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:19.471165  7732 raft_consensus.cc:606] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:19.480526  8849 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:19.481360  8863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5cde7dcc814e5d90f0b8e433f8f452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:19.499483  8725 raft_consensus.cc:993] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:19.500043  8725 raft_consensus.cc:1081] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:19.502207  8413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31230d31601445449fab8f0f9943db57"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:19.503024  8413 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:19.503486  8413 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.523860  8413 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.526918  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:19.527629  7661 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:19.527964  7661 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:19.528775  7661 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.529757  7661 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.538944  8414 raft_consensus.cc:1217] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:19.543815  7661 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:19.552732  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:19.553819  8849 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:19.554383  8849 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:19.555373  8849 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.565224  8413 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:19.624097  7558 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.648465  8890 raft_consensus.cc:993] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:19.670199  8890 raft_consensus.cc:1081] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:19.655041  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:19.675058  7810 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.684041  8414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:19.684762  8414 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:19.685200  8414 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.637291  8413 raft_consensus.cc:1217] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:19.694406  7810 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:19.625540  8863 tablet_copy_client.cc:806] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:19.696954  8863 tablet_copy_client.cc:670] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:19.701771  8414 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.704149  8414 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:19.705216  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:19.705937  7661 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:19.706278  7661 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:19.707551  7661 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.708917  7661 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:19.808924  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91214.8us
W20260709 12:25:19.820155  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93324.8us
I20260709 12:25:19.826692  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:19.830215  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:19.831353  8850 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:19.831938  8850 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:19.833052  8850 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.863724  8863 tablet_copy_client.cc:538] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:19.876232  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:19.876996  7806 raft_consensus.cc:606] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.879495  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:19.880148  7810 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.883328  8863 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:19.897337  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691f0d885861432f91375d5f288af312"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:19.898046  7662 raft_consensus.cc:606] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:19.944842  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:19.945834  7662 raft_consensus.cc:606] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:19.952947  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:19.953751  7557 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.954814  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:19.979693  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:19.977349  8762 raft_consensus.cc:993] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:19.981818  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:19.984724  8725 raft_consensus.cc:1064] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:19.987515  8725 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:19.989615  8827 raft_consensus.cc:1081] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:19.997946  8863 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:20.000777  8863 tablet_bootstrap.cc:492] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:20.002907  8863 ts_tablet_manager.cc:1403] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.120s	user 0.036s	sys 0.059s
I20260709 12:25:20.009837  7737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "691f0d885861432f91375d5f288af312"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:20.010861  7737 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:20.011329  7737 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.011075  8863 raft_consensus.cc:359] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:20.012182  8863 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:20.013558  7662 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.015196  8863 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:20.015689  8849 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:20.018311  8863 ts_tablet_manager.cc:1434] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:25:20.023938  7737 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.024816  7806 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.028187  8868 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.028643  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-6b5cde7dcc814e5d90f0b8e433f8f452 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:20.029139  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-6b5cde7dcc814e5d90f0b8e433f8f452 on tablet 6b5cde7dcc814e5d90f0b8e433f8f452 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:20.028975  8415 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.033191  8868 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.057729  7734 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.065968  8414 raft_consensus.cc:1217] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:20.072918  7737 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:20.074965  8903 raft_consensus.cc:993] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:20.075757  8903 raft_consensus.cc:1081] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:20.080253  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:20.081128  7658 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:20.081516  7658 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:20.082597  7658 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.084091  7658 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.085543  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:20.086542  7661 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:20.086638  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:20.087826  7806 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.087204  7661 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.100759  7810 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.102464  8850 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:20.103315  7806 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:20.104298  7658 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:20.106725  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:20.108605  8904 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:20.109197  8904 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:20.110144  8904 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.114979  8827 raft_consensus.cc:993] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:20.116132  8827 raft_consensus.cc:1081] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:20.119151  8413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:20.120105  8413 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:20.120568  8413 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.152129  7558 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.166028  8873 raft_consensus.cc:993] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:20.166857  8873 raft_consensus.cc:1081] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:20.173444  8412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40588
I20260709 12:25:20.174110  8412 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:20.174532  8412 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:20.181085  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 124235us
I20260709 12:25:20.183517  8412 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.186821  8412 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:20.188372  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:20.189087  7737 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.192250  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:20.193195  7811 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:20.193533  7811 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:20.194375  7811 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.195659  7811 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.200136  7737 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:20.201805  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:20.202808  8850 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:20.203344  8850 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:20.204233  8850 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.217398  8725 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.225062  8763 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:20.230206  8868 raft_consensus.cc:1064] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:20.233109  7811 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:20.237661  8868 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.189478  8413 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.244357  8413 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:20.245505  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:20.246263  7737 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.245505  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:20.252107  7659 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:20.252451  7659 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:20.253286  7659 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.255061  7659 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.266222  8415 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:20.263932  7661 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.271975  7661 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:20.281021  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 95087.8us
I20260709 12:25:20.281440  7737 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:20.296974  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:20.297734  7810 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.298007  7659 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:20.298265  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:20.299747  8914 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:20.301209  8914 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:20.302248  8914 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.341845  7810 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:20.408614  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:20.418354  7806 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:25:20.421252  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 133844us
I20260709 12:25:20.472879  8890 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.485716  8827 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:25:20.488291  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65037.9us
I20260709 12:25:20.498620  8415 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.494623  8827 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:20.507387  7659 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.507184  8763 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.509378  8827 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.555909  8921 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:25:20.441563  7810 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:20.580431  8763 raft_consensus.cc:1064] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:20.593576  7559 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:20.604013  7812 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.607939  7811 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:20.606901  7559 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.616943  7559 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.625368  8763 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:20.639856  8921 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:20.651476  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77896.5us
I20260709 12:25:20.711409  8904 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:25:20.712869  8904 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:20.713373  8904 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:20.743304  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:20.745694  7734 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.760703  7661 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.774989  7737 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.783334  7730 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:20.714787  8904 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:20.783916  8914 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:25:20.787357  8921 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:20.807840  8904 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.818349  8921 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:20.829886  7659 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:20.860877  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.882473  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.894771  7733 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:20.927158  8413 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:20.951836  8827 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 12:25:20.958396  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91786.5us
I20260709 12:25:20.974324  8413 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:20.978897  8762 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:21.011408  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51821.4us
I20260709 12:25:21.018821  8763 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:21.021879  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48962.2us
I20260709 12:25:21.033844  7812 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:21.033965  7809 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:21.052861  8415 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:21.130527  7809 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:21.139911  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115758us
I20260709 12:25:21.143325  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:21.166977  8921 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:21.166523  8762 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:21.180478  7804 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.183476  7662 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.184566  8827 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:21.189327  7557 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:21.203085  8868 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:25:21.210325  7735 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:21.213197  7810 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:21.304450  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 117237us
W20260709 12:25:21.305112  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119652us
I20260709 12:25:21.326097  8827 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:25:21.323098  8408 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:21.369427  8762 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.340202  8873 raft_consensus.cc:1064] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:21.378245  7729 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.389127  8895 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 12:25:21.413933  8938 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:21.447182  8923 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:21.448097  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115574us
I20260709 12:25:21.449149  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 79849a645ae64c3787630b03e7da6307 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:21.448170  8923 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:21.454334  8923 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:21.465135  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 96928.8us
I20260709 12:25:21.468070  8873 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:21.485818  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 142970us
W20260709 12:25:21.491243  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 124340us
I20260709 12:25:21.531623  8725 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:25:21.519968  7807 raft_consensus.cc:2955] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:21.555894  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 220978us
W20260709 12:25:21.575409  8947 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:25:21.584753  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:21.581310  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:21.527827  7809 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:21.602279  8949 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:25:21.652163  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.750183  7702 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:21.862074  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81043.8us
I20260709 12:25:21.889036  8409 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:21.912672  8940 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.879287  7737 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:21.979116  8827 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:21.987331  8971 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:25:22.196893  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 189868us
W20260709 12:25:22.207499  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 233800us
I20260709 12:25:22.258874  7656 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:22.269227  8938 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:22.304427  8975 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.141s	user 0.004s	sys 0.000s
I20260709 12:25:22.304981  8974 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:22.307693  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 92907.3us
W20260709 12:25:22.312973  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75465us
I20260709 12:25:22.345732  8926 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.346663  8926 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:22.350131  8926 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:22.361342  7810 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:22.363724  7734 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:22.340508  8980 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:22.383040  8869 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.326035  8974 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:22.398834  8972 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.399585  8972 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:22.408424  8869 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:22.424697  8974 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:22.434140  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:22.443794  8412 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:22.236966  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:22.439102  8869 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:22.459622  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:22.466288  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:22.462916  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 215) took 1279 ms. Trace:
I20260709 12:25:22.480157  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:22.487727  8972 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:22.509737  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 117) took 1356 ms. Trace:
I20260709 12:25:22.510274  7559 rpcz_store.cc:276] 0709 12:25:21.153482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.173850 (+ 20368us) service_pool.cc:224] Handling call
0709 12:25:22.509646 (+1335796us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2185,"reported_tablets":2}
I20260709 12:25:22.530175  8926 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.532526  8926 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:22.558987  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:22.547101  8926 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:22.554243  7558 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:22.588336  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46064.3us
I20260709 12:25:22.619657  8944 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:22.668960  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46608.8us
I20260709 12:25:22.678751  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:22.478747  7557 rpcz_store.cc:276] 0709 12:25:21.178576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.178746 (+   170us) service_pool.cc:224] Handling call
0709 12:25:22.457528 (+1278782us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:21.206237 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:21.206839 (+   602us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:21.206904 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:25:21.206922 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:21.207315 (+   393us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:21.207380 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:21.207408 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:21.207627 (+   219us) tablet.cc:701] Row locks acquired
0709 12:25:21.207638 (+    11us) write_op.cc:260] PREPARE: finished
0709 12:25:21.207800 (+   162us) write_op.cc:270] Start()
0709 12:25:21.207941 (+   141us) write_op.cc:276] Timestamp: P: 1783599921207752 usec, L: 0
0709 12:25:21.207959 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:25:21.208683 (+   724us) log.cc:844] Serialized 538 byte log entry
0709 12:25:21.424717 (+216034us) op_driver.cc:464] REPLICATION: finished
0709 12:25:21.830557 (+405840us) write_op.cc:301] APPLY: starting
0709 12:25:21.833477 (+  2920us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:21.838594 (+  5117us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:21.838763 (+   169us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:21.839099 (+   336us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:21.839295 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:21.839348 (+    53us) write_op.cc:312] APPLY: finished
0709 12:25:21.839576 (+   228us) log.cc:844] Serialized 23 byte log entry
0709 12:25:21.845909 (+  6333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.848249 (+  2340us) write_op.cc:454] Released schema lock
0709 12:25:21.850060 (+  1811us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:21.892993 (+ 42933us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.895244 (+  2251us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":387500,"apply.run_cpu_time_us":2690,"apply.run_wall_time_us":48123,"num_ops":1,"prepare.queue_time_us":189,"prepare.run_cpu_time_us":2776,"prepare.run_wall_time_us":2776,"replication_time_us":216558,"thread_start_us":135628,"threads_started":1}]]}
W20260709 12:25:22.706072  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82564.9us
I20260709 12:25:22.620515  8944 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:22.711980  8942 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.712708  8942 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:22.818526  8991 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.810761  8939 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.222s
W20260709 12:25:22.886268  8966 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:22.875839  8895 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:22.812187  8974 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:22.886623  7732 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:22.887892  8895 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:22.819517  8991 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:22.977391  8412 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.001s
I20260709 12:25:22.989683  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:22.923656  8974 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:23.010516  8944 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:23.012423  8991 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:23.015317  8974 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:23.017903  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.025027  8942 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:23.025437  8895 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:23.046206  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.047612  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.047513  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.046895  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.049424  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.051776  7728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:23.052835  7728 raft_consensus.cc:606] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:23.079617  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.082788  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.083853  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.094158  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.094910  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.096747  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:23.102167  8992 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.281s	user 0.007s	sys 0.000s
I20260709 12:25:23.103029  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.107671  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.116531  8957 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:23.133034  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130195us
W20260709 12:25:23.143771  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 128519us
I20260709 12:25:23.124740  8957 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:23.154366  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.167433  8942 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.171249  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:23.172297  8914 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.172104  8926 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.152652  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:23.184820  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:23.215592  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47793us
I20260709 12:25:23.219851  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.221095  8826 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.221791  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.225075  7702 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.229493  8974 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.219044  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:23.240180  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.240521  8926 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.241263  8926 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.241403  8957 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:23.242296  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.240250  7807 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:23.244139  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.230803  8940 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:23.262202  8826 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.266309  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40860756ca234260bb6147416168cd6a"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:23.243327  8957 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.267441  7807 raft_consensus.cc:606] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:23.267701  8411 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:23.280846  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:23.284691  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:23.288728  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:23.306905  7734 raft_consensus.cc:606] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.309965  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.320323  9001 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.332525  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:23.330423  7732 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:23.302186  7655 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:23.339183  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:23.339015  7662 raft_consensus.cc:606] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:23.342389  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:23.344766  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:23.346107  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:23.352980  8957 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.389483  7556 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:23.504240  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 79849a645ae64c3787630b03e7da6307 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:23.506705  7557 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.530082  8974 raft_consensus.cc:993] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:23.531679  8984 raft_consensus.cc:1081] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:23.534380  8984 raft_consensus.cc:993] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:23.534950  8984 raft_consensus.cc:1081] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:23.571833  9012 tablet_replica.cc:333] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:23.573413  9012 raft_consensus.cc:2243] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.906165  9006 raft_consensus.cc:993] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:23.906798  9006 raft_consensus.cc:1081] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:24.158927  9015 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.159726  9015 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.160565  9014 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.161304  9014 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.162613  9015 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.164189  9014 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.188334  9015 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.189009  9015 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.194826  9015 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.202045  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.202248  7656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40860756ca234260bb6147416168cd6a"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:24.202992  7656 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.203465  7656 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.202279  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:24.210464  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 673859us
I20260709 12:25:24.210979  9014 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.211751  9014 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.214310  9014 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.215453  7804 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:24.216173  7804 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.216660  7804 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.221457  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.229002  7656 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.232142  7656 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:24.238883  8763 raft_consensus.cc:993] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:24.239691  9015 raft_consensus.cc:993] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:24.239754  8763 raft_consensus.cc:1081] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:24.240347  9015 raft_consensus.cc:1081] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:24.241451  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:24.242161  7660 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.242570  7660 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.243047  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.253048  7656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:24.256222  7656 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.256780  7656 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.254586  7804 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.268095  7804 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.268780  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:24.269606  7655 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.280352  7655 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:24.259119  7660 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.285116  7660 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:24.260596  8411 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:24.286266  8411 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.286705  8411 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.290596  8988 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:24.291363  8988 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.292764  8989 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.293493  8989 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.294118  8988 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.280061  7656 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.315838  7656 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:24.320088  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.319702  7655 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.256968  8950 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:24.304971  8982 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:24.331909  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 216) took 1113 ms. Trace:
I20260709 12:25:24.344631  8982 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.344389  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:24.347649  9015 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:24.348238  9015 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:24.351595  9015 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.395905  8982 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:24.396838  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:24.397560  7806 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.397935  7806 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:25:24.404929  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.405134  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.343375  7557 rpcz_store.cc:276] 0709 12:25:23.218455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.218584 (+   129us) service_pool.cc:224] Handling call
0709 12:25:23.315691 (+ 97107us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001f2af3a 00007fc007ce0e9c 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c 00007fc00025c8c6
0709 12:25:24.331817 (+1016126us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:23.562195 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:23.564788 (+  2593us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:23.564839 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:25:23.564855 (+    16us) tablet.cc:662] Decoding operations
0709 12:25:23.565295 (+   440us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:23.565379 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:23.565415 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:23.565646 (+   231us) tablet.cc:701] Row locks acquired
0709 12:25:23.565663 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:25:23.565859 (+   196us) write_op.cc:270] Start()
0709 12:25:23.566006 (+   147us) write_op.cc:276] Timestamp: P: 1783599923565801 usec, L: 0
0709 12:25:23.566034 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:25:23.566846 (+   812us) log.cc:844] Serialized 595 byte log entry
0709 12:25:23.576590 (+  9744us) op_driver.cc:464] REPLICATION: finished
0709 12:25:23.578255 (+  1665us) write_op.cc:301] APPLY: starting
0709 12:25:23.579641 (+  1386us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:23.586338 (+  6697us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:23.587954 (+  1616us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:23.588456 (+   502us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:23.588684 (+   228us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:23.588745 (+    61us) write_op.cc:312] APPLY: finished
0709 12:25:23.589056 (+   311us) log.cc:844] Serialized 23 byte log entry
0709 12:25:23.589308 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.589430 (+   122us) write_op.cc:454] Released schema lock
0709 12:25:23.589487 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:23.590014 (+   527us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.590046 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16926,"reported_tablets":7,"spinlock_wait_cycles":181092224,"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":2605,"apply.run_wall_time_us":12290,"num_ops":1,"prepare.queue_time_us":330,"prepare.run_cpu_time_us":3350,"prepare.run_wall_time_us":13796,"replication_time_us":10371}]]}
I20260709 12:25:24.398780  7806 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.422348  7806 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.342942  8941 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:24.400935  8950 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.422875  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:24.423890  7811 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:24.429005  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120801us
I20260709 12:25:24.423183  8941 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.442498  8414 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:24.445377  9004 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:24.449826  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:24.453265  7735 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.463820  9012 raft_consensus.cc:2272] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:24.450964  8941 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:24.422395  8411 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.476284  7657 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.491638  8989 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:24.492668  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:24.493415  7808 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.494024  7808 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:24.455089  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.500037  7806 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.425391  7660 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.494904  7808 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.515954  8409 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.514366  9006 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:25:24.503110  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:24.525357  7811 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.530046  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.532408  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:24.534317  8950 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:24.535787  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:24.536696  7732 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.540104  7733 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:24.541392  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.551787  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.556728  7657 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.558269  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:24.559206  7736 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.559718  7736 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:24.559930  7657 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.561070  7736 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.562544  7736 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.582561  7811 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.599689  7734 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.608723  9003 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:25:24.610359  8949 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.610839  8949 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:24.612805  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 226) took 2039 ms. Trace:
I20260709 12:25:24.613404  7559 rpcz_store.cc:276] 0709 12:25:22.573408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.573694 (+   286us) service_pool.cc:224] Handling call
0709 12:25:24.612729 (+2039035us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7}
I20260709 12:25:24.527993  9012 ts_tablet_manager.cc:1916] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:24.587356  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:24.637840  7808 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.611918  8949 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.618821  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:24.640453  8949 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.646714  8950 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:24.647544  8950 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:24.648751  8950 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.606881  7556 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:24.639139  7735 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.620765  8411 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:24.657727  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:24.659701  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:24.660037  8980 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.662401  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:24.666198  8827 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:24.666787  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:24.668823  8827 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:24.669998  8827 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.671252  8957 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:24.672572  8895 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.672972  8895 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:25:24.673616  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.673964  8895 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.677881  8411 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:24.679940  8895 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.679852  8868 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:24.668040  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44580.1us
I20260709 12:25:24.689041  8868 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:24.691855  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:24.692843  8949 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:24.693354  8949 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:24.694322  8949 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:24.695588  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.695405  7626 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.699612  9013 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.712412  8959 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.716686  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.717568  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.726398  7808 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.737154  7807 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:24.735883  8414 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:24.739781  8868 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:24.750687  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:24.751398  7734 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.757696  7736 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:24.761951  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:24.763224  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:24.764684  7590 debug-util.cc:398] Leaking SignalData structure 0x7b080003c980 after lost signal to thread 6454
I20260709 12:25:24.765866  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.727502  9004 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:24.776014  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.777118  7737 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.777443  7734 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:24.779508  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:24.779680  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:24.793551  7590 debug-util.cc:398] Leaking SignalData structure 0x7b0800039b60 after lost signal to thread 7593
I20260709 12:25:24.798835  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.813829  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:24.819429  7590 debug-util.cc:398] Leaking SignalData structure 0x7b0800338be0 after lost signal to thread 7686
I20260709 12:25:24.845460  8763 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:24.849471  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7e80 after lost signal to thread 7761
I20260709 12:25:24.850270  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:24.823194  9014 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:24.856299  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.859117  8411 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:24.874492  7590 debug-util.cc:398] Leaking SignalData structure 0x7b0800341660 after lost signal to thread 7836
W20260709 12:25:24.876576  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1980 after lost signal to thread 8439
W20260709 12:25:24.877013  7590 debug-util.cc:398] Leaking SignalData structure 0x7b080019f020 after lost signal to thread 8440
W20260709 12:25:24.877532  7590 debug-util.cc:398] Leaking SignalData structure 0x7b080030de00 after lost signal to thread 8924
I20260709 12:25:24.877933  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.879334  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.853389  8981 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.886629  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.892688  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cee2ab66e8604e5bbd1f08acbef110d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:24.919978  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:24.877215  7655 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:24.889662  9006 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.923627  9018 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:24.926041  9012 ts_tablet_manager.cc:1929] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:24.926635  9018 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:24.928089  9018 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:24.916455  9014 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:24.916930  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66759.4us
I20260709 12:25:24.926822  9012 log.cc:1199] T 79849a645ae64c3787630b03e7da6307 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/79849a645ae64c3787630b03e7da6307
I20260709 12:25:24.942418  9012 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46534 (request call id 70) took 1438 ms. Trace:
I20260709 12:25:24.942854  9012 rpcz_store.cc:276] 0709 12:25:23.503632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.503903 (+   271us) service_pool.cc:224] Handling call
0709 12:25:23.571362 (+ 67459us) ts_tablet_manager.cc:1151] Deleting tablet 79849a645ae64c3787630b03e7da6307
0709 12:25:23.571481 (+   119us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:24.712817 (+1141336us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:24.925834 (+213017us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:24.942297 (+ 16463us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":889371,"tablet-delete.queue_time_us":66149,"thread_start_us":376182,"threads_started":1}
I20260709 12:25:24.912631  8978 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.945482  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 79849a645ae64c3787630b03e7da6307 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:24.955271  7733 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
W20260709 12:25:24.958105  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69964.6us
I20260709 12:25:24.948515  8989 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:25.043583  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83858us
I20260709 12:25:25.059501  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 674d58a219c247d091f8a10ca4b75af1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:25.076644  7657 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:25.077409  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9180 after lost signal to thread 6454
W20260709 12:25:25.121488  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60650.8us
W20260709 12:25:25.129295  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 209463us
W20260709 12:25:25.138273  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9860 after lost signal to thread 7593
W20260709 12:25:25.142406  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08001bff40 after lost signal to thread 7686
W20260709 12:25:25.142751  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800230b20 after lost signal to thread 7687
W20260709 12:25:25.143941  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800324700 after lost signal to thread 7761
W20260709 12:25:25.144271  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002b37c0 after lost signal to thread 7762
W20260709 12:25:25.145367  8436 debug-util.cc:398] Leaking SignalData structure 0x7b080031b740 after lost signal to thread 7836
W20260709 12:25:25.145682  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800322320 after lost signal to thread 7837
W20260709 12:25:25.146795  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08003381e0 after lost signal to thread 8439
I20260709 12:25:25.151706  8989 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260709 12:25:25.147078  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800286c20 after lost signal to thread 8440
I20260709 12:25:25.177102  7733 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.184497  7811 raft_consensus.cc:2955] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:25.189626  7808 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:25.193378  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58766us
I20260709 12:25:25.226418  7806 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:25.230799  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47340.3us
W20260709 12:25:25.221342  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002bf540 after lost signal to thread 8895
I20260709 12:25:25.257616  7733 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.271507  7806 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.274825  7731 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.234385  9039 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:25:25.235654  9034 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 12:25:25.283285  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49431.3us
I20260709 12:25:25.292726  9037 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:25:25.300462  9034 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:25.282943  7732 raft_consensus.cc:2955] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:25.301558  8763 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:25.316552  8981 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:25.352351  7735 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.325873  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:25.356812  7557 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:25.365659  7557 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:25.367735  7557 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:25.354960  7556 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 119) took 1753 ms. Trace:
I20260709 12:25:25.388948  8949 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:25.407295  7728 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:25.385303  7556 rpcz_store.cc:276] 0709 12:25:23.601530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.601728 (+   198us) service_pool.cc:224] Handling call
0709 12:25:25.354848 (+1753120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:24.708725 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:24.710117 (+  1392us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:24.710188 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:25:24.710219 (+    31us) tablet.cc:662] Decoding operations
0709 12:25:24.710869 (+   650us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:24.711003 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:24.711055 (+    52us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:24.756461 (+ 45406us) tablet.cc:701] Row locks acquired
0709 12:25:24.756483 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:25:24.756660 (+   177us) write_op.cc:270] Start()
0709 12:25:24.756809 (+   149us) write_op.cc:276] Timestamp: P: 1783599924756604 usec, L: 0
0709 12:25:24.756835 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:24.870971 (+114136us) log.cc:844] Serialized 595 byte log entry
0709 12:25:25.147188 (+276217us) op_driver.cc:464] REPLICATION: finished
0709 12:25:25.277565 (+130377us) write_op.cc:301] APPLY: starting
0709 12:25:25.283247 (+  5682us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:25.286618 (+  3371us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:25.293589 (+  6971us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:25.295593 (+  2004us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:25.295761 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:25.295813 (+    52us) write_op.cc:312] APPLY: finished
0709 12:25:25.299583 (+  3770us) log.cc:844] Serialized 23 byte log entry
0709 12:25:25.304412 (+  4829us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:25.304511 (+    99us) write_op.cc:454] Released schema lock
0709 12:25:25.304554 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:25.310163 (+  5609us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:25.310197 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":843443,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":121576,"apply.run_cpu_time_us":3037,"apply.run_wall_time_us":39260,"num_ops":1,"prepare.queue_time_us":7551,"prepare.run_cpu_time_us":95786,"prepare.run_wall_time_us":311666,"raft.queue_time_us":158199,"raft.run_cpu_time_us":73762,"raft.run_wall_time_us":107884,"replication_time_us":390065,"thread_start_us":337287,"threads_started":4,"wal-append.queue_time_us":114940}]]}
I20260709 12:25:25.466614  7555 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:25.471923  7555 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:25.474310  7555 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 1 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:25.493023  7659 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:25.479329  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64643.1us
I20260709 12:25:25.482775  9051 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 12:25:25.591423  9050 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
W20260709 12:25:25.619315  9055 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.142s	user 0.000s	sys 0.003s
I20260709 12:25:25.665676  7559 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:25.723253  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217840us
I20260709 12:25:25.724282  8982 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.735174  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet cee2ab66e8604e5bbd1f08acbef110d2 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:25.830417  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 674d58a219c247d091f8a10ca4b75af1 with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:25.866166  9019 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:25.866933  9019 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:25.870316  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:25.874238  9019 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:25.875769  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:25.875873  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:25.878293  7702 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:25.879374  9019 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:25.845513  7811 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:25.957459  9069 tablet_replica.cc:333] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:25.935029  8978 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:25.960944  8978 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:25.971024  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:25.933917  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46461.8us
I20260709 12:25:25.975118  9071 tablet_replica.cc:333] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:25.977054  9071 raft_consensus.cc:2243] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:25.983830  9071 raft_consensus.cc:2272] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:25.992897  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:25.996299  7778 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:25.997318  7778 raft_consensus.cc:2686] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:25:25.994827  8978 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:26.026069  9071 ts_tablet_manager.cc:1916] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:26.032011  7730 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:26.040035  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:26.068567  7807 raft_consensus.cc:606] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:26.069657  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:26.076912  8411 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:26.077159  9069 raft_consensus.cc:2243] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:26.078150  9069 raft_consensus.cc:2272] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:26.082643  9069 ts_tablet_manager.cc:1916] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:26.069690  8978 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:25:26.106056  9069 ts_tablet_manager.cc:1929] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:26.106654  9069 log.cc:1199] T cee2ab66e8604e5bbd1f08acbef110d2 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/cee2ab66e8604e5bbd1f08acbef110d2
I20260709 12:25:26.107452  9006 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:26.108877  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet cee2ab66e8604e5bbd1f08acbef110d2 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:26.134248  9071 ts_tablet_manager.cc:1929] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:26.147397  8981 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:26.151005  7655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:26.151865  7655 raft_consensus.cc:606] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:26.155969  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:26.156767  7660 raft_consensus.cc:606] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:26.161912  9036 raft_consensus.cc:993] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:26.162550  9036 raft_consensus.cc:1081] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:26.164063  7655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:26.164929  7655 raft_consensus.cc:606] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:26.164979  8407 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40594
I20260709 12:25:26.166095  8407 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:26.166712  8407 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.165458  7655 raft_consensus.cc:614] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.177695  7730 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:26.189767  9044 raft_consensus.cc:993] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:26.190383  9044 raft_consensus.cc:1081] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:26.191071  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:26.191795  7807 raft_consensus.cc:606] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:26.192916  9038 raft_consensus.cc:993] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:26.193461  9038 raft_consensus.cc:1081] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:26.141834  9071 log.cc:1199] T 674d58a219c247d091f8a10ca4b75af1 P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/674d58a219c247d091f8a10ca4b75af1
I20260709 12:25:26.213478  7807 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:26.213785  7735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:26.214205  7807 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:26.214679  7807 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.214490  7735 raft_consensus.cc:606] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:26.220861  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:26.222783  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:26.223676  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:26.219266  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55495.4us
I20260709 12:25:26.228989  7807 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.230823  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:26.231616  7660 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:26.232051  7660 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.245450  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 674d58a219c247d091f8a10ca4b75af1 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:26.260133  8980 raft_consensus.cc:993] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:26.261245  9076 raft_consensus.cc:1081] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:26.196367  8407 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.266249  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:26.268447  8414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40588
I20260709 12:25:26.269063  8407 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:26.269181  8414 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:26.269732  8414 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.270961  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:26.271637  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:26.272476  7656 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:26.272850  7656 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:26.273764  7656 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.271696  7658 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:26.275746  7658 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:26.275905  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:26.273368  7558 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.276593  7731 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.276912  7658 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.278689  7658 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.283720  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:26.284539  7735 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.285104  7731 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:26.286885  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:26.287901  9006 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:26.288422  9006 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:26.288586  7656 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.289732  8414 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.290083  9006 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.294181  7807 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:26.296200  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:26.297278  7729 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.299178  8414 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:26.300362  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:26.301177  7810 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:26.301549  7810 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:26.304731  7660 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.311057  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:26.315616  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:26.316337  7807 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:26.316619  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:26.317983  8982 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:26.318610  8982 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:26.316663  7807 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:26.319653  8982 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.322819  7660 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:26.374634  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49198.8us
I20260709 12:25:26.332448  7810 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.388935  7810 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.321611  7807 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.392001  7807 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.431565  7810 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:26.433431  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:26.434473  9018 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:26.435114  9018 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:26.436173  7807 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:26.438283  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:26.439497  7735 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.439601  7658 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:26.438362  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:26.436229  9018 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:26.335661  8411 raft_consensus.cc:2955] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:26.480108  7656 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:26.459926  9003 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:26.495787  9003 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:26.497170  9003 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.508554  7559 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.517509  7729 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:26.553202  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:26.593535  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5cde7dcc814e5d90f0b8e433f8f452 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:26.605346  7555 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.613987  7559 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.676586  9092 tablet_replica.cc:333] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:26.682905  9092 raft_consensus.cc:2243] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:26.684042  9092 raft_consensus.cc:2272] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:26.684769  9084 raft_consensus.cc:993] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:26.691463  9025 raft_consensus.cc:1081] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:26.696607  7658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:26.697811  7658 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:26.698496  7658 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.701025  7557 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.710765  7658 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.713874  7658 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:26.718101  9092 ts_tablet_manager.cc:1916] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:26.724886  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:26.726157  7810 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.729640  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:26.730589  7732 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:26.731009  7732 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:26.732707  7732 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.734330  7732 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:26.735764  7658 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:26.743829  9006 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:26.752162  7810 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:26.754009  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:26.755074  8763 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:26.755685  8763 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:26.756803  8763 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:26.778404  7732 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:26.786042  9092 ts_tablet_manager.cc:1929] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:26.786638  9092 log.cc:1199] T 6b5cde7dcc814e5d90f0b8e433f8f452 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/6b5cde7dcc814e5d90f0b8e433f8f452
I20260709 12:25:26.795349  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 6b5cde7dcc814e5d90f0b8e433f8f452 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:26.815157  7735 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:26.818037  8989 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:26.923130  7658 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.925158  9018 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:26.907467  7557 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.988698  7810 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.990787  7732 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.992383  9090 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:26.994710  9090 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:26.999882  7732 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:27.027505  8763 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:27.040758  7732 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:27.042512  9080 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:27.141422  7804 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:27.143471  9037 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:27.149853  7810 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:27.159098  9003 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:27.263643  7730 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:27.271134  9034 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:27.536554  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45343.4us
I20260709 12:25:27.651821  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:27.818778  7730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:27.819484  7730 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:27.827582  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:27.828365  7804 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:27.845602  7655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:27.858649  7655 raft_consensus.cc:606] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:27.898746  7655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:27.899446  7655 raft_consensus.cc:606] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:27.901185  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:27.903124  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:27.904059  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:27.933457  7607 auto_rebalancer.cc:629] Failed to schedule move for tablet 100d9b95a9cf4fab8305b579485c2332: Remote error: Service unavailable: leader transfer in progress
I20260709 12:25:27.939405  7729 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:27.962186  7700 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0733d7d23d9041e09a5ade18807c383e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:27.969079  7660 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:27.971122  7804 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:27.973121  9019 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:27.975502  9109 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:28.023639  9109 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.027490  7810 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.034955  9003 raft_consensus.cc:993] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:28.035707  9034 raft_consensus.cc:1081] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:28.038007  7729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:28.051338  7729 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:28.051993  7729 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.060243  7555 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.083529  7655 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.090886  7658 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:28.106941  7810 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:28.109419  9003 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:28.114133  7624 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 40860756ca234260bb6147416168cd6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:28.156913  7729 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:28.167986  7729 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:28.169502  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:28.170250  8411 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.178854  9019 raft_consensus.cc:993] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:28.179499  9108 raft_consensus.cc:1081] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:28.183864  8411 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:28.185724  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:28.186399  9119 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:28.186838  9085 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.194711  9085 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:28.195812  9085 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:28.196559  9111 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.201447  7808 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.227147  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:28.228047  7656 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:28.228430  7656 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:28.229444  7656 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:28.232528  7810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:28.233357  7810 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:28.233773  7810 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.244014  8980 raft_consensus.cc:993] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:28.244731  8980 raft_consensus.cc:1081] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:28.235684  7656 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.249343  7729 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:28.263612  7659 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:28.264400  7659 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:28.265018  7659 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.267202  7810 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:28.269933  9037 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:28.272521  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:28.272774  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:28.273360  7729 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:28.273502  8411 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.273798  7729 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:28.277495  7656 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:28.279325  7659 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.281847  7810 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:28.282323  7659 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:28.290652  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:28.294595  7735 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.285375  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:28.274914  7729 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:28.304657  7804 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:28.301816  7810 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:28.305171  7804 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:28.306466  7729 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.306242  7804 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.307818  7804 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.308507  7557 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.311862  7558 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.313935  7804 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:28.316062  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:28.319484  9037 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.320111  9037 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:28.321043  9037 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.324750  7735 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:28.296406  8411 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:28.331358  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:28.332281  9006 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.332849  9006 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:28.334146  7729 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:28.333813  9006 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:28.336979  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44211.1us
I20260709 12:25:28.350715  7730 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.371160  9003 raft_consensus.cc:993] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:28.371821  9003 raft_consensus.cc:1081] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:28.378862  7804 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:28.379729  7804 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:28.380177  7804 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.391001  7804 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.396195  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:28.396993  7656 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:28.397403  7656 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:28.398628  7656 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.399474  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:28.400236  7729 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.400256  7656 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.408655  7775 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e5b27e8abd04f2ea19a70a13d1958ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:28.410303  7729 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:28.411947  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:28.413131  8980 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.413730  8980 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:28.414772  8980 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:28.419337  7558 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.422143  7730 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:28.429283  9110 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:28.436112  7804 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:28.437609  7659 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:28.441654  9006 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:28.433458  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:28.449132  7557 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.473452  9127 ts_tablet_manager.cc:933] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:28.480679  7558 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:28.495981  9127 tablet_copy_client.cc:323] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:25:28.498421  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 0733d7d23d9041e09a5ade18807c383e from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:28.499069  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 0733d7d23d9041e09a5ade18807c383e from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-0733d7d23d9041e09a5ade18807c383e
I20260709 12:25:28.503953  9006 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.520900  7656 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.512640  7730 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:28.538174  7747 tablet_copy_source_session.cc:215] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:28.561913  9127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0733d7d23d9041e09a5ade18807c383e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:28.576660  7558 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 2, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.660630  9127 tablet_copy_client.cc:806] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:28.661403  9127 tablet_copy_client.cc:670] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:28.665983  9127 tablet_copy_client.cc:538] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:28.795365  8414 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:28.808135  9136 ts_tablet_manager.cc:933] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:28.797279  7807 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:28.831737  7656 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:28.848904  9134 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.849479  9003 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:28.851248  9137 ts_tablet_manager.cc:933] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:28.851666  8414 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:28.862425  9109 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.723207  9127 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:28.872613  7730 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:28.875646  7731 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:28.890347  9110 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.884603  9019 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:25:28.899466  9111 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:25:28.921797  9136 tablet_copy_client.cc:323] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
W20260709 12:25:28.958225  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 59666.9us
I20260709 12:25:28.961210  9137 tablet_copy_client.cc:323] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:28.985136  7735 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:28.988770  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:29.004022  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57345.5us
I20260709 12:25:29.056895  7656 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:29.078768  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107528us
I20260709 12:25:29.086170  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 3e5b27e8abd04f2ea19a70a13d1958ec from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:29.091979  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 3e5b27e8abd04f2ea19a70a13d1958ec from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-3e5b27e8abd04f2ea19a70a13d1958ec
I20260709 12:25:29.094255  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 40860756ca234260bb6147416168cd6a from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:29.094952  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 40860756ca234260bb6147416168cd6a from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-40860756ca234260bb6147416168cd6a
I20260709 12:25:29.097219  9135 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:29.182927  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 87586.2us
W20260709 12:25:29.194238  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 74548.9us
W20260709 12:25:29.256392  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69082.1us
I20260709 12:25:29.263931  7822 tablet_copy_source_session.cc:215] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:29.309162  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b27e8abd04f2ea19a70a13d1958ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:29.343255  9143 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.280s	user 0.002s	sys 0.004s
W20260709 12:25:29.429661  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 182746us
W20260709 12:25:29.490098  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59208.7us
I20260709 12:25:29.492399  7672 tablet_copy_source_session.cc:215] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:25:29.567729  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.018s
W20260709 12:25:29.571441  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.018s
W20260709 12:25:29.571985  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 571467us
I20260709 12:25:29.591311  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40860756ca234260bb6147416168cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:29.626142  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 105387us
I20260709 12:25:29.581789  9151 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 12:25:29.664255  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45377.2us
W20260709 12:25:29.683708  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51397.2us
W20260709 12:25:29.783054  9168 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:25:29.725602  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45519.3us
W20260709 12:25:29.843598  9158 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.272s	user 0.001s	sys 0.001s
W20260709 12:25:29.840862  9127 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:25:29.974738  9127 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:25:30.008612  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 375967us
W20260709 12:25:30.049221  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 324201us
I20260709 12:25:30.112468  9127 tablet_bootstrap.cc:492] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:30.113584  9127 ts_tablet_manager.cc:1403] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 1.391s	user 0.042s	sys 0.024s
I20260709 12:25:30.128923  9137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40588 (request call id 374) took 1296 ms. Trace:
I20260709 12:25:30.129508  9137 rpcz_store.cc:276] 0709 12:25:28.832519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.832709 (+   190us) service_pool.cc:224] Handling call
0709 12:25:28.851615 (+ 18906us) ts_tablet_manager.cc:934] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
0709 12:25:29.298159 (+446544us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:29.498587 (+200428us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:29.498730 (+   143us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:30.037955 (+539225us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:30.114695 (+ 76740us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:30.128739 (+ 14044us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:30.128812 (+    73us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":39213,"tablet-copy.queue_time_us":12702,"thread_start_us":79806,"threads_started":2}
I20260709 12:25:30.140650  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:30.141508  7659 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:30.163826  9137 tablet_copy_client.cc:806] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:30.166111  9137 tablet_copy_client.cc:670] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:30.118708  9127 raft_consensus.cc:359] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:30.214151  9137 tablet_copy_client.cc:538] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:30.228013  7735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:30.228751  7735 raft_consensus.cc:606] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:30.207316  9127 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:30.259581  9127 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:30.309587  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:30.310299  7808 raft_consensus.cc:606] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:30.318166  9127 ts_tablet_manager.cc:1434] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.204s	user 0.010s	sys 0.000s
I20260709 12:25:30.321131  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:25:30.324317  9136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40594 (request call id 299) took 1604 ms. Trace:
I20260709 12:25:30.335505  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-0733d7d23d9041e09a5ade18807c383e received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:30.336195  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-0733d7d23d9041e09a5ade18807c383e on tablet 0733d7d23d9041e09a5ade18807c383e with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:30.381129  9137 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:30.386649  9159 raft_consensus.cc:993] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:30.387257  9159 raft_consensus.cc:1081] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:30.390151  7732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:30.394608  7732 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:30.395183  7732 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.389999  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:30.403815  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:30.430099  7732 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.443354  7732 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:30.447521  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:30.448251  7656 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:30.448580  7656 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:30.449373  7656 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.450640  7656 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.456177  9185 raft_consensus.cc:993] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:30.456796  9185 raft_consensus.cc:1081] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:30.463451  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:30.464216  7808 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.329656  9136 rpcz_store.cc:276] 0709 12:25:28.719722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.719904 (+   182us) service_pool.cc:224] Handling call
0709 12:25:28.844924 (+125020us) ts_tablet_manager.cc:934] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
0709 12:25:29.498820 (+653896us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:30.123913 (+625093us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:30.124043 (+   130us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:30.318956 (+194913us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:30.320309 (+  1353us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:30.323483 (+  3174us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:30.324205 (+   722us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":57335,"tablet-copy.queue_time_us":19813,"thread_start_us":35732,"threads_started":2}
I20260709 12:25:30.469910  9136 tablet_copy_client.cc:806] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:30.470877  9136 tablet_copy_client.cc:670] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:30.484601  7807 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:30.485236  7807 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:30.485644  7807 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.489135  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:30.490698  9136 tablet_copy_client.cc:538] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:30.490749  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:30.498396  9166 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:30.503322  8411 raft_consensus.cc:1217] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:30.506059  7807 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.499030  9166 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:30.510798  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:30.513749  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:30.514479  7732 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:30.514906  7732 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:30.517979  9136 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:30.513481  7807 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:30.532045  7732 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.532711  9166 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.533574  7732 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.538935  7656 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:30.554736  7808 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:30.572480  7656 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:30.574098  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:30.582350  7732 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:30.584602  9147 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:30.590071  8980 raft_consensus.cc:993] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:30.590690  8980 raft_consensus.cc:1081] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:30.596928  7656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:30.598855  7656 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:30.599705  7656 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:30.585307  9147 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:30.604923  9147 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.616374  7656 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.619810  7656 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:30.621356  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:30.621519  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:30.621986  7735 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:30.622232  7806 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:30.622609  7806 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:30.627851  7806 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.629949  7806 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:30.631412  7735 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:30.684209  7627 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:30.737102  9144 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:30.738127  9144 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:30.791316  9144 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:30.836126  7557 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:30.892499  9137 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:30.895756  7806 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:30.930764  7555 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.957362  9137 tablet_bootstrap.cc:492] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:30.959419  9137 ts_tablet_manager.cc:1403] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.579s	user 0.025s	sys 0.017s
I20260709 12:25:31.022181  7804 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:31.046752  9163 raft_consensus.cc:1064] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:31.050907  9163 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:31.084318  8414 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:31.090222  9137 raft_consensus.cc:359] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:31.107434  9137 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:31.108327  9137 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:31.096699  9136 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:31.097180  7558 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.098855  9019 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:31.099888  9163 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:31.129632  8414 raft_consensus.cc:1217] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:31.136341  7658 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:31.138862  9136 tablet_bootstrap.cc:492] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:31.139832  9136 ts_tablet_manager.cc:1403] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.622s	user 0.046s	sys 0.001s
I20260709 12:25:31.141752  9137 ts_tablet_manager.cc:1434] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.181s	user 0.008s	sys 0.003s
I20260709 12:25:31.147060  7807 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:31.174695  7660 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:31.176932  7659 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:31.234524  9194 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:25:31.236519  9109 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:25:31.237066  9085 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:31.289654  9185 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:25:31.416795  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-3e5b27e8abd04f2ea19a70a13d1958ec received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:31.417687  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-3e5b27e8abd04f2ea19a70a13d1958ec on tablet 3e5b27e8abd04f2ea19a70a13d1958ec with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:31.385524  9136 raft_consensus.cc:359] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:31.463202  9136 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:31.464224  9136 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:31.437773  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115796us
I20260709 12:25:31.516522  9136 ts_tablet_manager.cc:1434] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.376s	user 0.018s	sys 0.003s
I20260709 12:25:31.530092  7807 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:31.540632  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122837us
I20260709 12:25:31.542201  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-40860756ca234260bb6147416168cd6a received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:31.542996  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-40860756ca234260bb6147416168cd6a on tablet 40860756ca234260bb6147416168cd6a with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:31.555874  7732 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:31.629696  9003 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:25:31.626247  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55395.9us
W20260709 12:25:31.630465  9218 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:31.654814  9212 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:25:31.694026  7732 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:31.661528  9163 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:31.715777  9169 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:31.795828  9147 raft_consensus.cc:1064] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:31.799043  9147 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:31.803084  8407 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:31.874296  7659 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:31.886358  7811 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:31.925549  8411 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:31.936205  8980 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:31.938488  7730 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:31.960652  9146 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:25:31.978451  8411 raft_consensus.cc:1217] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:31.982664  7655 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:32.006657  9147 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:32.011104  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:32.001822  7557 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.099010  9110 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.101192  7655 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:32.167268  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51056.7us
I20260709 12:25:32.171682  8407 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.187865  7732 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.225829  7729 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.231632  7558 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index 2 to 3, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.236948  8414 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:32.249719  7659 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:32.260416  9221 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:32.282419  9185 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:32.323930  9221 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.333855  8407 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.342303  7659 raft_consensus.cc:2955] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.357964  7807 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.366729  7558 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 1 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.385354  8407 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:32.407951  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 0733d7d23d9041e09a5ade18807c383e with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45822
I20260709 12:25:32.427337  7731 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:32.429507  9110 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:25:32.433179  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53888.7us
I20260709 12:25:32.439549  9147 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.441883  7732 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.466446  9212 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:25:32.519340  9240 tablet_replica.cc:333] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:32.520838  9240 raft_consensus.cc:2243] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:32.521777  9240 raft_consensus.cc:2272] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:32.522786  9176 raft_consensus.cc:1064] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:32.529881  7546 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0733d7d23d9041e09a5ade18807c383e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:32.539119  8414 raft_consensus.cc:2955] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.541150  9176 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.538923  9240 ts_tablet_manager.cc:1916] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:32.560288  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 83662us
W20260709 12:25:32.564239  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80860.8us
I20260709 12:25:32.578493  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e5b27e8abd04f2ea19a70a13d1958ec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:32.615877  7555 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 1 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.638459  8414 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:32.642437  7807 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:32.648141  9003 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:32.653990  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e5b27e8abd04f2ea19a70a13d1958ec with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45904
W20260709 12:25:32.665354  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103302us
I20260709 12:25:32.671859  7730 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:32.664319  9227 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:25:32.682687  7731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:32.705636  7731 raft_consensus.cc:606] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:32.717548  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:32.718276  7659 raft_consensus.cc:606] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:32.721410  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55501.1us
I20260709 12:25:32.729230  9241 raft_consensus.cc:993] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:32.729928  9241 raft_consensus.cc:1081] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:32.735921  9249 tablet_replica.cc:333] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:25:32.737970  9249 raft_consensus.cc:2243] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:32.738762  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:32.739285  9249 raft_consensus.cc:2272] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:32.739590  7810 raft_consensus.cc:606] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:32.742700  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:25:32.746662  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:32.754127  7731 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:32.754894  7731 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.755314  7731 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.748241  9242 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:25:32.762269  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:32.781934  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80634.8us
I20260709 12:25:32.795739  8411 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.789659  9242 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.832509  9249 ts_tablet_manager.cc:1916] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:32.835767  9251 raft_consensus.cc:993] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:32.836493  9185 raft_consensus.cc:1081] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:32.844491  7810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:32.845213  7810 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.845680  7810 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.780521  7731 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:32.854684  7555 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.859970  7811 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:32.878131  7731 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:32.880776  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:32.881659  7659 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:32.882009  7659 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:32.882666  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:32.883415  7807 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.886075  7659 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:32.887427  7659 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.914016  7655 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.913983  7810 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.927541  7659 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:32.931409  7810 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:32.932245  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:32.932989  8411 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.936319  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:32.937191  7730 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:32.937577  7730 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:32.945990  9110 raft_consensus.cc:993] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:32.946633  9110 raft_consensus.cc:1081] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:32.938457  7730 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:32.974272  7654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:32.975108  7654 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.975590  7654 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.975704  9240 ts_tablet_manager.cc:1929] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:32.976403  9240 log.cc:1199] T 0733d7d23d9041e09a5ade18807c383e P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/0733d7d23d9041e09a5ade18807c383e
I20260709 12:25:32.978608  8414 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:32.981611  8411 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:32.985050  7807 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:32.999426  7730 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.993166  9250 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:25:33.008267  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:33.009439  9147 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:33.010061  9147 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:33.011206  9147 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:33.016242  7811 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:33.025920  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 0733d7d23d9041e09a5ade18807c383e (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:33.026587  9003 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:33.004509  7654 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:33.028563  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:33.038447  9211 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:33.034684  7807 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:33.047716  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:33.048437  7811 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:33.048743  7811 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:33.038543  7654 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:33.053611  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:33.089320  7735 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.053790  7730 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:33.027486  9227 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:33.046316  9211 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:33.061498  7811 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:33.062177  8414 raft_consensus.cc:2955] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:33.094798  7811 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.094733  9211 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:33.115841  7811 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:33.144399  7558 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 4 to 5, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:33.158432  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 40860756ca234260bb6147416168cd6a with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:33.173107  7558 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.176970  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:33.180215  9249 ts_tablet_manager.cc:1929] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:33.201895  9249 log.cc:1199] T 3e5b27e8abd04f2ea19a70a13d1958ec P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/3e5b27e8abd04f2ea19a70a13d1958ec
I20260709 12:25:33.204200  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 3e5b27e8abd04f2ea19a70a13d1958ec (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:33.208930  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:33.214686  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 40860756ca234260bb6147416168cd6a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:33.209942  9242 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:33.219815  9242 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:33.225222  9242 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:33.248567  7559 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.286396  9258 tablet_replica.cc:333] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:33.287763  9258 raft_consensus.cc:2243] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:33.288659  9258 pending_rounds.cc:70] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Trying to abort 1 pending ops.
I20260709 12:25:33.289122  9258 pending_rounds.cc:77] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7305625323660345344 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "40860756ca234260bb6147416168cd6a" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } } }
I20260709 12:25:33.290701  9258 raft_consensus.cc:2917] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20260709 12:25:33.291292  9258 raft_consensus.cc:2889] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:25:33.291806  9258 raft_consensus.cc:2272] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:33.305107  9258 ts_tablet_manager.cc:1916] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:33.339164  7559 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.353554  9258 ts_tablet_manager.cc:1929] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:33.354219  9258 log.cc:1199] T 40860756ca234260bb6147416168cd6a P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/40860756ca234260bb6147416168cd6a
I20260709 12:25:33.356848  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 40860756ca234260bb6147416168cd6a (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:33.410650  7731 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:33.412737  9257 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:33.481029  8414 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:33.501217  9257 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:25:33.594208  7659 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.596189  9211 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:33.645264  7808 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.656194  9221 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:33.718436  7808 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.724134  9250 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:33.751785  7731 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.753921  9227 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:33.909030  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:34.011457  7654 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:34.012251  7654 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:34.021727  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:34.022423  7803 raft_consensus.cc:606] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:34.028723  7730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:34.029484  7730 raft_consensus.cc:606] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:34.059695  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:34.060549  7803 raft_consensus.cc:606] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:34.065573  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:34.067550  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:34.068482  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:34.089259  9257 raft_consensus.cc:993] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:34.089910  9257 raft_consensus.cc:1081] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:34.092167  8409 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40588
I20260709 12:25:34.093037  8409 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.093528  8409 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.108515  9211 raft_consensus.cc:993] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:34.109220  9211 raft_consensus.cc:1081] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:34.113556  7803 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:34.114287  7803 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.114739  7803 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.133728  7803 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.137602  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:34.137847  7803 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:34.138305  7654 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.134301  8409 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.139171  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:34.139906  7730 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:34.140254  7730 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:34.141106  7730 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.142951  7730 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.144915  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:34.145622  7735 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.153842  7735 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:34.156358  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:34.156603  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:34.157083  7808 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:34.157426  7808 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:34.157569  9153 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:34.158118  9153 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:34.158339  7808 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.159094  9153 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.160806  7654 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:34.164247  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:34.167179  9257 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:34.168907  9257 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:34.169902  9257 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.168335  7808 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.155054  8409 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:34.185418  7730 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:34.208248  7559 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:34.217545  7558 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:34.218387  7808 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:34.386320  9271 raft_consensus.cc:993] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:34.386950  9271 raft_consensus.cc:1081] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:34.389386  7731 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:34.390169  7731 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.390661  7731 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.415771  7731 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.419257  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:34.420064  7654 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:34.420485  7654 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:34.421553  7654 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.423029  7654 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.431288  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:34.432016  7806 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.439386  7731 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:34.443905  7654 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:25:34.446282  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:34.449885  7806 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:25:34.450804  9211 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:34.453428  9211 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:34.454592  9211 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:34.468997  9276 raft_consensus.cc:993] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:34.469741  9276 raft_consensus.cc:1081] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:34.472530  7654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:34.473443  7654 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.474381  7654 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.480295  7735 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:34.492159  7558 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:34.515352  7654 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.519323  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:34.520112  7806 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:34.520684  7806 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:34.521643  7806 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.522895  7654 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:34.522069  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:34.523124  7806 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.523635  8409 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.539744  7808 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:34.540099  7806 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:34.542104  9211 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.542402  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:34.543718  7659 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:34.545109  9271 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:34.545729  9271 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:25:34.546944  7700 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 24790f3be4e8465581029a2e0d526117. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:34.546757  9271 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:34.551240  8409 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:34.552928  9211 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.580103  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:34.588115  9248 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:34.597160  7662 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:34.603011  9214 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:34.848399  9283 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:25:34.872018  7735 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:34.892750  7731 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:25:34.957926  9285 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:25:34.949796  9248 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:25:34.933203  9298 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:25:34.967710  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 333989us
I20260709 12:25:34.976109  7558 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:34.982182  9300 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:35.051359  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115325us
I20260709 12:25:35.055896  7804 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:35.116981  9315 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:25:35.124984  7656 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:35.153357  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 85663.4us
I20260709 12:25:35.155709  7806 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:35.158850  8414 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:35.174157  9318 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:25:35.183689  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 106637us
W20260709 12:25:35.188918  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 124014us
I20260709 12:25:35.201483  7656 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:35.212666  7806 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:35.236512  9312 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:35.237809  9251 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:35.242959  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:35.266050  7558 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:35.284574  7659 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:35.325030  9325 ts_tablet_manager.cc:933] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:35.311757  9251 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:25:35.388782  9325 tablet_copy_client.cc:323] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
W20260709 12:25:35.414954  7624 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d997d33a05304f31b50341cc8c4bbf9b. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:35.419696  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110158us
I20260709 12:25:35.430847  7811 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:35.442080  9312 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:35.464238  7733 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:35.506116  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:35.512406  7659 raft_consensus.cc:606] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
W20260709 12:25:35.513069  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53638.5us
I20260709 12:25:35.572192  9307 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:25:35.557036  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 24790f3be4e8465581029a2e0d526117 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:25:35.580816  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 24790f3be4e8465581029a2e0d526117 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-24790f3be4e8465581029a2e0d526117
I20260709 12:25:35.593860  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:35.594662  7659 raft_consensus.cc:606] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:35.607853  9303 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:35.667696  7747 tablet_copy_source_session.cc:215] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:35.675132  9325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24790f3be4e8465581029a2e0d526117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:35.647743  7803 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:35.699065  7733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "539d703b24874183847fa9a8889111e9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:35.701915  7733 raft_consensus.cc:606] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:35.713088  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:35.713864  7804 raft_consensus.cc:606] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:35.716288  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:35.718607  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:35.719616  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:35.725944  9312 raft_consensus.cc:993] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:35.726632  9312 raft_consensus.cc:1081] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:35.729305  9290 raft_consensus.cc:993] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:35.729914  9290 raft_consensus.cc:1081] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:35.732043  7656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "539d703b24874183847fa9a8889111e9"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:35.732810  7656 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.733240  7656 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.734794  7728 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:35.738780  7811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:35.739524  7811 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.739941  7811 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.746914  9325 tablet_copy_client.cc:806] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:35.747741  9325 tablet_copy_client.cc:670] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:35.756381  9325 tablet_copy_client.cc:538] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:35.770231  7555 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:35.775876  7607 auto_rebalancer.cc:629] Failed to schedule move for tablet 539d703b24874183847fa9a8889111e9: Remote error: Service unavailable: leader transfer in progress
I20260709 12:25:35.777081  9325 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:35.793972  9317 raft_consensus.cc:993] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:35.794633  9318 raft_consensus.cc:1081] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:35.794961  7804 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:35.797262  7733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:35.798089  7733 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.798594  7733 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.910261  7656 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.929117  7811 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.932811  7811 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:35.936981  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:35.937728  7659 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:35.938074  7659 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:35.939702  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:35.940387  7731 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.934405  7733 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.938830  7659 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.948074  7659 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.949191  7733 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:35.951787  7656 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:35.952605  9325 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:35.953692  9325 tablet_bootstrap.cc:492] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:35.954546  9325 ts_tablet_manager.cc:1403] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.178s	user 0.023s	sys 0.051s
I20260709 12:25:35.956818  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:35.957960  7728 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.958474  7728 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:25:35.959995  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 137277us
I20260709 12:25:35.962190  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:35.962921  7662 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:35.963274  7662 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:35.964120  7662 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.965338  7662 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.969322  7728 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:35.970755  7728 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.984848  9325 raft_consensus.cc:359] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:35.985863  9325 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:35.986804  9325 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:35.989892  9325 ts_tablet_manager.cc:1434] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:25:35.990329  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:35.990998  7804 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.992919  7747 tablet_copy_service.cc:342] P fc963afbbf4c4b34b161448ef79bd2ed: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-24790f3be4e8465581029a2e0d526117 received from {username='slave'} at 127.0.0.1:53564
I20260709 12:25:35.993455  7747 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-24790f3be4e8465581029a2e0d526117 on tablet 24790f3be4e8465581029a2e0d526117 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:36.000718  7731 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:36.001390  9325 ts_tablet_manager.cc:933] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:36.002655  7656 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:36.002918  7659 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:36.007835  9306 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.012975  7728 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:36.026221  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:36.029911  7662 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:36.044426  9306 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:36.058776  9306 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:36.059916  9325 tablet_copy_client.cc:323] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:36.060092  9306 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.061970  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet d997d33a05304f31b50341cc8c4bbf9b from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:36.062649  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet d997d33a05304f31b50341cc8c4bbf9b from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-d997d33a05304f31b50341cc8c4bbf9b
I20260709 12:25:36.082216  9293 raft_consensus.cc:993] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:36.084379  9293 raft_consensus.cc:1081] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:36.088474  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:36.091902  7658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:36.094261  7728 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:36.099413  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:36.100950  7658 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:36.102795  7658 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.104951  7672 tablet_copy_source_session.cc:215] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:35.999068  7804 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:36.117569  9336 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:36.120723  9309 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.121299  9325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997d33a05304f31b50341cc8c4bbf9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:36.127038  9336 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:36.116878  7658 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.134413  9336 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.117671  9317 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:36.137887  9317 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:36.139510  7658 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:36.168884  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:36.169479  7729 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.181361  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:36.182049  7808 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.182140  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:36.138937  9317 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.189877  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:36.195436  9312 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:36.197890  7559 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:36.219506  7808 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:36.240957  8414 raft_consensus.cc:1217] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:36.265376  9312 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:36.267094  9312 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.269495  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:36.270460  7803 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.270874  7803 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:36.271881  7803 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:36.273212  7803 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.263132  9341 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:36.288498  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62830.3us
I20260709 12:25:36.291079  7728 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:36.304039  7729 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:25:36.332279  7775 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39c84f7f7e484afbae3c398efd5924b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:36.318001  7559 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:36.394449  9325 tablet_copy_client.cc:806] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:36.395197  9325 tablet_copy_client.cc:670] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
W20260709 12:25:36.410800  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57825.5us
I20260709 12:25:36.420776  7555 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:36.427788  7803 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:36.428213  7559 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:36.478250  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 133104us
W20260709 12:25:36.489512  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 128472us
I20260709 12:25:36.493456  9342 ts_tablet_manager.cc:933] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:36.509630  9325 tablet_copy_client.cc:538] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:36.518708  7558 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:36.526860  9325 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:36.540095  9342 tablet_copy_client.cc:323] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:25:36.545151  7658 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.554384  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 39c84f7f7e484afbae3c398efd5924b4 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:25:36.555061  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 39c84f7f7e484afbae3c398efd5924b4 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-39c84f7f7e484afbae3c398efd5924b4
I20260709 12:25:36.564319  9327 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.567422  7658 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:36.570446  7822 tablet_copy_source_session.cc:215] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:36.579058  9342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c84f7f7e484afbae3c398efd5924b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:36.638813  9342 tablet_copy_client.cc:806] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:36.639835  9342 tablet_copy_client.cc:670] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:36.653944  9342 tablet_copy_client.cc:538] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:36.677263  9342 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:36.695562  7733 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.703102  9306 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:36.705840  7658 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.710475  9357 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.743526  7804 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:36.746734  9348 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:36.787304  7731 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:36.814018  7804 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.821317  9336 raft_consensus.cc:1064] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:36.821264  9312 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:36.824695  9336 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:36.846840  9221 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.859918  7808 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.863044  9348 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:36.902823  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56625.7us
I20260709 12:25:36.917793  8414 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:36.921671  9211 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:36.936177  7733 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.951349  9325 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:36.952625  9325 tablet_bootstrap.cc:492] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:36.955341  9342 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:37.053649  7806 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:37.066540  9325 ts_tablet_manager.cc:1403] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.540s	user 0.044s	sys 0.005s
I20260709 12:25:37.087658  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.132201  7658 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:37.134244  9342 tablet_bootstrap.cc:492] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
W20260709 12:25:37.168238  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84271.1us
W20260709 12:25:37.181058  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 92359.8us
I20260709 12:25:37.183139  9211 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:37.196978  9342 ts_tablet_manager.cc:1403] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.520s	user 0.037s	sys 0.019s
W20260709 12:25:37.248968  9375 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
W20260709 12:25:37.250144  9374 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.163s	user 0.004s	sys 0.000s
W20260709 12:25:37.351292  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138634us
I20260709 12:25:37.368383  9317 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:25:37.401283  9342 raft_consensus.cc:359] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:37.409457  9342 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:25:37.410408  9342 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:37.416580  9342 ts_tablet_manager.cc:1434] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.202s	user 0.007s	sys 0.000s
I20260709 12:25:37.481834  9325 raft_consensus.cc:359] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:37.484267  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-39c84f7f7e484afbae3c398efd5924b4 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:25:37.485095  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-39c84f7f7e484afbae3c398efd5924b4 on tablet 39c84f7f7e484afbae3c398efd5924b4 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:37.482928  9325 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
W20260709 12:25:37.543310  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 176242us
I20260709 12:25:37.544947  9211 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:37.560166  8407 raft_consensus.cc:1217] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:37.535156  9325 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:37.598798  9325 ts_tablet_manager.cc:1434] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.507s	user 0.015s	sys 0.000s
W20260709 12:25:37.634649  9325 ts_tablet_manager.cc:1467] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Tablet startup took 543ms
I20260709 12:25:37.639541  7729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:37.640466  7729 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:25:37.636976  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48794.1us
W20260709 12:25:37.637960  9325 ts_tablet_manager.cc:1469] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Trace:
0709 12:25:35.968038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.974671 (+  6633us) service_pool.cc:224] Handling call
0709 12:25:36.001803 (+ 27132us) ts_tablet_manager.cc:934] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
0709 12:25:36.114040 (+112237us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:36.229508 (+115468us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.229639 (+   131us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:36.356973 (+127334us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:36.380871 (+ 23898us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:36.394142 (+ 13271us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:36.394211 (+    69us) inbound_call.cc:177] Queueing success response
0709 12:25:36.526446 (+132235us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:25:37.092055 (+565609us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:25:37.353902 (+261847us) tablet_replica.cc:261] Starting instrumentation
0709 12:25:37.479999 (+126097us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":250,"raft.queue_time_us":2310,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":3868,"tablet-copy.queue_time_us":26189,"tablet-download-pool.queue_time_us":1247,"tablet-download-pool.run_cpu_time_us":1220,"tablet-download-pool.run_wall_time_us":112586,"thread_start_us":106827,"threads_started":2}
I20260709 12:25:37.672263  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-d997d33a05304f31b50341cc8c4bbf9b received from {username='slave'} at 127.0.0.1:40408
I20260709 12:25:37.672902  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-d997d33a05304f31b50341cc8c4bbf9b on tablet d997d33a05304f31b50341cc8c4bbf9b with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:37.756906  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "268874518d1d4544a8500be048628b67"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:37.757616  7662 raft_consensus.cc:606] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:37.761188  9382 raft_consensus.cc:993] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:37.761749  9382 raft_consensus.cc:1081] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:37.768939  7662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:37.769757  7662 raft_consensus.cc:606] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:37.777266  9381 raft_consensus.cc:993] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:37.777920  9381 raft_consensus.cc:1081] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:37.781649  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:37.782383  7806 raft_consensus.cc:606] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:37.785303  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:37.787247  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:37.788157  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.788877  7806 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:37.789543  7806 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:37.789932  7806 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.806208  9336 raft_consensus.cc:993] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:37.806821  9336 raft_consensus.cc:1081] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:37.808112  9327 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:37.833400  7806 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:37.836026  7806 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:37.836759  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:37.837497  7662 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:37.837891  7662 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:37.844511  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:37.845266  8407 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.848655  8409 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:37.851475  7559 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.838824  7662 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:37.864697  7662 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.829939  7654 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:37.909826  7806 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:37.912737  7662 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:37.914537  7662 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:37.922667  7728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "268874518d1d4544a8500be048628b67"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:37.923764  7728 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:37.924288  7728 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:37.914762  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:37.957631  8408 raft_consensus.cc:1217] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:25:37.934509  9350 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:37.976781  7662 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:37.977494  7662 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.024281  7662 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.056877  7728 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:38.063284  7728 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:38.074704  7737 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:38.091089  8407 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:38.122586  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:38.123667  7804 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.036738  9350 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:25:38.212035  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118555us
I20260709 12:25:38.213025  7804 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:38.220032  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:38.220710  7658 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:38.221076  7658 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:38.221863  7658 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:38.222816  7658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.160609  9350 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:38.225386  9404 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:25:38.240221  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:38.264603  9407 raft_consensus.cc:993] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:38.265228  9407 raft_consensus.cc:1081] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:38.274127  7732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
I20260709 12:25:38.274896  7732 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:38.275352  7732 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.294874  7732 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.298367  7732 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:38.309701  7662 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:38.370357  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102267us
I20260709 12:25:38.381647  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:38.382663  8409 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.385272  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:38.385952  8414 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.400897  7808 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:38.402787  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:38.403486  7811 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.430392  7811 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:38.431588  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129224us
I20260709 12:25:38.432723  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:38.448938  9395 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:38.449607  9395 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:38.449769  9421 raft_consensus.cc:1064] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:38.460098  9395 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:38.474267  8414 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:38.475368  8413 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:38.478063  7808 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:38.480235  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:38.481009  7806 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:38.481323  7806 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:38.482136  7806 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.486493  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:38.487295  7728 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:38.487571  7806 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.487656  7728 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:38.496013  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:38.488821  7728 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.501122  9413 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:38.501780  9413 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:38.502736  9413 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.532298  7728 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.499641  7806 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:38.431170  8409 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:38.544615  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:38.546305  9317 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:38.547304  9317 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:38.548660  9317 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:38.575337  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.583921  7728 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:38.593804  9317 raft_consensus.cc:1064] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:25:38.598073  9317 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:38.662184  9421 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:39.421468  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.452669  9410 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:39.453891  9410 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:39.461229  9410 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:39.522753  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 251) took 1674 ms. Trace:
I20260709 12:25:39.532763  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 273) took 1305 ms. Trace:
I20260709 12:25:39.535449  7557 rpcz_store.cc:276] 0709 12:25:38.227090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.227328 (+   238us) service_pool.cc:224] Handling call
0709 12:25:39.532668 (+1305340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23238,"reported_tablets":1}
I20260709 12:25:39.540928  7811 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:25:39.546072  7546 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 24790f3be4e8465581029a2e0d526117 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica fc963afbbf4c4b34b161448ef79bd2ed is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:39.533663  7559 rpcz_store.cc:276] 0709 12:25:37.848035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.848197 (+   162us) service_pool.cc:224] Handling call
0709 12:25:39.522632 (+1674435us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:37.868283 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:37.868929 (+   646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:37.868970 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:25:37.868987 (+    17us) tablet.cc:662] Decoding operations
0709 12:25:37.869356 (+   369us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:37.869436 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:37.869477 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:37.869676 (+   199us) tablet.cc:701] Row locks acquired
0709 12:25:37.869694 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:25:37.869868 (+   174us) write_op.cc:270] Start()
0709 12:25:37.869997 (+   129us) write_op.cc:276] Timestamp: P: 1783599937869810 usec, L: 0
0709 12:25:37.870019 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:37.873149 (+  3130us) log.cc:844] Serialized 595 byte log entry
0709 12:25:38.384104 (+510955us) op_driver.cc:464] REPLICATION: finished
0709 12:25:38.704448 (+320344us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b180028e400. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:25:38.704624 (+   176us) write_op.cc:301] APPLY: starting
0709 12:25:38.704719 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:38.704867 (+   148us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:38.704901 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:38.705158 (+   257us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:38.705347 (+   189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:38.705405 (+    58us) write_op.cc:312] APPLY: finished
0709 12:25:38.705642 (+   237us) log.cc:844] Serialized 23 byte log entry
0709 12:25:38.706605 (+   963us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:38.706721 (+   116us) write_op.cc:454] Released schema lock
0709 12:25:38.706782 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:38.707807 (+  1025us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:38.707849 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":84771,"apply.run_cpu_time_us":1844,"apply.run_wall_time_us":2594,"num_ops":1,"prepare.queue_time_us":5876,"prepare.run_cpu_time_us":52091,"prepare.run_wall_time_us":138856,"raft.queue_time_us":106281,"raft.run_cpu_time_us":14882,"raft.run_wall_time_us":84285,"replication_time_us":513905,"spinlock_wait_cycles":54636288,"thread_start_us":262626,"threads_started":4,"wal-append.queue_time_us":264553}]]}
I20260709 12:25:39.573303  9292 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:39.573978  9426 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:39.577085  9426 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:39.579710  9426 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:39.584007  8413 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:39.582827  9354 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:25:39.591728  7558 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 288) took 1634 ms. Trace:
I20260709 12:25:39.592250  7558 rpcz_store.cc:276] 0709 12:25:37.957569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.957853 (+   284us) service_pool.cc:224] Handling call
0709 12:25:38.024544 (+ 66691us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001f2af3a 00007fc007ce0e9c 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c 00007fc00025c8c6
0709 12:25:39.591644 (+1567100us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":70261504}
I20260709 12:25:39.601802  7555 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 166) took 1131 ms. Trace:
I20260709 12:25:39.602325  7555 rpcz_store.cc:276] 0709 12:25:38.470535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.668287 (+197752us) service_pool.cc:224] Handling call
0709 12:25:39.601657 (+933370us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:25:39.586423  9292 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:39.642046  8409 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:39.643754  8413 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:39.647864  9380 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:39.649220  9312 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:39.640312  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57882.4us
I20260709 12:25:39.682410  9327 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:39.689815  7806 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:39.690200  9422 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.690945  9422 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:39.695264  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:39.695998  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:39.697469  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:39.703447  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:39.703550  9354 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.708087  9354 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.708926  9354 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.712023  7558 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.726575  7558 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.718555  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:39.733362  7733 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:39.737685  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:39.740931  7732 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:39.743881  7655 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:39.645740  9292 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:39.747249  7656 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:39.746245  7559 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.748595  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:39.749852  9348 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:25:39.735983  7683 debug-util.cc:398] Leaking SignalData structure 0x7b0800085640 after lost signal to thread 6454
W20260709 12:25:39.752298  7683 debug-util.cc:398] Leaking SignalData structure 0x7b08000cbbc0 after lost signal to thread 7593
I20260709 12:25:39.736172  9422 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:39.765571  7728 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:25:39.765559  7683 debug-util.cc:398] Leaking SignalData structure 0x7b080031bee0 after lost signal to thread 7686
I20260709 12:25:39.771863  7811 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:39.772452  7683 debug-util.cc:398] Leaking SignalData structure 0x7b080036b740 after lost signal to thread 7761
W20260709 12:25:39.774757  7683 debug-util.cc:398] Leaking SignalData structure 0x7b0800366a00 after lost signal to thread 7836
W20260709 12:25:39.776223  7683 debug-util.cc:398] Leaking SignalData structure 0x7b080032aa80 after lost signal to thread 8439
I20260709 12:25:39.779873  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:39.781876  9354 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:39.783420  7660 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:39.775031  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:39.791092  9389 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:39.804327  8414 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:39.807492  9354 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:39.807397  9395 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:39.809293  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:39.810110  8413 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.814744  9441 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:39.816828  9441 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:39.817492  9441 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:39.818307  9441 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:39.820005  9441 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.826365  9417 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:39.829313  9439 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:39.830312  8413 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:39.830081  9439 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:39.831832  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:39.832875  9439 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:39.835094  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:39.841544  9354 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:39.842195  8414 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:39.842703  9354 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:39.843936  9354 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:39.863673  9426 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:39.881232  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:39.884521  7808 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:39.885043  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:39.887979  7811 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:25:39.888736  9412 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:39.888904  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:39.887851  9417 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:25:39.924134  9411 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:39.929490  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:39.932785  7702 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:39.937471  9336 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:39.932287  9426 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:39.932143  9412 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
W20260709 12:25:40.087710  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63771.9us
W20260709 12:25:40.084153  9450 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:25:40.111116  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:40.125258  9447 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:40.193732  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:40.212605  9440 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:40.194780  9409 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.301s
I20260709 12:25:40.282487  9473 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.285297  9453 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 12:25:40.289111  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 265237us
I20260709 12:25:40.275331  9354 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:40.293797  9292 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:40.382072  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 62414.3us
W20260709 12:25:40.396219  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 79176.1us
W20260709 12:25:40.400411  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 94733.5us
W20260709 12:25:40.404594  9454 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:40.311331  9354 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:40.306847  9473 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:40.383078  9413 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:40.331650  9292 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:40.484458  9292 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:40.530974  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 141074us
I20260709 12:25:40.463258  9413 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:40.534369  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149838us
W20260709 12:25:40.621677  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.022s
I20260709 12:25:40.603075  9473 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:40.633103  9446 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:25:40.634960  9477 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:40.623708  9354 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:40.645423  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.003s	sys 0.023s
W20260709 12:25:40.669529  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 651253us
I20260709 12:25:40.702950  9413 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:40.707440  9477 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:40.757381  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58630.7us
W20260709 12:25:40.752978  7546 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 24790f3be4e8465581029a2e0d526117 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica fc963afbbf4c4b34b161448ef79bd2ed is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:40.746308  9467 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:40.801905  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 192672us
W20260709 12:25:40.894690  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 350435us
I20260709 12:25:40.884150  9477 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:40.931351  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.019s	sys 0.079s
W20260709 12:25:40.931874  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.019s	sys 0.079s
W20260709 12:25:40.932343  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 537561us
W20260709 12:25:41.047863  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 172916us
W20260709 12:25:41.046222  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 132440us
I20260709 12:25:41.061897  9406 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.109007  8407 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.115833  9406 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.122313  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:41.115197  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60018.7us
I20260709 12:25:41.125257  9458 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:41.119714  9496 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.143963  9458 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.154305  9461 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.155997  9461 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.161114  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.179881  9494 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:41.150164  9496 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.186997  9458 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.193768  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.193929  9458 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:41.195039  9458 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.196604  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.195403  9312 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.196434  7656 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.198320  9458 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.191490  9406 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.199446  9499 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.200567  9499 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.205790  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.180991  9494 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.219261  9496 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.239369  9461 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.236928  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.236970  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.246011  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.255642  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.305307  9494 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:41.329762  9456 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:41.332108  9456 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.358702  9472 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:41.344535  9499 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.366798  9479 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:41.369192  9479 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.380043  9456 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.360014  9472 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.389638  9467 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.392244  9423 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.409729  8414 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.413399  9423 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:41.419975  9503 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 12:25:41.393038  9467 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.426146  9467 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:41.446633  9423 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.473364  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 289) took 1756 ms. Trace:
W20260709 12:25:41.481290  9505 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:25:41.481586  9489 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:41.461557  9472 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:41.491694  9442 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.492714  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.493132  9489 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.494702  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.498759  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.501956  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:41.502974  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.509801  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.513865  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.512806  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.520820  9490 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.521762  9490 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:41.523041  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72021.9us
I20260709 12:25:41.509282  9479 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:41.524190  9479 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.524914  9490 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.492749  9442 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.528615  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.535995  9489 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:41.510264  9472 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.536808  7558 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 252) took 1933 ms. Trace:
I20260709 12:25:41.537456  9472 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.537768  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.226064  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:41.542383  7555 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.543926  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.543593  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 1763) took 1183 ms. Trace:
I20260709 12:25:41.552204  8407 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:41.555383  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.554940  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.555578  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 1762) took 1196 ms. Trace:
I20260709 12:25:41.546691  7659 rpcz_store.cc:276] 0709 12:25:40.359714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.359851 (+   137us) service_pool.cc:224] Handling call
0709 12:25:40.360287 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.531976 (+171689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.532010 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.227633 (+695623us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:25:41.228447 (+   814us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.289761 (+ 61314us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:41.290532 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.543046 (+252514us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001a40d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:41.543108 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.543167 (+    59us) raft_consensus.cc:1685] finished
0709 12:25:41.543181 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.543378 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125118,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7091,"spinlock_wait_cycles":258114176,"thread_start_us":121602,"threads_started":2,"wal-append.queue_time_us":130215}
I20260709 12:25:41.557278  7655 rpcz_store.cc:276] 0709 12:25:40.359399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.360358 (+   959us) service_pool.cc:224] Handling call
0709 12:25:40.360771 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.470828 (+110057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.470879 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.470943 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:40.470994 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.522918 (+1051924us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540028e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:41.523017 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.555355 (+ 32338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252544768}
I20260709 12:25:41.563915  9442 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:41.568670  9506 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:41.569172  9380 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:41.552004  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.537314  7558 rpcz_store.cc:276] 0709 12:25:39.602859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.603054 (+   195us) service_pool.cc:224] Handling call
0709 12:25:41.536551 (+1933497us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.941803 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:40.050514 (+108711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:40.050560 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:25:40.050583 (+    23us) tablet.cc:662] Decoding operations
0709 12:25:40.125967 (+ 75384us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:40.126019 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:40.126064 (+    45us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:40.126442 (+   378us) tablet.cc:701] Row locks acquired
0709 12:25:40.126463 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:25:40.126626 (+   163us) write_op.cc:270] Start()
0709 12:25:40.126825 (+   199us) write_op.cc:276] Timestamp: P: 1783599940126577 usec, L: 0
0709 12:25:40.126867 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:25:40.131685 (+  4818us) log.cc:844] Serialized 918 byte log entry
0709 12:25:40.770916 (+639231us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1de28 00007fc005958581 00007fc005957d58 00007fc00595c9f1 00007fc00595c9a9 00007fc00595c939 00007fc00595c901 00007fc00595bb2d 0000000000388634 0000000000388479
0709 12:25:40.851063 (+ 80147us) op_driver.cc:464] REPLICATION: finished
0709 12:25:40.876832 (+ 25769us) write_op.cc:301] APPLY: starting
0709 12:25:40.882996 (+  6164us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:40.896385 (+ 13389us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:40.897336 (+   951us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:40.910486 (+ 13150us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:40.922707 (+ 12221us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:25:40.922792 (+    85us) write_op.cc:312] APPLY: finished
0709 12:25:40.923123 (+   331us) log.cc:844] Serialized 29 byte log entry
0709 12:25:40.923458 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.923618 (+   160us) write_op.cc:454] Released schema lock
0709 12:25:40.923678 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:40.924201 (+   523us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.924258 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":394224,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":3088,"apply.run_cpu_time_us":3020,"apply.run_wall_time_us":47272,"num_ops":2,"prepare.queue_time_us":175640,"prepare.run_cpu_time_us":27995,"prepare.run_wall_time_us":836537,"replication_time_us":676841,"spinlock_wait_cycles":9026944}]]}
I20260709 12:25:41.586354  9292 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.599457  7803 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:25:41.590065  9479 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.590695  7734 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.475349  7559 rpcz_store.cc:276] 0709 12:25:39.708327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.711992 (+  3665us) service_pool.cc:224] Handling call
0709 12:25:39.742234 (+ 30242us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5400002210. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfe0b2 00007fc007d77759 00007fc007d0fa76 00007fc007ce0d58 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc
0709 12:25:41.462239 (+1720005us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:39.813735 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:39.814390 (+   655us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:39.814438 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:25:39.814458 (+    20us) tablet.cc:662] Decoding operations
0709 12:25:39.814848 (+   390us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:39.814911 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:39.814939 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:39.815140 (+   201us) tablet.cc:701] Row locks acquired
0709 12:25:39.815159 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:39.815344 (+   185us) write_op.cc:270] Start()
0709 12:25:39.815492 (+   148us) write_op.cc:276] Timestamp: P: 1783599939815299 usec, L: 0
0709 12:25:39.815516 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:25:39.816338 (+   822us) log.cc:844] Serialized 533 byte log entry
0709 12:25:40.476597 (+660259us) op_driver.cc:464] REPLICATION: finished
0709 12:25:40.779240 (+302643us) write_op.cc:301] APPLY: starting
0709 12:25:40.779362 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:40.779499 (+   137us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:40.779689 (+   190us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:40.780024 (+   335us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:40.780260 (+   236us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:40.780329 (+    69us) write_op.cc:312] APPLY: finished
0709 12:25:40.780622 (+   293us) log.cc:844] Serialized 23 byte log entry
0709 12:25:40.780862 (+   240us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.780970 (+   108us) write_op.cc:454] Released schema lock
0709 12:25:40.781029 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:40.781482 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:40.781525 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":7227264,"child_traces":[["op",{"apply.queue_time_us":298800,"apply.run_cpu_time_us":2254,"apply.run_wall_time_us":2260,"num_ops":1,"prepare.queue_time_us":61089,"prepare.run_cpu_time_us":17490,"prepare.run_wall_time_us":94763,"raft.queue_time_us":264242,"raft.run_cpu_time_us":13400,"raft.run_wall_time_us":474167,"replication_time_us":600848,"thread_start_us":396022,"threads_started":4,"wal-append.queue_time_us":228100}]]}
I20260709 12:25:41.594027  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.600106  7803 pending_rounds.cc:85] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 5
I20260709 12:25:41.594347  7809 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:41.607861  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79360.8us
I20260709 12:25:41.608614  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.601220  9472 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.623391  9479 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.626152  9479 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.626838  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.635620  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.635864  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.638187  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.638551  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.639674  9376 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:41.639930  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.641672  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.642377  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:41.642472  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.643592  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.645152  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.645998  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.651895  9416 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:41.652184  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.652459  7729 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:41.653435  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.654316  9489 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:41.655938  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.650233  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:41.657502  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.663416  9479 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.664384  9479 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:41.665043  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.666909  9479 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:41.613637  7803 pending_rounds.cc:107] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:25:41.667723  7803 raft_consensus.cc:2889] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.671880  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.648523  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.694801  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.699882  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.702633  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:41.702800  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:41.703852  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.705642  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.705212  7728 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:41.652622  9416 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.707549  9306 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:41.708956  9306 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:41.714599  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.718364  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.731176  9394 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.737068  7778 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.727583  9376 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.738268  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:41.741250  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:41.737185  9306 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.749984  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.753480  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.745128  9419 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:41.767390  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:41.756577  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:41.773353  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.777272  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.750313  9467 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:41.782436  9467 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.782968  9467 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:41.789620  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.802269  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.803910  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.775715  9419 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:41.793232  9292 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:41.816385  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46477.8us
W20260709 12:25:41.817898  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50636.6us
I20260709 12:25:41.751835  9306 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:41.814116  9419 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:41.810276  9442 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:41.837208  9483 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.838528  9477 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.839002  9426 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:41.841001  9508 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:25:41.844287  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85004.2us
I20260709 12:25:41.855201  9412 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:41.850940  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:41.869616  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139644us
I20260709 12:25:41.872474  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:41.875900  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72065.2us
I20260709 12:25:41.858139  9306 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:41.886457  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:41.893711  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:41.903770  9470 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.920804  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:41.922027  9388 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:41.930775  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149455us
I20260709 12:25:41.933702  9211 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.883595  9416 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:41.939487  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 88688.7us
I20260709 12:25:41.942319  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:42.009660  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:42.024253  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:42.026698  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97653.4us
I20260709 12:25:42.029776  9494 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.046293  9467 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.097985  9467 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:42.144285  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 200132us
W20260709 12:25:42.146440  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 204234us
I20260709 12:25:42.148732  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:42.150115  9490 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:42.146656  9211 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:42.046810  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:42.161659  9380 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:42.161857  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 115144us
I20260709 12:25:42.162204  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:42.163022  7737 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.194607  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:42.197012  7734 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:42.190187  9380 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:25:42.211061  9419 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.217319  9494 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:42.224442  9494 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.240999  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:42.259881  9416 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:42.274629  9494 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:42.267967  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:42.285703  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89985.1us
I20260709 12:25:42.291968  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:42.301362  9419 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:42.322625  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:42.330709  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:25:42.333149  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71827.9us
I20260709 12:25:42.352126  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:42.368969  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 167) took 2570 ms. Trace:
I20260709 12:25:42.370085  7557 rpcz_store.cc:276] 0709 12:25:39.798240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.798444 (+   204us) service_pool.cc:224] Handling call
0709 12:25:42.368746 (+2570302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3046,"reported_tablets":1}
I20260709 12:25:42.369652  7555 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 274) took 2740 ms. Trace:
I20260709 12:25:42.362533  9389 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.371881  7555 rpcz_store.cc:276] 0709 12:25:39.629126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.629308 (+   182us) service_pool.cc:224] Handling call
0709 12:25:42.369583 (+2740275us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:41.816167 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:41.872949 (+ 56782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:41.879939 (+  6990us) write_op.cc:435] Acquired schema lock
0709 12:25:41.881459 (+  1520us) tablet.cc:662] Decoding operations
0709 12:25:41.937810 (+ 56351us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:41.953500 (+ 15690us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:41.987878 (+ 34378us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:42.047850 (+ 59972us) tablet.cc:701] Row locks acquired
0709 12:25:42.047909 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:25:42.048100 (+   191us) write_op.cc:270] Start()
0709 12:25:42.048297 (+   197us) write_op.cc:276] Timestamp: P: 1783599942048022 usec, L: 0
0709 12:25:42.048343 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:25:42.049199 (+   856us) log.cc:844] Serialized 595 byte log entry
0709 12:25:42.341227 (+292028us) op_driver.cc:464] REPLICATION: finished
0709 12:25:42.343862 (+  2635us) write_op.cc:301] APPLY: starting
0709 12:25:42.343982 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:42.344108 (+   126us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:42.344302 (+   194us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:42.344639 (+   337us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:42.344884 (+   245us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:42.344956 (+    72us) write_op.cc:312] APPLY: finished
0709 12:25:42.345259 (+   303us) log.cc:844] Serialized 23 byte log entry
0709 12:25:42.359645 (+ 14386us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:42.359763 (+   118us) write_op.cc:454] Released schema lock
0709 12:25:42.359845 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:42.360379 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:42.360414 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":134665,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":2362,"apply.run_cpu_time_us":2399,"apply.run_wall_time_us":16474,"num_ops":1,"prepare.queue_time_us":254371,"prepare.run_cpu_time_us":70123,"prepare.run_wall_time_us":451655,"raft.queue_time_us":30541,"raft.run_cpu_time_us":1157,"raft.run_wall_time_us":13023,"replication_time_us":292669,"thread_start_us":453579,"threads_started":4,"wal-append.queue_time_us":210333}]]}
I20260709 12:25:42.354712  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:42.380123  9419 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:42.380985  9419 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:42.382483  9419 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.387029  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:42.390906  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:42.373790  7559 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 3 to 4, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.416080  7559 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 4 to 5, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.414532  9484 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:42.425388  9312 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:42.425835  9312 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:42.407691  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:42.435636  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:42.442880  7807 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:42.426806  9312 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.448565  9312 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.434314  9423 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:42.459801  7737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:42.461279  7737 raft_consensus.cc:606] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:42.446991  9494 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:42.482137  7555 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.494916  7661 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:42.507580  8414 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:42.525357  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248d73a0462c48fdad073c597b243806"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:42.526136  7732 raft_consensus.cc:606] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:42.527693  7730 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:42.516417  9484 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:42.551954  9388 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:42.559688  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:42.560402  7807 raft_consensus.cc:606] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:42.573348  9306 raft_consensus.cc:993] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:42.574167  9306 raft_consensus.cc:1081] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:42.581641  9479 raft_consensus.cc:993] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:42.587796  9412 raft_consensus.cc:1081] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:42.589011  9479 raft_consensus.cc:993] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:42.589627  9479 raft_consensus.cc:1081] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:42.588625  9494 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.597896  7804 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:42.605461  7804 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "248d73a0462c48fdad073c597b243806"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:42.606186  7804 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:42.606564  7804 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.584024  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:42.611696  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:42.612437  7661 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:42.612521  7732 raft_consensus.cc:606] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:42.612983  7661 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:42.609899  7811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45836
I20260709 12:25:42.614192  7811 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:42.614621  7811 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.619788  7660 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:42.624104  7807 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:42.653513  7559 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.654090  7811 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.662246  7811 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:42.665606  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:42.658304  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:42.666384  7737 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:42.666767  7737 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:42.667591  7737 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.668308  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:42.668821  7737 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.673333  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:42.674108  7657 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.651846  7732 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.686892  9484 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:42.688256  9484 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:42.688717  9484 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:42.689730  9484 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.688496  7804 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.691375  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:42.701330  7804 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:42.701172  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:42.702625  8414 raft_consensus.cc:2955] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:42.704344  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:42.707337  7730 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:42.707813  7730 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:42.716156  7737 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:42.718438  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:42.723469  7661 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.724025  7656 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.728479  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:42.738888  9389 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:42.746392  7661 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:42.725843  9484 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:42.741185  7730 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.760226  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:42.761026  7804 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:42.761466  7804 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:42.769531  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:42.770146  7737 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:42.775767  8415 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:42.757252  9389 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:42.785902  7656 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:42.792671  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:42.792564  7804 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.797880  9441 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:42.814301  7811 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:42.807618  9389 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.818354  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:42.819716  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:42.826511  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 24790f3be4e8465581029a2e0d526117 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
I20260709 12:25:42.821796  9441 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:42.821215  9494 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:25:42.832464  9312 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:42.841254  9312 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:42.864656  9412 raft_consensus.cc:993] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:42.864480  9312 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.867352  9483 raft_consensus.cc:1081] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:42.877203  7656 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:42.841282  8408 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:42.873307  9441 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:42.929172  9417 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
W20260709 12:25:42.921440  9511 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.008s
I20260709 12:25:42.959198  7730 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.971799  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:42.972898  7661 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:42.973464  7661 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:42.992820  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:42.996404  9442 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:43.007630  7732 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:43.010471  7804 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:43.010951  9306 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:43.013775  7732 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:43.015650  7555 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 4 to 6, VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195) evicted. New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:25:43.023196  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49393.3us
I20260709 12:25:43.024556  7730 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:43.039616  9442 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:43.039965  7811 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.038431  7661 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:43.049669  7661 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:43.067497  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:43.068342  8408 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:43.075312  7804 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:43.092085  8408 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:43.064637  9519 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:25:43.094933  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:43.051746  9520 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.113472  9442 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:43.114516  9442 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:43.096678  9474 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:43.138511  9474 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:43.161018  8408 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.169955  9474 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:43.173306  9442 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:43.179850  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:43.197814  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d997d33a05304f31b50341cc8c4bbf9b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:43.201359  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:43.204586  9306 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.224432  7775 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:43.277814  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:43.278920  7735 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:43.279277  7735 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:25:43.291920  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67634.4us
W20260709 12:25:43.301239  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 167065us
I20260709 12:25:43.304343  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:43.304957  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 87466.9us
I20260709 12:25:43.298921  7735 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:43.316249  7735 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:43.316727  7728 raft_consensus.cc:2955] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.328995  8415 raft_consensus.cc:2955] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.305510  9442 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:43.394397  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95088.4us
I20260709 12:25:43.396770  7793 tablet_service.cc:1558] Processing DeleteTablet for tablet 24790f3be4e8465581029a2e0d526117 with delete_type TABLET_DATA_TOMBSTONED (TS d72176ccaed049958ac68f706d4f052d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45822
W20260709 12:25:43.405768  9542 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:43.421072  7557 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.449875  7557 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:43.468263  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 123094us
I20260709 12:25:43.485582  7735 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:25:43.495548  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 153784us
I20260709 12:25:43.498864  7658 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:43.626300  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 261527us
I20260709 12:25:43.640599  9567 tablet_replica.cc:333] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:25:43.642151  9567 raft_consensus.cc:2243] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:43.630753  9526 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:43.643613  9567 raft_consensus.cc:2272] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:43.672483  9306 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45822 (request call id 78) took 1074 ms. Trace:
I20260709 12:25:43.673002  9306 rpcz_store.cc:276] 0709 12:25:42.598485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.598604 (+   119us) service_pool.cc:224] Handling call
0709 12:25:42.652922 (+ 54318us) log.cc:844] Serialized 494 byte log entry
0709 12:25:42.799532 (+146610us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b180032b180. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:25:42.871671 (+ 72139us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b54002c8920. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e97dbe 00007fc001ea2930 00007fc001ea28d8 00007fc001ea2858 00007fc001ea2813 00007fc001ea1a9c 00007fc007d8d2f6 00007fc007d87aa8 00007fc001ede7cb 00007fc001ee12c0
0709 12:25:43.672338 (+800667us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2271,"raft.queue_time_us":23569,"raft.run_cpu_time_us":2093,"raft.run_wall_time_us":77668,"spinlock_wait_cycles":16193664,"thread_start_us":58324,"threads_started":1,"wal-append.queue_time_us":32536}
I20260709 12:25:43.678740  9567 ts_tablet_manager.cc:1916] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:43.691578  9456 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:43.692495  9456 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:43.698009  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:43.699579  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 149348us
I20260709 12:25:43.704876  7732 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:43.706499  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.710232  8408 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:43.700305  9456 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:43.714028  7730 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:43.715751  9494 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:43.717336  9568 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:43.715109  7732 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:43.723479  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 39c84f7f7e484afbae3c398efd5924b4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:43.726672  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:43.726961  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:43.735188  7811 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:43.738909  7662 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:43.740134  7659 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:43.738023  9547 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:43.757310  9520 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:43.826460  7555 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 4 to 5, VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) evicted. New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.892736  9409 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:43.910682  9547 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:43.897435  9552 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:43.950383  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 210116us
I20260709 12:25:43.956291  7557 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 5 to 6, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:43.962265  7702 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:43.963282  9211 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:44.013782  9540 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:44.012804  7643 tablet_service.cc:1558] Processing DeleteTablet for tablet 39c84f7f7e484afbae3c398efd5924b4 with delete_type TABLET_DATA_TOMBSTONED (TS be9c043c6d424ba1ab8def55dc9bb3e2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45904
I20260709 12:25:44.030575  9479 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:44.031426  9479 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:44.068087  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:44.069129  7659 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:44.020574  9540 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:44.085913  7734 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:44.098680  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:44.099767  9211 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:44.100208  9211 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:44.100586  9211 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:44.120151  9574 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.174s	user 0.005s	sys 0.000s
I20260709 12:25:44.122035  9479 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:44.196821  9580 tablet_replica.cc:333] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
W20260709 12:25:44.201359  9578 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:25:44.205915  9540 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:44.207261  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:44.214103  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:44.235492  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55609.1us
I20260709 12:25:44.235667  9211 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:44.247196  9211 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:44.258167  9571 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.308s	user 0.000s	sys 0.004s
I20260709 12:25:44.306716  9580 raft_consensus.cc:2243] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:44.307965  9580 raft_consensus.cc:2272] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:44.311159  9555 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:44.313854  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:44.315160  7806 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:44.314186  9580 ts_tablet_manager.cc:1916] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:44.322691  7558 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.325109  7558 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.329102  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet d997d33a05304f31b50341cc8c4bbf9b with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:46534
I20260709 12:25:44.344256  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:44.353808  9585 tablet_replica.cc:333] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:25:44.355518  9585 raft_consensus.cc:2243] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:44.364346  7626 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:44.361967  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:44.366868  7806 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:44.381215  7661 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:25:44.389523  7626 raft_consensus.cc:2686] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:25:44.427585  9585 raft_consensus.cc:2272] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:44.322679  9567 ts_tablet_manager.cc:1929] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:25:44.439512  9567 log.cc:1199] T 24790f3be4e8465581029a2e0d526117 P d72176ccaed049958ac68f706d4f052d: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/wals/24790f3be4e8465581029a2e0d526117
I20260709 12:25:44.443370  9585 ts_tablet_manager.cc:1916] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:44.452414  9567 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45822 (request call id 79) took 1129 ms. Trace:
I20260709 12:25:44.453011  9567 rpcz_store.cc:276] 0709 12:25:43.338066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.390339 (+ 52273us) service_pool.cc:224] Handling call
0709 12:25:43.631872 (+241533us) ts_tablet_manager.cc:1151] Deleting tablet 24790f3be4e8465581029a2e0d526117
0709 12:25:43.640069 (+  8197us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:44.162754 (+522685us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:44.297880 (+135126us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:44.452297 (+154417us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":220766,"thread_start_us":171164,"threads_started":1}
I20260709 12:25:44.466508  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:44.465392  9580 ts_tablet_manager.cc:1929] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:44.473290  9580 log.cc:1199] T 39c84f7f7e484afbae3c398efd5924b4 P be9c043c6d424ba1ab8def55dc9bb3e2: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/wals/39c84f7f7e484afbae3c398efd5924b4
I20260709 12:25:44.484918  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:44.485256  9472 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:44.486102  9472 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:44.487429  9472 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:44.463447  7545 catalog_manager.cc:5028] TS d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): tablet 24790f3be4e8465581029a2e0d526117 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:44.494494  7545 catalog_manager.cc:5028] TS be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): tablet 39c84f7f7e484afbae3c398efd5924b4 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:44.659675  9565 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.660287  9565 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:44.662822  9565 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:44.664386  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:44.665141  7660 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:44.667181  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:44.668198  9442 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:44.668643  9442 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.669112  9442 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.701747  9442 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:44.708385  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 142658us
I20260709 12:25:44.711616  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:44.718288  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45408.5us
I20260709 12:25:44.733673  7555 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.754019  9585 ts_tablet_manager.cc:1929] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:25:44.754688  9585 log.cc:1199] T d997d33a05304f31b50341cc8c4bbf9b P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/d997d33a05304f31b50341cc8c4bbf9b
I20260709 12:25:44.757669  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet d997d33a05304f31b50341cc8c4bbf9b (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:25:44.770753  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:44.771579  7656 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.800781  9442 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:44.804060  7656 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:44.805528  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:44.816197  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:44.817381  9441 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:44.818039  9441 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:44.819190  9441 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:44.862114  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:44.896248  7555 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:45.013505  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:45.014520  7660 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:45.019251  7808 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:45.019387  9519 raft_consensus.cc:993] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:45.020166  9519 raft_consensus.cc:1081] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:45.022127  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:45.022967  7660 raft_consensus.cc:606] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:45.027426  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:45.028131  7803 raft_consensus.cc:606] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:45.034271  7811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:45.035105  7811 raft_consensus.cc:606] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:45.040525  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:45.042986  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:45.050905  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:45.057085  7732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:45.057804  7732 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:45.058228  7732 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.086450  7660 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:45.089218  9519 raft_consensus.cc:993] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:45.089723  9519 raft_consensus.cc:1081] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:45.116298  9569 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:45.117542  9327 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:45.117985  9327 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:45.118882  9327 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.120862  9327 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:45.134971  7732 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.139520  7733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:45.140661  7733 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:45.141074  7732 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:45.141150  7733 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.147035  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:45.147891  7661 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.148249  7661 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:45.152148  7661 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.153620  7661 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.216626  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:45.217463  7811 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.223688  7733 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.233786  7661 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:45.234570  7733 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:45.245443  7733 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:45.245904  7733 pending_rounds.cc:85] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 2
I20260709 12:25:45.246134  7733 pending_rounds.cc:107] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:45.246358  7733 raft_consensus.cc:2889] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.251890  7811 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:45.253629  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:45.263806  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:45.264503  7660 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.264830  7660 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:45.264881  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:45.266171  7808 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.267207  7660 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.268733  7660 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.263995  9560 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:45.307507  9442 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:45.283437  9560 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:45.309593  9560 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.319617  7808 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:45.341305  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:45.342336  9327 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:45.343000  9327 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:45.344074  9327 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.355752  7660 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:25:45.378582  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71703.8us
W20260709 12:25:45.394306  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76507.4us
I20260709 12:25:45.401279  7660 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:45.403720  7555 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:45.427651  9596 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:25:45.448406  9594 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:25:45.456241  9441 raft_consensus.cc:993] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:45.457525  9441 raft_consensus.cc:1081] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:45.467527  9593 raft_consensus.cc:993] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:45.469228  9593 raft_consensus.cc:1081] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
W20260709 12:25:45.515336  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96546.8us
I20260709 12:25:45.518754  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:45.519640  7660 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:45.520166  7660 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.533728  7656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:45.534648  7656 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:45.535137  7656 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.556638  7660 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.560055  7660 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:45.561362  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:45.562139  7733 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:45.560245  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 146856us
I20260709 12:25:45.568823  7656 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.572439  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:45.573393  7732 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.572528  7656 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:45.590166  7733 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:45.591082  7734 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:45.591733  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:45.611202  9388 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:45.611893  9388 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:45.612958  9388 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.618029  7732 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:45.620875  9604 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:45.650146  9442 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.650751  9442 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:45.654059  9442 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.661204  9442 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:45.618193  9600 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.003s	sys 0.000s
I20260709 12:25:45.671955  7734 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:45.679261  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:45.687695  9539 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:45.688283  9539 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:45.689234  9539 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.702728  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:45.710448  9566 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:45.713083  9604 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.713501  9604 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:45.714408  9604 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:45.717856  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:45.785482  7656 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:45.825613  7660 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:45.828541  9604 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.947113  7811 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:45.951364  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 204753us
I20260709 12:25:46.140937  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:46.178714  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 179209us
I20260709 12:25:46.326242  7661 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:46.339305  9622 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:25:46.389230  7803 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:46.409103  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 208939us
W20260709 12:25:46.411435  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 156113us
W20260709 12:25:46.400475  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217809us
I20260709 12:25:46.436249  7811 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:46.540004  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70197.1us
W20260709 12:25:46.549067  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61851.1us
I20260709 12:25:46.561061  9635 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:25:46.564751  7559 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:46.634399  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.029s	sys 0.075s
W20260709 12:25:46.634928  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.029s	sys 0.075s
W20260709 12:25:46.635953  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 623636us
I20260709 12:25:46.648231  7807 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:46.661564  7804 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:46.663625  9563 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:46.664608  9563 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:46.634708  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56073.8us
W20260709 12:25:46.680702  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130807us
W20260709 12:25:46.717965  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 219188us
I20260709 12:25:46.720964  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:46.767599  9635 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:46.774078  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47262.9us
W20260709 12:25:46.779690  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55607.5us
W20260709 12:25:46.794791  7624 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59f766c3d13d4360bcdeb1d5f0f318a8. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:46.795589  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75230us
I20260709 12:25:46.819872  9649 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:46.820779  9649 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:46.862154  9623 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:46.891454  9623 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:46.893922  9623 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:46.894563  9597 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:46.895124  9597 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:46.898020  9597 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:46.898604  9597 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.720727  9563 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:46.974428  9649 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:46.851431  9595 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:46.977209  9595 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:46.902619  9327 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:46.980160  9327 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:46.899071  9597 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:46.986246  9597 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:46.982918  9327 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.030519  9638 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.140275  9638 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.143874  9638 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.177974  7811 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.195420  9646 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.196314  9646 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.196880  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.178519  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2127) took 1470 ms. Trace:
I20260709 12:25:47.182189  9595 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.203441  7803 rpcz_store.cc:276] 0709 12:25:45.707677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.717599 (+  9922us) service_pool.cc:224] Handling call
0709 12:25:47.177358 (+1459759us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:47.178406 (+  1048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435208576}
I20260709 12:25:47.217450  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.218431  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:47.229015  7735 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:47.231443  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.234601  7734 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.280494  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.281013  7734 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.281304  7660 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:47.307538  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.313935  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:47.315812  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 574626us
I20260709 12:25:47.227639  9623 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:47.253651  9595 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.254390  9647 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.260598  9569 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:47.334815  9646 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.353984  9654 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:25:47.414937  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 92598.5us
I20260709 12:25:47.338332  9623 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.426916  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 633) took 1167 ms. Trace:
I20260709 12:25:47.427423  8414 rpcz_store.cc:276] 0709 12:25:46.255188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.255361 (+   173us) service_pool.cc:224] Handling call
0709 12:25:47.421565 (+1166204us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:25:47.421867 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.422119 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.422172 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.422245 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:47.422282 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.422363 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.422528 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7931264}
I20260709 12:25:47.397032  9604 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:47.363677  9595 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.364255  9569 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.438443  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.358515  9647 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.434558  9593 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.464862  9593 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.484719  7657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:47.448668  9667 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.485483  7657 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:47.490336  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 176498us
I20260709 12:25:47.455452  9657 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:47.493921  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:47.445191  9604 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.484783  9591 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.478376  9593 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.425191  9654 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:47.502107  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82972.9us
W20260709 12:25:47.493245  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80020.7us
I20260709 12:25:47.516150  9655 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.516487  7807 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.517267  9655 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.527855  9647 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.505187  9591 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.533493  9591 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.533797  7662 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.534665  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.537022  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.540321  9595 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.541257  9569 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.509510  9663 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.542685  9663 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.508947  9664 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.545351  9663 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.544613  7811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248d73a0462c48fdad073c597b243806"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:47.548044  9675 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.548576  7811 raft_consensus.cc:606] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:47.552130  9654 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.493917  9667 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.555505  9667 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.555686  8407 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:47.556756  9623 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.500833  9657 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.548935  9593 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.560209  9657 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.560438  9677 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.560828  9657 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.560616  9593 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.561020  9677 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.561455  9657 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.563164  9593 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.550426  9604 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.579326  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:47.605934  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:47.606364  9657 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.607415  9604 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:47.607851  9604 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.608196  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:47.609011  7808 raft_consensus.cc:606] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.608242  9604 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:47.582396  9677 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.583606  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:47.584396  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48506.9us
I20260709 12:25:47.587708  9597 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.612443  9597 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:47.613392  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107501us
I20260709 12:25:47.615026  9597 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.617017  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:47.617568  9654 raft_consensus.cc:993] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:47.618171  9654 raft_consensus.cc:1081] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:47.621337  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.622867  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.624354  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:47.595600  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 157286us
I20260709 12:25:47.548776  9675 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.629628  9675 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.572005  9655 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:47.574250  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.631748  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:47.632084  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 140313us
I20260709 12:25:47.563781  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.571587  9518 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:47.633994  9518 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.637089  9518 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.549798  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:47.638623  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.639256  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:47.639962  7803 raft_consensus.cc:606] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:47.601392  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.601989  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.641249  9657 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:47.602571  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.642733  7807 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:25:47.629513  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.643206  9654 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:47.570173  9641 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.644476  9641 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.644865  9635 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:47.644886  9669 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:47.646929  9641 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.647042  9617 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:47.649927  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.652627  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.659633  7557 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.662168  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:47.664163  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 262) took 1689 ms. Trace:
I20260709 12:25:47.665463  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:47.666473  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:47.665057  7559 rpcz_store.cc:276] 0709 12:25:45.974859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.975011 (+   152us) service_pool.cc:224] Handling call
0709 12:25:47.664058 (+1689047us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:46.582438 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:46.583018 (+   580us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:46.583066 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:25:46.583083 (+    17us) tablet.cc:662] Decoding operations
0709 12:25:46.800946 (+217863us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:46.801077 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:46.801128 (+    51us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:46.801746 (+   618us) tablet.cc:701] Row locks acquired
0709 12:25:46.801777 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:25:46.801977 (+   200us) write_op.cc:270] Start()
0709 12:25:46.802125 (+   148us) write_op.cc:276] Timestamp: P: 1783599946801911 usec, L: 0
0709 12:25:46.802150 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:25:46.802960 (+   810us) log.cc:844] Serialized 538 byte log entry
0709 12:25:47.477933 (+674973us) op_driver.cc:464] REPLICATION: finished
0709 12:25:47.635875 (+157942us) write_op.cc:301] APPLY: starting
0709 12:25:47.635971 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:47.636097 (+   126us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:47.636251 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:47.636509 (+   258us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:47.636708 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:47.636760 (+    52us) write_op.cc:312] APPLY: finished
0709 12:25:47.636976 (+   216us) log.cc:844] Serialized 23 byte log entry
0709 12:25:47.637206 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.637306 (+   100us) write_op.cc:454] Released schema lock
0709 12:25:47.637351 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:47.637844 (+   493us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.637876 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20879,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":157555,"apply.run_cpu_time_us":1965,"apply.run_wall_time_us":1971,"num_ops":1,"prepare.queue_time_us":7851,"prepare.run_cpu_time_us":10065,"prepare.run_wall_time_us":630187,"raft.queue_time_us":243393,"raft.run_cpu_time_us":8013,"raft.run_wall_time_us":192253,"replication_time_us":645077,"thread_start_us":720384,"threads_started":4,"wal-append.queue_time_us":376511}]]}
I20260709 12:25:47.673890  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.674671  7806 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:47.673794  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:47.683080  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49990.6us
I20260709 12:25:47.684826  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.695652  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.699805  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.700450  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.700802  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.701694  7730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46562
I20260709 12:25:47.702006  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.702437  7730 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.702884  7730 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.704754  9518 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.705334  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.705464  9518 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:47.706282  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:47.707585  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.711506  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 281) took 1310 ms. Trace:
I20260709 12:25:47.716745  9518 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:47.722414  7730 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.711992  7557 rpcz_store.cc:276] 0709 12:25:46.401176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.401372 (+   196us) service_pool.cc:224] Handling call
0709 12:25:47.711433 (+1310061us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:47.668840 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:47.669432 (+   592us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:47.669469 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:25:47.669486 (+    17us) tablet.cc:662] Decoding operations
0709 12:25:47.669862 (+   376us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:47.669924 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:47.669955 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:47.670188 (+   233us) tablet.cc:701] Row locks acquired
0709 12:25:47.670209 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:25:47.670393 (+   184us) write_op.cc:270] Start()
0709 12:25:47.670520 (+   127us) write_op.cc:276] Timestamp: P: 1783599947670334 usec, L: 0
0709 12:25:47.670548 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:25:47.671369 (+   821us) log.cc:844] Serialized 538 byte log entry
0709 12:25:47.674275 (+  2906us) op_driver.cc:464] REPLICATION: finished
0709 12:25:47.690741 (+ 16466us) write_op.cc:301] APPLY: starting
0709 12:25:47.690849 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:47.690985 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:47.691105 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:47.691395 (+   290us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:47.691593 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:47.691655 (+    62us) write_op.cc:312] APPLY: finished
0709 12:25:47.691890 (+   235us) log.cc:844] Serialized 23 byte log entry
0709 12:25:47.703630 (+ 11740us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.703728 (+    98us) write_op.cc:454] Released schema lock
0709 12:25:47.703769 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:47.704136 (+   367us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.704172 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5245,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":16164,"apply.run_cpu_time_us":1973,"apply.run_wall_time_us":13366,"num_ops":1,"prepare.queue_time_us":142,"prepare.run_cpu_time_us":2822,"prepare.run_wall_time_us":2822,"replication_time_us":3595}]]}
I20260709 12:25:47.725327  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.727056  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.732824  7730 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:47.733634  9669 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:47.740741  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:47.743343  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:47.746682  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.747584  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.749607  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:47.749738  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.725911  9646 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:25:47.749073  9327 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:47.752362  9327 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:47.752833  9327 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.753964  9593 raft_consensus.cc:993] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:47.754590  9593 raft_consensus.cc:1081] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:47.743718  9569 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:47.759804  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.761294  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.762794  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:47.763154  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.763626  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.763626  7656 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:47.764678  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.765272  7656 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:47.762053  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.768530  7806 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:47.770684  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.776196  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.778532  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.781083  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.785264  9669 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.785463  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:47.786600  9635 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.786957  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:47.786842  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.788496  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:47.788854  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:47.788966  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:47.790438  9662 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:47.790773  9668 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.791998  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:47.797544  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.798220  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.800290  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:47.803508  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.805081  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.796437  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:47.809574  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:47.810343  8409 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.813325  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "248d73a0462c48fdad073c597b243806"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:47.815264  7660 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.815793  7660 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:47.733055  9657 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:25:47.816989  9658 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:47.820506  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:47.823654  9661 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.823879  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:47.820873  9658 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.845391  8409 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:47.848189  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.842787  7656 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:47.808900  9623 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:47.902484  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54392.6us
I20260709 12:25:47.890072  9638 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.903389  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.904752  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.852584  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:47.816990  9593 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:47.839393  9627 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:47.848852  9658 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.825896  9675 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:47.914572  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62086us
I20260709 12:25:47.890514  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:47.887991  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67933.4us
I20260709 12:25:47.913355  9657 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.913232  9627 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:47.919713  9627 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.913039  9678 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:47.920145  9659 raft_consensus.cc:993] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:47.920722  9419 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.944999  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.946677  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.923599  9621 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:47.943819  8409 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:47.959950  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.961831  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.976675  9658 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:47.979893  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49113.3us
I20260709 12:25:47.958475  9675 raft_consensus.cc:1081] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:47.980839  9594 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:47.998656  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.982492  9649 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.004804  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:47.983081  9620 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.002741  9518 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:48.011734  9566 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:48.012352  9603 raft_consensus.cc:993] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:47.996104  9604 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.010372  7811 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:48.022694  9569 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:47.986966  9627 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.062916  9662 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.084095  9662 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:48.063786  9592 raft_consensus.cc:1081] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:25:48.086439  7804 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:48.093389  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:48.088253  9569 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.095091  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:48.107987  7660 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.109577  9627 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:48.136891  7660 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.137308  9441 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.145277  9327 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.177276  9627 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.176357  9661 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:48.187474  9629 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:48.188428  9604 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:48.181476  9658 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:48.194285  9650 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.201442  7732 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:48.190080  9597 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.186151  9592 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.195379  9673 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:48.199292  9661 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:25:48.299204  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.206125  9686 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:48.304154  9665 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:48.195622  9627 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.307374  9657 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:48.301671  9673 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.222208  9597 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:48.352329  9692 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 12:25:48.353752  9640 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:48.212456  9650 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.336359  9665 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.367511  9642 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.372051  9327 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.299055  9629 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.354534  9640 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.402339  9688 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.339987  9657 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.296944  9592 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:48.437682  9690 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:25:48.355829  9661 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.453678  9654 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:25:48.454695  9687 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 12:25:48.455447  9689 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:25:48.434180  9688 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.457288  9650 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.458864  9597 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:48.456074  9627 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:48.543903  9693 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:25:48.544615  9518 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.545157  9518 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:48.548264  9665 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.453997  9673 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.408818  9642 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.525902  9654 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.555486  9657 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:48.569830  9654 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.576160  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:25:48.576891  7660 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:48.577310  7660 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:48.584792  9688 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.558581  9629 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:48.585038  9694 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:25:48.591418  7656 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:48.542156  9592 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:48.603667  7555 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:25:48.618819  7660 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:48.629776  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.022s	sys 0.047s
W20260709 12:25:48.630474  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.023s	sys 0.047s
W20260709 12:25:48.594440  9647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.016s	sys 0.043s
W20260709 12:25:48.634608  7808 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.021s	sys 0.060s
W20260709 12:25:48.637127  7808 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.060s
I20260709 12:25:48.572033  9662 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.643471  7807 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:48.631844  9621 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.009s	sys 0.052s
W20260709 12:25:48.652853  8377 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.016s	sys 0.031s
W20260709 12:25:48.653548  8377 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.016s	sys 0.031s
W20260709 12:25:48.643779  9647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.016s	sys 0.043s
I20260709 12:25:48.664973  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:48.665730  7811 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:48.642076  9640 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.681176  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:48.676807  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:48.646103  9655 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:48.651505  9621 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.010s	sys 0.052s
W20260709 12:25:48.655263  9677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.043s	sys 0.030s
W20260709 12:25:48.681802  7700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.009s	sys 0.010s
I20260709 12:25:48.685791  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:48.687804  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:48.683113  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:48.664287  9569 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:48.698729  7700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.010s
W20260709 12:25:48.699963  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 604924us
I20260709 12:25:48.701371  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:48.695602  7812 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:25:48.710090  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:48.710090  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:48.718713  7732 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:48.722121  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:48.722970  7734 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:48.724568  7811 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
W20260709 12:25:48.692250  9677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.044s	sys 0.030s
I20260709 12:25:48.656811  9635 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:48.734035  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:48.679893  9673 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:48.743036  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:48.752490  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.041s	sys 0.017s
W20260709 12:25:48.753044  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.042s	sys 0.017s
I20260709 12:25:48.759954  7808 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:48.725749  9597 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.764569  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:48.867986  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:25:48.820189  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66761.2us
W20260709 12:25:48.893107  7701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.029s	sys 0.034s
W20260709 12:25:48.895725  7701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.030s	sys 0.034s
I20260709 12:25:48.868414  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:48.898303  7734 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:48.892885  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.008s	sys 0.021s
W20260709 12:25:48.899505  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.009s	sys 0.021s
I20260709 12:25:48.901484  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:48.902712  7656 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:48.898473  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:48.909722  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:48.911532  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:48.734325  9617 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:48.828572  7736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
W20260709 12:25:48.965854  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60937us
I20260709 12:25:48.977620  7736 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:48.742174  9699 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:49.001897  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:48.832476  9696 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:49.043356  9705 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.051084  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:48.817683  9662 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:48.818444  9703 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:49.131735  9703 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.053215  9688 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:48.978142  7736 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.134483  9688 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.137488  9688 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:48.856365  9642 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.119253  7812 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:49.138376  7812 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:49.139317  7812 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:48.909580  9665 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:49.140529  7812 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.118381  9699 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.120224  9646 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:49.143630  9646 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.143918  9705 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.143818  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:49.144869  9699 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.146056  9646 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.146299  9705 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.123852  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:49.148078  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:49.148684  7806 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:49.149019  7806 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:49.124941  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:49.125707  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:49.126961  9594 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.000344  9617 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.758169  9673 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:48.818360  9518 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:49.146140  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:49.180729  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.162490  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:49.185209  7812 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:49.166703  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.009s	sys 0.036s
W20260709 12:25:49.201154  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.010s	sys 0.036s
W20260709 12:25:49.201632  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 539978us
W20260709 12:25:49.205211  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48849.4us
W20260709 12:25:49.205294  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47323.5us
W20260709 12:25:49.168887  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 189109us
I20260709 12:25:49.161626  7806 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.174633  7736 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.210861  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:49.554739  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:49.556259  7656 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:49.210750  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.552726  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:49.499619  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.005s
W20260709 12:25:49.568235  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.006s
W20260709 12:25:49.550213  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 387780us
W20260709 12:25:49.568686  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 601405us
W20260709 12:25:49.331684  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 120077us
I20260709 12:25:49.270118  9327 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:49.610996  9617 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.161585  9594 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.611943  9617 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.612528  9617 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.613621  9594 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:49.610716  9327 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.621598  9327 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.614758  9617 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.622959  9617 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.450443  9703 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.423044  9597 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:49.623572  9617 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.628568  9597 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.630179  9617 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.401098  9632 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.501308  9673 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.444591  9518 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.225203  9699 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:49.618213  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:49.614584  9702 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:49.619347  9677 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:49.619522  7736 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.650240  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:49.210525  9663 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:49.669750  9720 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.674492  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:49.676215  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:49.680423  8415 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.677747  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:49.640554  9702 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.637897  9699 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.225530  9646 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:49.400648  9642 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.605774  9705 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:49.607035  9644 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:49.681711  8411 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:49.730587  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:49.731424  7734 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:49.731598  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:25:49.687372  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 117016us
I20260709 12:25:49.706451  9720 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:49.715914  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76106.2us
I20260709 12:25:49.631162  9597 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:49.720678  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 1928) took 1037 ms. Trace:
I20260709 12:25:49.666611  9621 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:49.738111  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.791872  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.797310  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.798312  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:49.813714  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:49.817687  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:49.805531  7662 rpcz_store.cc:276] 0709 12:25:48.682815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.686730 (+  3915us) service_pool.cc:224] Handling call
0709 12:25:48.687125 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.687437 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:49.183132 (+495695us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.720112 (+536980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.720202 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:49.720287 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.720393 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.720577 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54138,"prepare.queue_time_us":112462,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":9002880,"thread_start_us":577295,"threads_started":3,"wal-append.queue_time_us":81130}
W20260709 12:25:49.820021  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 145689us
I20260709 12:25:49.733072  9627 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.679999  9677 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:49.826424  9518 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:49.825511  9665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40396 (request call id 12) took 4119 ms. Trace:
I20260709 12:25:49.836303  9597 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:49.840788  9720 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:49.844890  9604 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:49.732360  9642 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.703650  9663 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:49.904445  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.781860  9705 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:49.723922  9646 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:49.971271  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.974637  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58265.2us
I20260709 12:25:49.968056  9700 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.805768  9621 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:49.964066  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 142730us
I20260709 12:25:50.000252  9719 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:49.949568  9665 rpcz_store.cc:276] 0709 12:25:45.706305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.731418 (+ 25113us) service_pool.cc:224] Handling call
0709 12:25:45.833890 (+102472us) log.cc:844] Serialized 499 byte log entry
0709 12:25:47.572343 (+1738453us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b180035f620. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001e7bb0b 00007fc001f00b63 00007fc001f28752
0709 12:25:49.825341 (+2252998us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25054,"raft.run_cpu_time_us":6852,"raft.run_wall_time_us":137777,"spinlock_wait_cycles":13797248,"thread_start_us":449153,"threads_started":2,"wal-append.queue_time_us":345783,"wal-append.run_cpu_time_us":3451,"wal-append.run_wall_time_us":1166622}
I20260709 12:25:49.951715  9627 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.002879  9717 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:50.051776  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:50.102377  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50687.3us
W20260709 12:25:50.091820  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69404us
W20260709 12:25:50.106429  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 464822us
I20260709 12:25:50.107702  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:25:50.115418  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 290960us
I20260709 12:25:50.120862  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:50.119728  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:50.126579  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:25:50.130430  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 127860us
I20260709 12:25:50.140882  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:49.782291  9644 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.121678  9623 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:50.134792  9704 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:50.294025  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152390us
I20260709 12:25:50.077035  9700 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.318068  9700 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:50.135347  9656 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.158s
I20260709 12:25:50.204360  9641 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:50.208242  9623 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:50.342531  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 222896us
W20260709 12:25:50.324885  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 204458us
I20260709 12:25:50.201952  9732 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:50.293017  9704 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:50.122510  9719 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.315093  9675 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:50.314414  9729 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:50.323750  9518 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.372092  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.372860  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.374035  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.363297  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:50.356153  9623 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:50.376655  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.393030  7808 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:50.377738  7811 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:50.335094  9742 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.381086  7735 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:50.395747  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:50.538074  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 149687us
W20260709 12:25:50.550840  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 255304us
I20260709 12:25:50.436916  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:50.553144  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157491us
I20260709 12:25:50.378116  7804 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:50.364672  9704 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:50.550169  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.360529  9699 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:50.540200  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 408221us
I20260709 12:25:50.560683  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:50.553066  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1680) took 1713 ms (client timeout 2009 ms). Trace:
I20260709 12:25:50.564419  9747 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:50.564311  7735 rpcz_store.cc:269] 0709 12:25:48.839154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.857607 (+ 18453us) service_pool.cc:224] Handling call
0709 12:25:50.363013 (+1505406us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:50.552969 (+189956us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54742784}
I20260709 12:25:50.565593  9747 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.567550  9741 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:50.568506  9741 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.572278  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:50.575997  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:50.576562  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.577081  7805 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:50.578414  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:50.395309  9700 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.578754  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.390537  9663 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:50.389767  9646 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.437548  9677 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.579329  7803 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:50.580859  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:50.590588  7811 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:50.583397  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:50.584964  7659 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:50.592940  7808 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:50.593365  7808 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:50.542734  9739 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.375769  9738 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.549307  9705 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:50.559768  9642 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.455281  9702 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.504456  9621 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:50.525198  9644 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.368428  9732 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.460479  9742 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.403370  9518 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:50.598752  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:50.706305  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.712101  7812 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:50.681768  7660 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:50.713868  7811 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:50.715402  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:50.717450  7803 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:25:50.603819  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50515.7us
I20260709 12:25:50.719633  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45918 (request call id 1487) took 2139 ms. Trace:
W20260709 12:25:50.738628  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.748s (SENT) [suppressed 5 similar messages]
W20260709 12:25:50.760109  9712 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.040s	user 0.001s	sys 0.017s
W20260709 12:25:50.705072  9632 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.073s	user 0.011s	sys 0.002s
I20260709 12:25:50.710178  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 468) took 2018 ms. Trace:
I20260709 12:25:50.702807  9736 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:50.768682  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48799.6us
I20260709 12:25:50.722980  7660 rpcz_store.cc:276] 0709 12:25:48.575849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.575985 (+   136us) service_pool.cc:224] Handling call
0709 12:25:50.663928 (+2087943us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4c00005940. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc0001eea20 00007fc0001ffa29 00007fc00020013f 00007fc0001c42ab 00007fc0001f1f3c 00007fc0001f23fe 00007fc00047f88b 00007fc001e7e495 00007fc001ecf30f 00007fc001f19a55 00007fc006ee9cd0
0709 12:25:50.714487 (+ 50559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39811840}
I20260709 12:25:50.690418  9746 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:50.603461  9754 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.818106  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1673) took 1992 ms. Trace:
I20260709 12:25:50.587641  9699 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:50.718868  9675 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.664867  9627 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:50.837608  7731 rpcz_store.cc:276] 0709 12:25:48.826002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.828005 (+  2003us) service_pool.cc:224] Handling call
0709 12:25:49.623727 (+795722us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:25:49.623981 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.804912 (+180931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.804968 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.805042 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:49.805096 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.817700 (+1012604us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54000721d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:50.817773 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.817998 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421244032}
I20260709 12:25:50.604357  9731 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:50.664388  9663 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.781065  9719 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.604972  9744 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:50.718990  9642 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:25:50.866415  9641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.026s	sys 0.034s
I20260709 12:25:50.836812  9632 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:50.594262  9646 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:50.725571  9733 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.578264  9604 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.720615  9591 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.588486  9700 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.796788  9743 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:25:50.797508  9735 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
W20260709 12:25:50.881603  9641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.026s	sys 0.035s
I20260709 12:25:50.845861  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:50.852648  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.072s	sys 0.154s
W20260709 12:25:50.930025  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.072s	sys 0.154s
W20260709 12:25:50.930378  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 799913us
I20260709 12:25:50.855494  9741 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:50.856518  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.001s	sys 0.001s
W20260709 12:25:50.931340  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.002s	sys 0.001s
W20260709 12:25:50.857739  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.003s
I20260709 12:25:50.661839  7808 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:50.932152  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.000s	sys 0.003s
I20260709 12:25:50.932590  7808 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:50.933991  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88234.5us
I20260709 12:25:50.853621  9688 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.913506  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:50.777241  9738 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.919457  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:50.866804  9705 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:25:50.936443  8415 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
W20260709 12:25:50.937703  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 199162us
I20260709 12:25:50.938019  8409 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:50.935917  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:50.878904  9740 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.876824  9601 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.871765  9699 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.915182  9744 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.943334  8411 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:50.953102  7808 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:50.957219  9749 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:50.957718  9737 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.940968  9621 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:50.962360  9732 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:50.871796  9675 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.925554  9702 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.612108  9739 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.933912  9641 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:50.960829  9700 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:50.978302  7626 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:50.879369  9663 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.883854  9642 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.877004  9746 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.925762  9604 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.954574  9741 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:50.877105  9736 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.928486  9591 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.049943  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:50.815280  8408 rpcz_store.cc:276] 0709 12:25:48.691514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.695520 (+  4006us) service_pool.cc:224] Handling call
0709 12:25:50.701535 (+2006015us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfde9c 00007fbffedfda5b 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:25:50.709475 (+  7940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.709737 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.709771 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.709827 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:50.709874 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.709952 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.710100 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3970432}
I20260709 12:25:51.040521  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:51.059965  8411 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:50.941648  9688 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.061717  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:51.041931  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63747.3us
I20260709 12:25:51.051788  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:51.051844  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.052826  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:51.053951  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.927282  9646 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.073182  7660 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 2: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:51.046784  9745 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:51.067502  7803 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:51.079059  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.008222  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:51.050756  7662 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:51.061949  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:50.927541  9757 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:25:51.047806  9608 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:51.082119  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:50.965339  9744 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.082135  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.089876  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:51.035948  9518 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.087622  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:51.078675  9623 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.001s
I20260709 12:25:51.048193  9621 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:51.072551  9656 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:50.983035  9768 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:50.964082  9740 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:50.879652  9754 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.944485  9743 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:51.084064  9729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.042s	sys 0.060s
I20260709 12:25:51.006306  9702 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.059369  9741 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.112442  9678 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:51.086269  9700 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:50.933182  9733 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.194381  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:51.152979  9729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.043s	sys 0.060s
I20260709 12:25:51.383074  8408 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:51.383420  9764 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:51.011937  9737 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.151847  9742 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.011507  9749 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:50.965415  9601 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.143394  9623 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.151096  9608 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:51.445250  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.023s	sys 0.020s
W20260709 12:25:51.454468  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.449903  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 360145us
I20260709 12:25:51.462500  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:51.454584  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.023s	sys 0.020s
I20260709 12:25:51.472299  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.474499  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.481731  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:51.482739  9678 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:51.483309  9678 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:51.483436  9749 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.483800  9678 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:51.486589  9740 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.488916  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:51.492197  7731 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:51.503350  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.398154  9729 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:51.514349  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2148) took 3777 ms. Trace:
I20260709 12:25:51.517550  7807 rpcz_store.cc:276] 0709 12:25:47.736674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.739684 (+  3010us) service_pool.cc:224] Handling call
0709 12:25:48.643156 (+903472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.645266 (+  2110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.514040 (+2868774us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400132ed0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:51.514249 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076934016}
I20260709 12:25:51.528195  9601 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.528385  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.220093  9768 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.530294  7736 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:51.532575  9593 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.533200  9593 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.496943  9738 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.536005  9736 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:51.515290  9765 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.529s	user 0.000s	sys 0.001s
I20260709 12:25:51.531469  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.541402  7728 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:51.493671  9647 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:51.491343  9518 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:51.551103  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:51.554225  8413 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.395651  9700 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.504536  9749 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:51.409925  9746 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.569653  9593 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.572166  9623 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.574038  9663 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.508553  9756 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:51.539887  9739 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.584074  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.408767  9641 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.550030  9647 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:51.557386  9737 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.582660  9749 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.441706  9621 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.604292  9754 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.605130  9699 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.450197  9764 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:51.606350  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.024s	sys 0.048s
W20260709 12:25:51.611737  8409 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.027s	sys 0.057s
W20260709 12:25:51.614738  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.025s	sys 0.048s
I20260709 12:25:51.595250  9642 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:51.619093  8409 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.027s	sys 0.057s
I20260709 12:25:51.560312  9518 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.614286  9604 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.614248  9591 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.632732  9678 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.633085  9764 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.634234  9758 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:51.602730  9756 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.623911  9601 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.626143  9702 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.638994  9758 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.623849  9646 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.646467  9778 ts_tablet_manager.cc:933] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:51.636227  9741 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.642190  9726 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:51.654220  9743 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.658938  9749 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.644286  9762 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.708642  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 409121us
I20260709 12:25:51.723709  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:51.726709  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:51.591024  7728 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:25:51.732067  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.732304  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.738992  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:51.740250  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:51.741775  8414 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:51.744560  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:51.745105  9718 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 2.107s	user 0.037s	sys 0.044s
I20260709 12:25:51.670540  9762 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.630859  9647 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.756883  9647 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:51.668570  9726 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:51.749764  8409 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:51.762463  9745 raft_consensus.cc:2764] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:25:51.764021  7730 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:51.754768  9518 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.764511  9737 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:51.765383  9518 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:51.765724  9737 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:51.769814  9714 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:51.776099  9678 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.631424  9729 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.784464  9700 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.783746  9668 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:51.787129  9779 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:51.787773  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 381) took 1978 ms. Trace:
I20260709 12:25:51.788326  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 380) took 1977 ms. Trace:
W20260709 12:25:51.784482  9560 consensus_queue.cc:1175] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 3 index: 3 } }
I20260709 12:25:51.766443  9518 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.788717  9758 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.789614  9749 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:25:51.792493  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.766593  9737 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.791756  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.797405  9778 tablet_copy_client.cc:323] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:25:51.794306  7656 rpcz_store.cc:276] 0709 12:25:49.795031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.795309 (+   278us) service_pool.cc:224] Handling call
0709 12:25:49.795597 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.021497 (+225900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.021548 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.021614 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:50.021696 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.771848 (+1750152us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540028df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:51.771966 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.772330 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518835456}
I20260709 12:25:51.803857  9601 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:51.808804  9715 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.165s	user 0.014s	sys 0.020s
I20260709 12:25:51.811703  9762 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.794375  7655 rpcz_store.cc:276] 0709 12:25:49.793376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.793581 (+   205us) service_pool.cc:224] Handling call
0709 12:25:49.794388 (+   807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.122059 (+327671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.122090 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.122135 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:50.122167 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.770387 (+1648220us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54004353d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:51.770532 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.770941 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752936192}
I20260709 12:25:51.811758  7729 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:51.815595  8408 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:51.816222  9601 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:51.816908  9713 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.088s	user 0.100s	sys 0.153s
I20260709 12:25:51.817979  9675 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.803985  9749 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:51.803993  9710 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 2.175s	user 0.024s	sys 0.016s
I20260709 12:25:51.821768  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.762063  9726 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.830382  9756 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.833267  9749 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.827994  9737 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.771364  9714 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.824143  9779 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.841636  9673 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.844449  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 1945) took 1270 ms. Trace:
I20260709 12:25:51.853145  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.854588  7731 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:51.845234  7657 rpcz_store.cc:276] 0709 12:25:50.574280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.574521 (+   241us) service_pool.cc:224] Handling call
0709 12:25:50.574967 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.113651 (+538684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.113696 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.113743 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:51.113784 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.843683 (+729899us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:51.843779 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.844199 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238182784}
I20260709 12:25:51.842980  9673 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:51.867498  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54476.4us
I20260709 12:25:51.860062  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:51.878731  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:51.882798  9768 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.892393  9708 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:25:51.865342  9704 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.287s	user 0.013s	sys 0.009s
I20260709 12:25:51.856966  9518 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.894371  9714 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.857471  9608 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.893898  9708 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.906119  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 422) took 2300 ms. Trace:
W20260709 12:25:51.901537  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 175176us
W20260709 12:25:51.909888  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72928.6us
I20260709 12:25:51.910475  9748 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:51.869647  9688 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.879745  9749 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.911844  8407 rpcz_store.cc:276] 0709 12:25:49.605281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.607377 (+  2096us) service_pool.cc:224] Handling call
0709 12:25:49.607725 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.009923 (+402198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.009979 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.010045 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:50.010105 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.905360 (+1895255us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:51.905474 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.905789 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924204288}
I20260709 12:25:51.925870  9749 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.928241  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:51.928911  9726 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.930774  9767 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:51.913796  9723 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:51.934875  9723 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.919090  9673 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:51.946687  9734 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:51.955729  9734 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:51.957047  9675 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:51.960961  9675 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.964411  7737 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:51.949271  9714 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:51.966192  9725 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.968082  9675 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:51.968415  9779 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.961494  7806 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.921594  9748 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:51.977583  9594 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:51.979439  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:51.978797  9594 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:51.989141  9733 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.913371  9704 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.000891  7555 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 264) took 3860 ms. Trace:
I20260709 12:25:51.968689  9725 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:52.007790  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79762.9us
I20260709 12:25:51.969516  9714 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.018105  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.019675  7809 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:51.992435  9749 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:51.990404  9594 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.023746  9714 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.032081  9733 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:52.035272  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.044410  7806 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:52.022401  9734 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.047191  9419 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.044142  8413 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:52.035656  9733 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.059839  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.059571  9419 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.060087  9747 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.037837  9730 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:25:52.063648  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:52.065022  8413 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:52.062904  9733 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:52.069758  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2171) took 2924 ms (client timeout 2179 ms). Trace:
W20260709 12:25:52.077349  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83627.1us
I20260709 12:25:52.001505  7555 rpcz_store.cc:276] 0709 12:25:48.140084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.140287 (+   203us) service_pool.cc:224] Handling call
0709 12:25:52.000542 (+3860255us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:49.620695 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:49.621338 (+   643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:49.621375 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:25:49.621396 (+    21us) tablet.cc:662] Decoding operations
0709 12:25:49.621757 (+   361us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:49.621823 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:49.621854 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:49.622054 (+   200us) tablet.cc:701] Row locks acquired
0709 12:25:49.622069 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:25:49.622250 (+   181us) write_op.cc:270] Start()
0709 12:25:49.622361 (+   111us) write_op.cc:276] Timestamp: P: 1783599949622192 usec, L: 0
0709 12:25:49.622384 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:25:49.623127 (+   743us) log.cc:844] Serialized 538 byte log entry
0709 12:25:51.647101 (+2023974us) op_driver.cc:464] REPLICATION: finished
0709 12:25:51.670979 (+ 23878us) write_op.cc:301] APPLY: starting
0709 12:25:51.671101 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:51.671250 (+   149us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:51.671932 (+   682us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:51.672301 (+   369us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:51.672539 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:51.672635 (+    96us) write_op.cc:312] APPLY: finished
0709 12:25:51.672893 (+   258us) log.cc:844] Serialized 23 byte log entry
0709 12:25:51.814390 (+141497us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:51.859243 (+ 44853us) write_op.cc:454] Released schema lock
0709 12:25:51.859355 (+   112us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:51.860420 (+  1065us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:51.860458 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":23537,"apply.run_cpu_time_us":3972,"apply.run_wall_time_us":189257,"num_ops":1,"prepare.queue_time_us":1000094,"prepare.run_cpu_time_us":12879,"prepare.run_wall_time_us":484774,"raft.queue_time_us":656832,"raft.run_cpu_time_us":10349,"raft.run_wall_time_us":22717,"replication_time_us":2024409,"thread_start_us":823625,"threads_started":4,"wal-append.queue_time_us":739890}]]}
W20260709 12:25:52.074293  7806 rpcz_store.cc:269] 0709 12:25:49.139738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.147824 (+  8086us) service_pool.cc:224] Handling call
0709 12:25:51.958801 (+2810977us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54002cc520. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e97a1d 00007fc001e7e1a4 00007fc001f00447 00007fc001f18fe7 00007fc001f1a7de 00007fc001f2627c 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591
0709 12:25:52.064261 (+105460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36085632}
I20260709 12:25:52.086308  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 1940) took 1589 ms. Trace:
I20260709 12:25:52.088753  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.086802  7658 rpcz_store.cc:276] 0709 12:25:50.495206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.496650 (+  1444us) service_pool.cc:224] Handling call
0709 12:25:50.505892 (+  9242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.975645 (+469753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.975703 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.975765 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:50.975827 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.076935 (+1101108us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400116054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:52.082177 (+  5242us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.084837 (+  2660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057445248}
I20260709 12:25:52.084813  9725 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:52.095778  9706 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:52.100896  9706 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:52.101547  9706 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.099972  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:52.089650  9748 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.106012  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:52.107760  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:52.114246  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.119876  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1691) took 1056 ms. Trace:
I20260709 12:25:52.120657  7737 rpcz_store.cc:276] 0709 12:25:51.063151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.063304 (+   153us) service_pool.cc:224] Handling call
0709 12:25:51.063729 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.967918 (+904189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.119261 (+151343us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:52.119726 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065474176}
I20260709 12:25:52.124002  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.131531  9419 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:52.132122  9784 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 12:25:52.110949  9708 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:52.135713  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46108.6us
I20260709 12:25:52.144114  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.143883  9730 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:52.148083  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.147862  9730 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.150303  9730 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.150189  9673 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:52.155143  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 301) took 3246 ms. Trace:
I20260709 12:25:52.156301  9608 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:25:52.159904  9757 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:52.182452  9708 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:52.158737  9704 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.163887  9673 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:52.184183  9673 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.145519  9734 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.179803  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:52.146867  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:52.174053  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.174391  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.189278  7807 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:52.182940  9757 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:52.173955  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.194085  7655 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.194595  9608 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:52.194101  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.195325  9608 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:52.165206  9723 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.195858  9608 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.180685  7557 rpcz_store.cc:276] 0709 12:25:48.908475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.908656 (+   181us) service_pool.cc:224] Handling call
0709 12:25:52.154344 (+3245688us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b24003769e8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fbffefcd970 00007fbffefdaa8f 00007fc007ddfae4 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc
0709 12:25:52.154876 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48963,"reported_tablets":2,"spinlock_wait_cycles":12946816}
W20260709 12:25:52.194725  9744 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.050s	user 0.007s	sys 0.016s
I20260709 12:25:52.199735  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:52.192883  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47932.6us
W20260709 12:25:52.202035  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61070.9us
I20260709 12:25:52.203351  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:52.203771  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.204581  9748 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:52.205080  9748 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.194551  9757 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.205519  9748 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.183421  9708 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:52.208175  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:52.210295  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.212548  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.214481  9621 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:52.216282  9621 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:52.217724  9621 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.219390  9745 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:52.219954  9745 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.220409  9745 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.221165  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.226449  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.228042  9747 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:25:52.227950  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.233386  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.234280  7807 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:52.232255  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:52.237129  9640 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:52.242551  9708 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:52.245072  9708 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.253094  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:52.231755  8379 connection.cc:256] RPC call timeout handler was delayed by -1.11266087399963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:25:52.266502  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86981.7us
I20260709 12:25:52.208848  9744 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.272125  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.272285  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.273083  9640 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.275134  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.279771  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.280529  7804 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
I20260709 12:25:52.284431  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.285501  7657 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.285981  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.284914  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.278983  9640 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.296674  7804 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:52.301851  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:52.303612  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.290035  9757 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.305945  8412 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:52.306137  9617 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.295739  9754 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.306222  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.309456  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.312959  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.314939  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.316656  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.318796  7655 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.319535  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.318881  7658 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.320986  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:52.318625  9767 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:52.331691  9767 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.332126  9767 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.332466  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.318817  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.335695  7737 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:52.345156  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.345988  7658 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:52.348024  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.294118  7557 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 4 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:52.349795  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.310105  9714 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.350590  8409 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.351799  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.353886  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.356537  7737 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 2: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:52.356547  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:52.356921  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55191.9us
I20260709 12:25:52.355017  7732 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:52.353080  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.362310  7557 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:52.364126  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53341.8us
I20260709 12:25:52.360334  7807 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.358414  9617 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.365377  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.371089  7809 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:25:52.371847  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.374661  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.377216  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.340485  9754 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.376428  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.379693  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:52.366531  9726 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:52.376469  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.385036  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.381034  7557 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:52.389463  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:25:52.393316  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:52.394744  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.395331  9617 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.395565  7810 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:52.397626  9747 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:52.398605  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:52.399739  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.400302  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.401224  7807 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:52.402550  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.366003  9725 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.397796  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 668) took 1339 ms. Trace:
I20260709 12:25:52.401894  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:52.381732  9640 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.403702  7662 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.403580  7806 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:52.383183  9726 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.412999  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.415393  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.403704  9725 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.419814  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.415634  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:52.418866  9747 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:52.419051  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.377763  9608 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.422622  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:52.406350  9754 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.411991  9744 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.430047  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.432533  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.416775  9726 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.436214  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.440438  9632 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:52.441017  9632 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.441459  9632 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.443861  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.407581  8412 rpcz_store.cc:276] 0709 12:25:51.058256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.141932 (+ 83676us) service_pool.cc:224] Handling call
0709 12:25:51.438117 (+296185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.326907 (+888790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.397365 (+ 70458us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54005e0e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:52.397695 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952893824}
I20260709 12:25:52.461321  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.461258  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.422616  9640 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:25:52.495741  9640 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:52.496300  9640 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:52.436908  9617 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:52.499416  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.532516  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.544873  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.546079  7662 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.562692  9714 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.563701  9704 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:25:52.580104  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57133.8us
I20260709 12:25:52.590803  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.611276  9704 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:52.620242  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 208032us
I20260709 12:25:52.587934  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.519148  9617 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.648741  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.654894  9717 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.655349  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.655815  7804 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:52.656260  7655 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:52.649684  9419 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.657588  9419 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.655725  9717 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.659925  9753 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:52.665254  9753 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:52.665838  9753 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:52.667148  9753 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.669381  9753 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.639958  9748 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.671913  7659 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:25:52.672796  9748 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.517601  9640 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.631456  9617 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.682905  9419 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.688493  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.611433  9724 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.651806  7660 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.697374  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.712246  9632 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.716493  9717 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:52.650300  9745 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.720865  9724 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.722648  9745 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.741338  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:52.732497  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.749480  7732 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:52.734567  9608 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.738767  9704 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.737211  9724 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:52.730476  9673 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.010s
I20260709 12:25:52.746798  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.746305  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:52.742725  9706 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.637s	user 0.015s	sys 0.001s
I20260709 12:25:52.760432  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.762068  7662 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:52.763480  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.763715  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.652694  9621 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:52.768844  7731 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.771538  7804 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:52.773223  9744 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:52.773556  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.773864  9744 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.778980  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:52.781646  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.773439  7660 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:52.778367  9621 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.790911  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:52.791917  8409 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.778424  7737 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:52.747047  9767 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.793583  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.775725  9744 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.800698  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:52.801563  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:52.802642  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
I20260709 12:25:52.807981  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:52.808787  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 173751us
I20260709 12:25:52.809660  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.784663  9640 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.761633  9673 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.811363  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.812107  7737 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:52.769954  9706 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.815115  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.821425  8407 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:52.821667  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.812927  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:52.813774  9594 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.815107  9725 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.834789  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.816172  9632 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.836406  8414 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:52.835592  7737 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:52.842825  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:52.831213  9730 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.678s	user 0.015s	sys 0.016s
I20260709 12:25:52.816812  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 59f766c3d13d4360bcdeb1d5f0f318a8 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8
I20260709 12:25:52.849560  9644 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:52.850921  9644 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:52.851593  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.851755  9644 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.852567  7660 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:52.849776  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:52.855672  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:52.858891  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 195568us
I20260709 12:25:52.860031  7657 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:52.865399  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:52.869886  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.879096  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:52.880297  7733 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:52.886443  9708 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.641s	user 0.022s	sys 0.011s
I20260709 12:25:52.887468  9706 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.888940  9621 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.896847  7655 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:52.898342  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.899184  7662 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:52.899401  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.901286  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:52.901747  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52101.3us
W20260709 12:25:52.903560  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.904991  8409 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:25:52.865311  9632 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:52.917366  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.900715  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.854844  9730 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.922441  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:52.922389  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57115.8us
I20260709 12:25:52.922295  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.921257  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:52.924767  8409 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:25:52.926378  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.926601  7661 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:52.926576  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:52.929885  9650 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.865831  9594 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.934120  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:52.930629  9650 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.927541  9704 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:52.943861  7661 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:52.944412  9757 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.627s	user 0.016s	sys 0.012s
I20260709 12:25:52.926368  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.945344  9788 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:52.946177  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.946283  9788 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.949031  9650 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.895040  9726 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.953720  7656 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:52.954202  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.935091  9786 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.956382  9786 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.957607  9747 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:52.958235  9747 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:52.959254  9747 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:52.960080  9786 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.961122  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:52.953387  9704 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:52.952435  9714 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:52.965822  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:52.964422  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:52.966818  7810 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.967018  7804 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 2: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:52.962916  9704 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:52.972129  9733 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.884s	user 0.018s	sys 0.000s
I20260709 12:25:52.972311  9730 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:52.973233  9673 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:52.979472  9617 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:52.980075  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:52.981761  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:52.984206  9788 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:52.986282  9757 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.987335  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:52.987650  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:52.988829  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:52.989171  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:52.989880  9594 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:52.989997  7804 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:52.991824  7672 tablet_copy_source_session.cc:215] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:52.997442  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:52.998615  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64563.4us
I20260709 12:25:53.037223  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.038293  9617 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:53.038502  9327 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.039172  7659 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.039337  9327 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.040189  9327 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.042790  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:53.044412  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.046554  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.047258  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.050675  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.051712  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.010049  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.016345  9757 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:53.056325  9757 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.047736  7656 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:53.056769  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.057878  9753 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:53.058284  9753 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.058696  9753 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.059952  7735 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:53.059464  9644 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.026856  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.008549  9767 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.067780  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:53.069279  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80186.1us
I20260709 12:25:53.001610  7810 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:53.081068  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152677us
I20260709 12:25:53.083374  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.083369  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:25:53.086558  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 97938.1us
I20260709 12:25:53.087425  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.087809  9786 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.087898  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.088979  9749 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.088280  9327 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.082041  9594 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.091743  9594 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.092249  9594 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.093379  9327 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.094600  9327 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.095160  9327 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.096269  9327 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.090546  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:53.101711  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.102284  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.107304  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:53.114264  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46609.1us
I20260709 12:25:53.119163  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.120195  9767 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.121771  9749 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.101938  9725 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.119417  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:53.123194  8409 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:53.014201  9733 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.125026  9644 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.125629  9725 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.131924  9725 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.133157  9726 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:53.128774  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.129002  9717 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.138864  9717 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:53.118420  9795 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:53.130939  9731 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.143116  9717 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:53.143337  9795 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.125773  9749 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.135555  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.134999  9767 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:25:53.146956  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f766c3d13d4360bcdeb1d5f0f318a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:53.137405  9797 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.137437  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:53.130455  9786 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.155298  9786 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.155783  9786 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:53.157267  9786 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.144153  9795 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.147522  9767 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.157958  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.159691  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.160061  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.157439  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.148459  7807 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:53.147413  9749 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.149199  9723 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:53.166424  9723 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.167146  9723 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:53.149546  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:53.144855  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.146348  9327 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.147590  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.172206  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.161674  7660 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:53.151940  9744 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.164793  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:53.167591  7656 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:53.177819  8414 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:53.179553  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.179658  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:53.180233  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.180383  7661 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:53.144315  9731 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:53.178391  9789 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:25:53.185109  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:53.185228  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44376.9us
I20260709 12:25:53.183013  9327 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.185887  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.186185  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.185914  7655 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:53.158910  9767 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:53.159070  9753 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.190527  9419 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.186326  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.191046  9704 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.191337  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.191922  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.192022  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:25:53.184731  9788 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.192812  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.192945  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.193231  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.193553  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.194449  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:53.163410  9797 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:25:53.208663  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.188356  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.201058  7807 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Received request to transfer leadership to TS 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:53.201627  9747 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.183732  9731 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:53.205332  8407 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:53.206487  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:53.207369  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.136917  9757 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:25:53.202987  7803 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:53.216132  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.181414  9644 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:25:53.217984  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.199612  9419 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.202759  9704 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.244222  9744 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.150499  9717 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.246171  9731 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.211936  9797 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.253547  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.256188  9753 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.257387  9786 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.258795  9747 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.259176  9704 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.259969  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.267371  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:53.246691  9419 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.281600  9731 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.287263  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.265053  9650 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.282186  9795 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.266705  9747 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.294371  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.305341  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:53.307006  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.295855  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107570us
I20260709 12:25:53.317085  9717 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.318395  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:53.319609  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82336.6us
I20260709 12:25:53.309738  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:53.320299  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 107256us
I20260709 12:25:53.322101  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.323148  9706 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.323750  9706 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.324218  9706 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:53.327908  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60667.8us
I20260709 12:25:53.330922  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:53.331991  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 114170us
I20260709 12:25:53.333443  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.292244  9650 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:53.336163  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.337983  7810 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:53.307271  9747 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.340111  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.341014  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.345404  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.341403  9803 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.347936  9327 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:53.280016  9767 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.347791  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:53.337122  9733 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.384804  9327 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:53.338543  9806 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.343015  9789 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.343590  9744 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.392374  9644 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:53.392791  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45089.6us
I20260709 12:25:53.397157  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.397157  7735 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:53.411850  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.397445  9327 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.403905  9767 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.387369  9673 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.416488  9327 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:53.417557  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 156951us
I20260709 12:25:53.417877  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.388443  9753 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:53.417624  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.391526  9804 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:53.426391  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 116807us
I20260709 12:25:53.423123  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:53.430058  7660 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:53.425333  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104351us
I20260709 12:25:53.420300  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.410084  9644 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.431528  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:53.433374  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.436427  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:53.416366  9803 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.421263  9806 raft_consensus.cc:993] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: : Instructing follower 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:53.441268  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:53.445549  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 114139us
I20260709 12:25:53.413749  9798 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:53.421041  9733 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.456506  7729 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:53.453867  7659 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.459935  7655 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.460268  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.460551  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.454385  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:53.462669  8412 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 2: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:53.440378  9644 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.426195  9804 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.468637  8414 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.473043  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:53.479671  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.480334  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.428556  9734 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.485692  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.441721  9795 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.427408  9807 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:53.445204  9805 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.502014  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.495177  9807 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.504238  9807 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:53.508635  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.513137  9814 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:53.513804  9814 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.513609  9723 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.514249  9814 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.518527  9807 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.523947  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.468190  9733 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.455015  9798 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.480401  9804 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.524986  7735 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:25:53.552162  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110983us
I20260709 12:25:53.558794  7661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:25:53.559844  7661 raft_consensus.cc:606] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.563194  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:53.564426  7660 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.566322  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.567026  9807 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.430420  9650 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.572587  9723 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.572480  9747 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:53.573392  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47201.5us
I20260709 12:25:53.574330  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.575685  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.497506  9734 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.574983  9747 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.511068  9797 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.583575  9640 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.584941  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.585588  9745 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.585666  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.586199  9745 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.586722  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.587142  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.587347  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.497221  9805 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.588129  9805 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.588541  9815 raft_consensus.cc:1081] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Signalling peer 992d6df95ef74f7f8b12534534a45501 to start an election
I20260709 12:25:53.588658  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.596695  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:53.591817  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 153835us
I20260709 12:25:53.605209  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.578452  9725 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.582868  9797 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:53.621587  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:53.607059  9706 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.626821  8408 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.628185  9706 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.629354  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.630781  9818 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.586750  9745 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.535832  9733 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:53.610566  9594 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.004s
I20260709 12:25:53.596658  9809 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:53.667719  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 146417us
I20260709 12:25:53.542001  9798 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.674328  7808 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:53.674847  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53391.1us
I20260709 12:25:53.615953  9723 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:53.678442  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61656.9us
I20260709 12:25:53.677906  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.679843  8414 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.682350  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.683971  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.685554  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.689670  8408 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:53.703066  9802 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:53.706907  9725 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:53.707757  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.708695  7735 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 2: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:53.712849  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.671314  9809 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.715431  9809 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.717432  7730 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.699182  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.677800  9806 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.721460  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.684805  9706 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:53.722949  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.723646  9706 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.681246  9723 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.729682  7660 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:53.739334  9726 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.740690  9726 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:53.741118  9726 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:53.741750  9734 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.670279  9594 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.672572  9749 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.677524  9419 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.749338  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.804612  7736 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.833245  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.750594  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.702024  9818 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:53.847302  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 159958us
W20260709 12:25:53.851832  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152766us
I20260709 12:25:53.857146  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.858007  8414 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:53.858611  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.852738  7731 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:53.859162  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:53.859896  7732 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:53.860247  9757 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.861102  8408 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:53.861164  9757 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.839797  9706 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:53.863396  9797 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.864671  9810 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.865127  9810 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:53.866287  9810 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.868238  9810 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.690073  9795 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:53.822255  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:53.856472  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.858669  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.888471  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.889693  7804 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 2: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:53.889611  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:53.886649  7735 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:53.892238  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.895198  7804 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:53.896927  7807 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.872107  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:53.809257  7734 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.873729  9640 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:53.910841  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.911640  7812 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.873929  9650 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.874624  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:53.913241  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:53.742278  9644 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.795893  9734 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.875193  9797 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.853009  9594 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.876238  7805 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:53.877501  9704 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.000s
W20260709 12:25:53.840030  9816 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:53.900981  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.904332  7730 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.909533  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:53.806680  9725 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:53.737715  9723 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:53.744894  9726 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.914605  7728 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:25:53.916426  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:53.917054  7661 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 2: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:53.832846  9822 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.788964  9802 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:25:53.854255  9818 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.934190  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.893738  9795 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:53.935211  7803 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.925458  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:53.805521  9617 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:25:53.945284  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.941296  9704 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.935010  9807 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:25:53.945966  9749 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.945842  9813 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.946137  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:53.953308  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:25:53.955029  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.957311  7559 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:25:53.959339  9805 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.913239  9767 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.965345  9745 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:53.965971  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 265) took 1708 ms. Trace:
I20260709 12:25:53.964154  9734 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.975971  7728 pending_rounds.cc:85] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:25:53.976733  7808 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 2: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:53.977871  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:53.982168  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56836.1us
I20260709 12:25:53.983772  7810 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:25:53.984726  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50591.9us
I20260709 12:25:54.027984  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.029757  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:53.978770  9818 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:53.978519  9802 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.979390  9795 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.056928  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.012665  7812 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:53.982571  9800 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:54.024910  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.063874  7734 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:53.975947  9726 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:53.993508  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.016368  7557 rpcz_store.cc:276] 0709 12:25:52.257748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.258053 (+   305us) service_pool.cc:224] Handling call
0709 12:25:53.965776 (+1707723us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:53.096202 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:53.133160 (+ 36958us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:53.133221 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:25:53.133244 (+    23us) tablet.cc:662] Decoding operations
0709 12:25:53.137951 (+  4707us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:53.138053 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:53.138084 (+    31us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:25:53.138610 (+   526us) tablet.cc:701] Row locks acquired
0709 12:25:53.138659 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:25:53.138840 (+   181us) write_op.cc:270] Start()
0709 12:25:53.138995 (+   155us) write_op.cc:276] Timestamp: P: 1783599953138769 usec, L: 0
0709 12:25:53.139016 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:25:53.181894 (+ 42878us) log.cc:844] Serialized 1298 byte log entry
0709 12:25:53.746161 (+564267us) op_driver.cc:464] REPLICATION: finished
0709 12:25:53.954235 (+208074us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b1800091c60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:25:53.954486 (+   251us) write_op.cc:301] APPLY: starting
0709 12:25:53.954652 (+   166us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:53.954912 (+   260us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:53.954976 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:53.955843 (+   867us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:53.956257 (+   414us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:25:53.956311 (+    54us) write_op.cc:312] APPLY: finished
0709 12:25:53.956618 (+   307us) log.cc:844] Serialized 35 byte log entry
0709 12:25:53.959514 (+  2896us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:53.959666 (+   152us) write_op.cc:454] Released schema lock
0709 12:25:53.959731 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:53.960341 (+   610us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:53.960367 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":143380,"apply.run_cpu_time_us":3221,"apply.run_wall_time_us":5830,"num_ops":3,"prepare.queue_time_us":233036,"prepare.run_cpu_time_us":9054,"prepare.run_wall_time_us":200499,"raft.queue_time_us":369388,"raft.run_cpu_time_us":11612,"raft.run_wall_time_us":261734,"replication_time_us":540623,"spinlock_wait_cycles":27208576,"thread_start_us":567019,"threads_started":4,"wal-append.queue_time_us":60198}]]}
I20260709 12:25:53.988596  7737 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:53.984575  9821 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:54.068490  7736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:25:54.069267  7736 raft_consensus.cc:606] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:53.989406  9419 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:53.994643  9820 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:54.075474  9820 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:54.076313  9734 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:53.980360  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.076666  9820 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:53.988300  9704 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.051952  9802 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.078673  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:54.080569  9745 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.026042  7728 pending_rounds.cc:107] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:25:54.081444  7728 raft_consensus.cc:2889] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.056217  8408 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:53.983521  9730 consensus_queue.cc:1175] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 12:25:53.984252  9823 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.974990  9725 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:54.083752  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.084913  7812 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:25:53.971161  9733 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:54.093926  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 203749us
I20260709 12:25:54.009217  9804 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.051263  9749 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.097290  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:53.995831  9814 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.084906  9815 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.124707  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:54.125193  7659 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:54.068535  9821 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:53.977735  9822 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:54.121222  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:54.128126  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:25:54.128289  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 88235.9us
I20260709 12:25:54.095021  9725 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.130157  7804 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.092530  9734 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.137225  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.138731  7658 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:54.138605  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.093025  9800 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.136786  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.053198  9798 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.152300  7655 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.114986  9826 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:54.145576  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.155372  7805 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:25:54.156977  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90049us
I20260709 12:25:54.146365  9419 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:54.160667  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63432.3us
I20260709 12:25:54.164017  9733 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.164932  9798 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.167389  9833 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:54.167878  9833 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.168838  9833 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:54.173409  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52295.5us
I20260709 12:25:54.142246  9821 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.078941  9802 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.148099  8407 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
 from {username='slave'} at 127.0.0.1:40588
I20260709 12:25:54.181360  8407 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:54.181813  8407 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.180011  7654 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:25:54.191572  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.193080  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.194584  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:54.180505  9786 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.905s	user 0.014s	sys 0.000s
I20260709 12:25:54.200846  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.196959  9826 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:54.203147  9617 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:54.202786  9800 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.203486  9806 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.203737  9617 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.204566  9806 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:54.204823  9617 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.206629  9617 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.206959  9822 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.208246  9788 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:54.208915  9788 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.204841  9747 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:54.210158  9747 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:54.209913  9788 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.211195  9747 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.212371  9747 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.212863  9757 raft_consensus.cc:993] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: : Instructing follower d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:54.214406  9805 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.093281  9823 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.217592  9823 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:54.220134  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.207016  7803 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:54.223531  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47807.8us
I20260709 12:25:54.225540  9814 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.226438  9644 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.148372  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.180254  9734 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.181684  9745 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:54.227183  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.230432  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.254729  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.244855  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:54.261353  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60574.4us
I20260709 12:25:54.227514  9827 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:25:54.265470  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.265697  9726 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.266316  7737 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.266578  9731 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.245304  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:54.267900  8411 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:54.267810  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.268582  7653 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.239914  9825 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:54.269883  8408 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:54.270205  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.272176  9327 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:54.272572  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.262523  8407 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.265220  9822 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:54.239743  9804 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:54.269282  9825 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.244166  9829 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:54.241319  9730 raft_consensus.cc:1081] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Signalling peer d72176ccaed049958ac68f706d4f052d to start an election
I20260709 12:25:54.240353  9745 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.233366  9704 raft_consensus.cc:993] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:54.238538  9734 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.285387  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:54.274712  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100142us
I20260709 12:25:54.278656  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.288851  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.281332  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:54.294287  7730 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.291565  9704 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:54.292068  9789 raft_consensus.cc:1081] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:25:54.279273  9829 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.296062  8407 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.297911  7728 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:54.296763  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.302313  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.314191  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.317833  7803 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:54.293593  9804 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.320794  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:25:54.314996  9778 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.000s
I20260709 12:25:54.329663  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:25:54.336447  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43687.7us
I20260709 12:25:54.337394  7812 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:54.288578  9757 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.337685  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:54.319890  9809 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.002s
I20260709 12:25:54.347862  9809 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.352006  9836 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.352983  9836 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.355783  9825 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.298398  9704 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:54.313265  9832 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.352169  9809 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.358249  7807 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:54.316555  9730 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.337925  7812 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:54.362643  7737 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:54.364197  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:54.364965  7809 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
W20260709 12:25:54.368464  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47733.6us
I20260709 12:25:54.369364  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.372087  9833 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.274556  9821 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.329167  9644 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:54.356091  9828 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:54.387181  9327 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:54.389257  9327 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.389678  9327 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.384655  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.280757  9745 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.394447  9834 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:54.427485  7737 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.376667  9807 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:54.431735  7653 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:54.379736  9822 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.437959  7730 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
W20260709 12:25:54.438091  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44433.1us
I20260709 12:25:54.390376  9820 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:54.395990  9810 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.005s
I20260709 12:25:54.366746  9827 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.458732  9327 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:54.461102  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 166574us
I20260709 12:25:54.463832  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 317) took 1272 ms. Trace:
I20260709 12:25:54.360181  9818 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.466451  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:25:54.457305  9816 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:54.460582  9644 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.477077  9644 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.479839  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.471093  7809 rpcz_store.cc:276] 0709 12:25:53.186985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.187155 (+   170us) service_pool.cc:224] Handling call
0709 12:25:54.459566 (+1272411us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:54.459717 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291144960}
I20260709 12:25:54.473814  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.457005  9825 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.472252  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:54.473820  7736 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:54.483533  9327 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.483511  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.485778  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:54.485561  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:54.486949  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.487450  9810 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:54.487939  9810 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.488343  9810 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.460901  9834 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.496898  9327 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:54.497184  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.498097  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.498575  9327 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:54.499202  9327 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.491923  9827 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.501045  9757 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:54.501502  9757 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.503257  9834 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.506954  9811 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:54.507510  9730 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:54.507702  9811 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.509123  9617 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:54.510079  9778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40594 (request call id 474) took 3534 ms. Trace:
I20260709 12:25:54.511416  9836 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.510520  9778 rpcz_store.cc:276] 0709 12:25:50.980119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.980859 (+   740us) service_pool.cc:224] Handling call
0709 12:25:51.668810 (+687951us) ts_tablet_manager.cc:934] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
0709 12:25:53.143040 (+1474230us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:53.567770 (+424730us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:53.568009 (+   239us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:54.377286 (+809277us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:54.456559 (+ 79273us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:54.493059 (+ 36500us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:54.509946 (+ 16887us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":40344,"tablet-copy.queue_time_us":603076,"thread_start_us":621982,"threads_started":2}
I20260709 12:25:54.494861  9828 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.470335  9807 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:54.502462  9757 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.520898  9795 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.495359  9822 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.514317  9778 tablet_copy_client.cc:806] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:54.507050  9813 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:54.495164  9803 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:54.533612  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.534125  9803 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.532829  9789 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:54.535451  9789 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.535961  9789 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.472909  9723 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.513975  9836 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:54.541986  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:54.548928  7809 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:54.539870  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.545400  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:54.545961  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.388614  9829 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.550022  7808 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:54.551476  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.552690  7657 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.474931  9704 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.558408  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.559455  7660 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.558461  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:54.560743  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63618.1us
I20260709 12:25:54.561262  7808 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:54.563319  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.564057  9757 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.565133  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.562594  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.560477  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:54.563593  9704 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.571039  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.568321  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.570196  7810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:25:54.572878  7810 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:54.573319  7810 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.573556  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.510754  9811 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.574697  7806 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.559314  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.575745  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:54.533727  9813 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:25:54.589233  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49441.8us
I20260709 12:25:54.593995  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.584232  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.587739  7807 raft_consensus.cc:2368] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3.
I20260709 12:25:54.584599  9726 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:54.580807  7808 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:54.598434  7660 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:25:54.537784  9822 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:25:54.602890  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.600943  7810 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.609476  9811 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.610553  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:54.611449  9811 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.612883  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.603941  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.598264  9726 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.614347  9726 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.587306  9778 tablet_copy_client.cc:670] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:54.586503  9821 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.583964  9747 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:25:54.623137  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.616075  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.578640  9805 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.617106  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.615696  9726 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.608726  9327 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.559188  9650 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:25:54.565497  9797 consensus_queue.cc:1175] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 12:25:54.626356  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.627568  9747 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:54.627575  8411 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:25:54.635804  7736 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.636169  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.638741  7730 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.634809  9733 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:54.664887  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.600976  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.639547  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:54.666101  7728 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:54.617413  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.533665  9807 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.608340  9786 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:54.648273  9327 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:54.650504  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.650712  7810 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.653359  9821 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.587770  9803 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.660574  9832 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.662250  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:54.663031  9734 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:54.663980  9778 tablet_copy_client.cc:538] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:54.664156  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:54.619786  9811 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:54.666654  7660 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.667181  7812 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.634976  9795 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.679399  7812 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.683573  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.684666  7730 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:54.692206  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:54.678226  9733 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:54.693713  9733 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:54.695515  9733 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.675408  9734 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:54.698671  9734 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.700081  9734 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:54.667107  9723 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.705263  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:54.714865  8409 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:54.716524  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.717339  7730 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.724413  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:54.728433  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 124586us
I20260709 12:25:54.729565  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:54.730005  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:54.730815  9327 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:54.731353  9327 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:54.732853  9327 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:54.736770  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.738907  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.739521  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:54.740962  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.748492  7730 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:25:54.757124  7812 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:54.728039  9795 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:54.759795  7626 leader_election.cc:400] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3.
I20260709 12:25:54.755007  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:54.738916  9723 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:54.744441  9786 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:54.749572  9617 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:54.750525  9818 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.772951  7657 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:54.775053  9810 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.783176  7734 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.789706  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.779309  9617 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.768800  9786 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:54.779009  9810 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.769117  9795 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:54.800839  7810 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:54.779666  9731 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:25:54.723006  9813 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:54.840560  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.806406  9617 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.810374  9786 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:54.910209  9725 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.743s	user 0.007s	sys 0.005s
I20260709 12:25:54.910746  9725 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.912868  9805 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.913498  9725 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:54.915943  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:54.920897  7661 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:54.933102  9795 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.944567  9802 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:54.947827  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:54.948875  7654 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:54.949774  9617 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:54.950378  9818 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.950928  9818 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:54.952405  9617 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:54.955507  9826 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.956463  9826 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.967651  7662 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:54.954869  9822 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:54.975905  9822 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:54.977367  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.979878  7736 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:25:54.980455  7736 pending_rounds.cc:85] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:25:54.980799  7736 pending_rounds.cc:107] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:25:54.981091  7736 raft_consensus.cc:2889] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.902670  9810 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:54.991652  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:54.996129  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 241187us
I20260709 12:25:54.979626  9789 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:54.978828  9822 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:55.017400  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.008566  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:54.899206  9834 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.018935  7810 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:55.020951  7807 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 5. Ops: []
I20260709 12:25:54.994199  9747 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.030859  7733 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:55.032407  7737 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:55.032434  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 285885us
I20260709 12:25:55.038364  7804 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:55.039894  7736 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:55.037428  9745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.033s
W20260709 12:25:55.044978  9745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.000s	sys 0.033s
I20260709 12:25:55.012651  9841 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:25:55.015970  9844 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:25:55.051136  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.057680  7734 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:55.056747  7810 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:25:55.060699  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.029951  9810 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:25:55.058138  8412 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.103272  9810 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:55.103809  9810 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:55.104166  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 447) took 1064 ms. Trace:
I20260709 12:25:55.070402  9818 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.034596  9747 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.076495  9803 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.078420  7656 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:55.081881  7736 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:55.058563  9847 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.085176  9800 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.086561  9789 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.088043  9834 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.092133  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1798) took 1161 ms. Trace:
I20260709 12:25:55.093422  9829 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.166366  9801 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:55.095389  9778 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:25:55.098606  9640 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.104688  8412 rpcz_store.cc:276] 0709 12:25:54.039754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.052162 (+ 12408us) service_pool.cc:224] Handling call
0709 12:25:54.069920 (+ 17758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.103255 (+1033335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.103875 (+   620us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b5400780354. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:55.104096 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222076672}
I20260709 12:25:55.112068  9836 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:55.114410  9826 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.117169  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:55.128567  7810 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.133822  7804 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:25:55.164494  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.104959  9810 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.093703  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:55.274259  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 180625us
I20260709 12:25:55.165897  7806 rpcz_store.cc:276] 0709 12:25:53.930778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.083901 (+153123us) service_pool.cc:224] Handling call
0709 12:25:54.098947 (+ 15046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.579838 (+480891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.091741 (+511903us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400011654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:55.092042 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093581184}
I20260709 12:25:55.177596  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.179792  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.170569  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.277150  7659 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:55.277643  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 107174us
I20260709 12:25:55.278604  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:55.189364  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 165577us
I20260709 12:25:55.249665  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:55.250805  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64538.3us
I20260709 12:25:55.252978  7658 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:55.282277  7660 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.282470  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:55.261487  7654 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.164750  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:55.271775  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 158302us
I20260709 12:25:55.284194  7661 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:55.286906  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97709.5us
I20260709 12:25:55.253254  9640 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:55.293025  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:55.285313  9821 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:55.190450  9833 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:25:55.296504  8413 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.255336  9836 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:55.188004  9847 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.167192  9803 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:55.288156  9804 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.943s	user 0.013s	sys 0.000s
I20260709 12:25:55.285197  9726 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:55.288823  9800 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:25:55.303542  7660 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:55.259449  9650 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:25:55.255753  9825 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:25:55.302945  7661 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:55.289276  9815 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.286825  9823 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:55.342139  9826 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:55.351266  9826 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.353037  9805 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:55.342433  9804 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:55.355676  9804 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:55.356272  9804 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.357582  9804 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.337116  9831 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.359817  9831 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.361346  9800 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.302801  9820 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:25:55.364956  9831 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.365454  9723 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.366036  9818 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:55.371690  9802 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.372046  9818 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.372393  9802 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.373438  9847 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:55.342326  9644 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:55.367383  7653 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:55.343241  9821 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:25:55.368620  9826 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.295738  9828 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:25:55.295563  9617 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:25:55.303094  9725 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:55.371119  9804 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:55.383560  7654 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:55.370378  9803 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.299453  9844 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:25:55.338358  9747 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.401190  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:55.403065  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.374522  9818 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:55.344758  9856 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:25:55.409562  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.386696  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.419724  7733 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
W20260709 12:25:55.413034  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120025us
I20260709 12:25:55.371757  9723 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.386142  7659 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:55.371673  9823 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.419734  7736 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:55.417883  9731 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.399039  9802 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:55.402701  9725 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.427006  7659 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.429291  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.407837  9726 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:55.430867  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:55.440320  7659 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.445264  9823 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.446534  7661 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.450168  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.451296  7658 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:55.452453  7655 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.455396  7654 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:55.454209  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:55.456934  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 168369us
I20260709 12:25:55.458611  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.435510  9725 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.460049  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.462455  7660 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.461344  7659 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45926
I20260709 12:25:55.428586  9747 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:25:55.463199  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.467777  7659 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:55.460651  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:55.456235  9813 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:55.474325  9813 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.474845  9813 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.475428  9844 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:55.476145  9844 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.479938  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:55.434271  9731 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.467262  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:55.461796  9644 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.471006  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:55.483726  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.463902  9805 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.498315  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:55.497404  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.497325  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.470082  7659 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.499624  7661 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:55.490130  7660 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:55.485747  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.498698  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.504747  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:55.504527  7733 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:55.485407  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.496929  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:55.480865  9861 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:55.505250  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.511837  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:55.503479  9786 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.004s
I20260709 12:25:55.532126  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.529871  9704 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:55.534272  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.534714  9704 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:55.536144  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:55.535822  9704 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.537186  9704 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.540419  9733 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:55.481452  9831 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.521147  7737 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:25:55.508023  7732 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:55.522297  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.509930  9731 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.523483  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.524377  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.519533  9726 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:55.479260  9821 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:55.528040  9861 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.479912  9723 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.511168  9644 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.531092  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:55.541776  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.516306  9805 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:25:55.544308  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.549634  9734 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:55.550251  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1551) took 1004 ms. Trace:
I20260709 12:25:55.552417  7735 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:55.471554  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.522475  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:55.551174  9617 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.508594  9809 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.511559  9815 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:55.510543  9844 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.590715  9809 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.592279  9809 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.593073  7810 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.593541  9809 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.594638  7812 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:55.595340  9617 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.595824  7808 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:55.597074  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.598018  9617 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.599086  7811 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:55.603518  7807 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.604486  9644 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:55.605870  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.602762  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.607971  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.555428  9786 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:55.549678  9857 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:25:55.628929  7732 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.606392  9640 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.630309  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:55.631086  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:55.632615  9807 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.633349  9807 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.641146  7657 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:55.641666  7657 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:55.644083  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:55.645030  7661 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:55.646196  9640 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.648784  9836 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.650522  9836 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.653174  7808 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:25:55.653702  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.654796  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:55.655987  7653 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:55.656751  7555 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: config changed from index -1 to 4, term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to <none>, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:55.658473  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:55.661432  9846 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:55.662117  9846 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.662642  9846 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.642625  7657 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.663673  7657 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:25:55.668133  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.666589  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:55.669466  7658 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:55.670841  9786 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.634567  9807 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.676187  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:55.686657  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:55.687793  7654 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:55.693364  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:55.558502  7659 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:55.559428  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.700s (SENT)
I20260709 12:25:55.571617  9723 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.572788  9861 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.579743  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:55.579541  7656 rpcz_store.cc:276] 0709 12:25:54.546071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.546904 (+   833us) service_pool.cc:224] Handling call
0709 12:25:55.550030 (+1003126us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54003985d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:55.550177 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106854144}
W20260709 12:25:55.586414  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 114953us
I20260709 12:25:55.712215  8379 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.588485  9734 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:55.588665  9818 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:55.557327  9831 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.592656  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:55.719138  8414 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:55.560331  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.618156  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.618865  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:55.620541  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98176us
I20260709 12:25:55.721690  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:55.621260  9747 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:55.727684  9747 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.621812  9844 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.623472  7731 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:55.623860  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:55.739522  7734 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:55.624656  9805 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.570181  9801 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:55.626483  9810 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.672426  7730 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:55.741672  7803 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:55.694643  7807 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:55.695464  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.710582  7657 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:25:55.716455  7804 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 5. Ops: []
I20260709 12:25:55.581661  9733 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.720958  7733 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:25:55.731757  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:55.682964  7555 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.748833  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.749521  7659 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.745762  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:55.750437  7806 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:55.750690  7731 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.743939  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:55.754390  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:55.707777  9807 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.767021  8413 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:55.769817  7734 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:55.725276  9852 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.766366  9861 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:55.766395  9832 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:55.771921  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:55.772539  9832 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.773087  9832 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:55.771497  9852 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.775923  7736 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:55.777271  9852 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:55.715855  9822 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:55.782755  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 223333us
I20260709 12:25:55.638897  9813 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.782789  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.789286  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:55.789817  7737 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:55.790015  7807 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.791371  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:55.763163  9829 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:55.777262  9827 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:25:55.800644  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:55.802269  7806 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:55.726143  9826 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:25:55.779285  9841 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:55.809522  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:55.789840  7803 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:55.811175  9861 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:55.810791  9734 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:55.796164  9804 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.820250  9840 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:55.821165  9825 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:55.825481  9829 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.829895  8409 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
I20260709 12:25:55.829293  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:55.783437  9822 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.836683  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:55.797851  9828 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:55.791280  9821 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:55.832077  9789 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:55.833029  9841 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.789978  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.858987  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:55.888624  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98875.5us
I20260709 12:25:55.889564  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.890671  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:55.880645  9861 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:55.768812  9801 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.786528  9847 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:25:55.856433  9834 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.820299  9725 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.865178  9829 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.841799  9617 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:25:55.855878  9733 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:55.864807  9822 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.864843  9804 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.888437  9841 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.895560  9757 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:25:55.940081  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81153.3us
I20260709 12:25:55.879962  9864 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:25:55.866058  9825 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:55.943208  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.903400  9823 raft_consensus.cc:2764] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:25:55.905047  9805 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:55.929793  9786 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.906088  9862 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:55.906057  9834 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:55.934028  9788 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.896242  9767 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:25:55.936116  9821 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.965406  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:55.966975  7733 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.946751  9803 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:55.972507  9805 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.982246  9805 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:55.969633  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.502s (SENT) [suppressed 25 similar messages]
W20260709 12:25:55.987166  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.587s (SENT)
I20260709 12:25:55.973577  9786 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:55.969677  9788 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:55.989034  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:56.012580  7805 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:25:56.015113  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.856s (SENT)
I20260709 12:25:55.997856  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:56.025765  7734 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:56.004325  9801 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.026729  9813 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:56.028321  9873 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:25:56.031484  9804 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.034884  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:56.036993  9826 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:56.037544  9826 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:25:55.987918  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 158912us
W20260709 12:25:55.993330  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 153586us
I20260709 12:25:55.967355  9862 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:56.072751  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.072983  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.072461  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:56.073411  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:55.988271  9831 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.091244  7733 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:56.142331  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:56.038532  9826 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.070734  7812 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:56.025187  9840 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:56.146724  8414 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:55.996991  9725 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.007568  9788 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:56.144511  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 253877us
I20260709 12:25:56.151086  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:56.073907  9868 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:25:56.042198  9640 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.054823  9861 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.072543  9745 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.155241  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:56.158380  9879 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:25:56.160004  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:56.171499  7659 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:56.144672  9864 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.154634  9704 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:56.005254  9786 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:56.227109  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 76104.8us
W20260709 12:25:56.232484  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.864s (SENT)
I20260709 12:25:56.232872  7655 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:56.226287  7654 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:56.233286  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:56.235110  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90117.4us
I20260709 12:25:56.233289  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:56.242990  9848 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:56.242025  9826 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:56.223976  9789 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:56.076668  9821 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.235530  9805 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:56.234891  9725 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.244872  9809 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.001s
I20260709 12:25:56.179356  9640 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.241160  9836 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:56.173318  9861 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.280231  9825 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.076308  9834 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:56.247555  9841 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.269701  9725 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.253378  9862 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.255815  9815 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:25:56.288208  9725 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.292621  7734 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:56.289239  9861 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.241238  9856 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:25:56.250451  9704 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:25:56.267293  9726 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.225363  9745 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:56.325165  9853 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:56.324699  9884 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.325942  9883 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.086s
W20260709 12:25:56.350374  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.333701  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.935s (SENT)
I20260709 12:25:56.308385  9805 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:56.239831  9864 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.315812  9809 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:56.352679  7811 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:56.345091  7658 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:56.356482  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:56.330344  9726 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.326881  9886 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:25:56.358321  9805 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.365918  9841 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.366370  9745 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:56.368932  9832 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.004s
I20260709 12:25:56.368505  9809 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:56.372994  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.970s (SENT)
I20260709 12:25:56.372808  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:25:56.376322  9846 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.001s
W20260709 12:25:56.377162  9878 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:25:56.379834  9726 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.385958  9825 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:56.372714  9884 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.405340  7658 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:56.406046  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:56.400449  9844 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.005s
W20260709 12:25:56.458491  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 219235us
W20260709 12:25:56.465571  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92610.8us
W20260709 12:25:56.466547  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.990s (SENT)
W20260709 12:25:56.444864  9810 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.000s
I20260709 12:25:56.467131  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:25:56.472208  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.069s (SENT)
I20260709 12:25:56.482407  7728 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:56.386065  9864 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.465142  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.465905  9844 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:56.468967  9726 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:56.490798  9829 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.005s
I20260709 12:25:56.479468  9747 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.401511  9832 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.466228  9883 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:25:56.496274  9885 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.472986  9789 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:25:56.448362  9846 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.478864  9848 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:25:56.516304  9831 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.516726  9864 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.525239  9822 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.003s
W20260709 12:25:56.557416  9875 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.689s	user 0.008s	sys 0.014s
I20260709 12:25:56.558099  9809 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.535534  9831 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.562695  9804 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.519476  9894 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:25:56.564283  9847 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:56.565069  9801 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:56.565883  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:56.566082  7624 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:56.616338  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50461.3us
I20260709 12:25:56.531421  9747 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:25:56.567569  9650 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.544406  9877 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:56.520411  9810 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:25:56.523168  9829 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:56.604432  7627 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.184s (SENT)
I20260709 12:25:56.649076  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:56.571138  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:25:56.650611  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79555.6us
W20260709 12:25:56.610215  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85106.6us
I20260709 12:25:56.610448  7658 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:56.611675  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:56.614308  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:56.617254  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.531067  9885 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.620728  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.652590  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:56.660261  7807 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:56.621673  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:56.622375  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.661432  7654 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:25:56.622522  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.624714  9831 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.625424  9900 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:56.626536  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.605695  9726 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.632328  9889 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:25:56.632730  9650 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:56.633780  9897 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:25:56.634475  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.638305  9899 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:56.537386  9864 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.537774  9822 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.653831  8411 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:56.608286  9884 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.661738  7806 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:56.657117  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:56.613886  9804 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.653736  9846 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.630348  9809 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.811484  9899 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:56.812516  9822 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.812197  9809 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.813598  7660 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:56.813205  9899 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.814561  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.815059  9809 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.815557  9900 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:25:56.816277  9895 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:25:56.817086  9900 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.818828  9900 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:56.821000  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 164033us
I20260709 12:25:56.821987  9846 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:56.688275  7811 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:56.823186  9831 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:56.823993  9831 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.825217  9821 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:56.825759  9704 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.825865  9821 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.826879  9846 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:25:56.828078  9778 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:56.826393  9831 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.829069  9846 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.830003  9778 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:56.681361  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:56.695267  8407 raft_consensus.cc:2368] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:25:56.831188  7658 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:56.665066  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:25:56.832970  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 168027us
W20260709 12:25:56.837126  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 232740us
I20260709 12:25:56.838984  7809 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:25:56.815045  9899 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:56.853137  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.855406  7811 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:56.855465  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.868980  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.869843  7806 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:25:56.871418  7803 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:25:56.689280  9826 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:56.747485  9825 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.869364  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:56.731654  9757 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:56.869324  7804 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:56.876423  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.753474  9810 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:56.658401  9877 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:56.878347  8412 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:56.883850  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:56.879558  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:56.893040  7809 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:56.731340  9745 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.895605  9745 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.898101  9745 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.826475  9704 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:56.886361  9789 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:56.845358  9726 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.900939  9877 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:56.916108  9877 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.918619  8411 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:56.906682  9778 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:25:56.899130  9810 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:56.923660  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:56.885964  9825 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.889407  9757 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:56.928709  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.919857  9847 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:56.931138  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1812) took 1321 ms. Trace:
I20260709 12:25:56.920864  9789 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:56.958778  9757 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.936065  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:56.963004  9810 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:56.939445  9726 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.922118  9831 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:56.948484  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1810) took 1346 ms. Trace:
I20260709 12:25:56.935140  9825 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.950438  9704 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.953980  9731 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.954638  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:56.955468  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 374) took 1848 ms. Trace:
I20260709 12:25:56.996523  7728 rpcz_store.cc:276] 0709 12:25:55.106592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.106817 (+   225us) service_pool.cc:224] Handling call
0709 12:25:55.111382 (+  4565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.483335 (+1371953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.955056 (+471721us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002481d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:56.955352 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3133643136}
I20260709 12:25:56.956282  9650 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:56.948630  7812 rpcz_store.cc:276] 0709 12:25:55.609416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.617637 (+  8221us) service_pool.cc:224] Handling call
0709 12:25:56.930915 (+1313278us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400134cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:56.931060 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723351424}
I20260709 12:25:56.996529  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:25:56.956454  9829 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:25:56.957428  9885 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.959637  9778 ts_tablet_manager.cc:1403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 1.877s	user 0.049s	sys 0.023s
I20260709 12:25:56.959748  9804 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.960700  9844 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:56.961040  9864 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:56.961809  9821 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:56.964076  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.010455  7657 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 1.
I20260709 12:25:56.988637  9831 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.974164  9833 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.974040  8406 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:56.975068  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:57.033594  8408 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:57.033810  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:56.927531  8411 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.034090  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.034515  7804 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.034735  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:57.036087  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.036415  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:25:56.988225  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:56.988377  7805 rpcz_store.cc:276] 0709 12:25:55.601784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.602277 (+   493us) service_pool.cc:224] Handling call
0709 12:25:56.948249 (+1345972us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54003ebcd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:56.948402 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868922112}
W20260709 12:25:57.007963  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 124219us
I20260709 12:25:56.935747  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 701) took 1204 ms. Trace:
I20260709 12:25:57.037812  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:56.993695  9723 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.038241  7654 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:57.037510  8409 rpcz_store.cc:276] 0709 12:25:55.730982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.731449 (+   467us) service_pool.cc:224] Handling call
0709 12:25:56.935481 (+1204032us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:56.935641 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188055296}
I20260709 12:25:57.025169  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:57.031752  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:57.039485  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.040223  7809 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:57.044118  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:57.046098  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.037865  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.049017  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.051378  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.052125  7660 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:57.052271  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.052433  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.008260  9844 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:57.092590  7812 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:57.101446  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:25:56.958823  9789 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.008306  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.056124  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.099884  8411 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:57.015455  9831 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:57.092238  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 253995us
I20260709 12:25:57.047539  9828 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:57.004865  9731 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.092574  9836 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:57.107267  7657 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:25:57.104322  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51967us
W20260709 12:25:57.109494  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.091579  9757 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:57.107337  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:25:57.140110  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91193.4us
I20260709 12:25:57.050833  8405 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:57.092067  9864 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:57.048671  9733 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:25:57.044679  9833 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.048398  9723 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.046631  9829 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:25:57.295152  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157711us
I20260709 12:25:57.106258  9650 raft_consensus.cc:2764] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:57.193599  7804 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:57.236573  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129640us
I20260709 12:25:57.104425  9883 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:57.106148  9902 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.297576  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.136709  9836 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.154438  9844 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.310317  9844 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:57.312317  9726 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:57.146404  7660 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:57.312925  9726 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.315521  9904 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:57.315793  9726 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:57.316016  7660 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:57.140120  9757 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.223305  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1772) took 1146 ms. Trace:
I20260709 12:25:57.109349  9805 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:57.151748  9846 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:57.115954  9828 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.308856  9877 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.309480  9836 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.139810  9827 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.159s
I20260709 12:25:57.109750  9899 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.210906  9905 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.156191  9885 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:57.323045  7730 rpcz_store.cc:276] 0709 12:25:56.077032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.077228 (+   196us) service_pool.cc:224] Handling call
0709 12:25:56.078332 (+  1104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.288389 (+210057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.288428 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.289043 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.468128 (+179085us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:56.468303 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.149173 (+680870us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54001c4dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:57.149227 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.157148 (+  7921us) raft_consensus.cc:1685] finished
0709 12:25:57.157176 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.223148 (+ 65972us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5189,"raft.queue_time_us":6477,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":20733,"spinlock_wait_cycles":481896704,"thread_start_us":178628,"threads_started":1,"wal-append.queue_time_us":238791}
I20260709 12:25:57.331146  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:57.321336  9902 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.211489  9907 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:57.210197  9900 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 12:25:57.116214  7653 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:57.207512  9731 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:57.336315  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:57.209124  9911 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:57.209029  9704 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.132s
I20260709 12:25:57.393392  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:57.367981  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.394444  7728 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.336972  9757 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.373445  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.337563  9904 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:25:57.380687  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:57.400365  7732 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:57.374769  9789 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:57.374424  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.384878  7804 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:57.401671  7731 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:57.337294  9805 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:57.347792  9828 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:57.391120  7809 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:57.393638  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.395272  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.408008  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.408355  7730 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:57.373234  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:57.321197  9883 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.396957  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.414525  7737 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:57.397656  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.331578  9921 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.409258  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.417747  7731 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 1.
I20260709 12:25:57.418484  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.398420  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 437) took 1217 ms. Trace:
I20260709 12:25:57.420758  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.426491  7736 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:57.369449  9899 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.423699  7731 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:57.423595  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.402758  9919 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.429102  9833 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:57.429337  9723 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:57.404737  9911 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:57.431488  7812 raft_consensus.cc:1217] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Deduplicated request from leader. Original: 3.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:25:57.445983  7812 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:57.435338  9917 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
W20260709 12:25:57.436666  9913 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:57.400075  9885 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:25:57.438318  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:57.431439  9883 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:57.409790  9805 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:57.414620  9804 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.440757  7731 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:57.382864  9907 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.370888  9905 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.415815  9789 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:57.466027  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.569s (SENT)
I20260709 12:25:57.663411  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.447705  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.661468  7732 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:25:57.665550  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 199580us
W20260709 12:25:57.446846  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50714.6us
I20260709 12:25:57.466292  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:57.672798  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.607s (SENT)
I20260709 12:25:57.521606  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.674029  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.673434  7737 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:57.517148  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:25:57.685662  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.502s (SENT)
I20260709 12:25:57.424752  9921 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:57.495855  9914 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:25:57.445785  9911 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:57.671809  9804 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.512754  9902 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:57.494443  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:57.671869  9827 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:25:57.691865  9899 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:57.693044  9885 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:57.446820  9904 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.248s
I20260709 12:25:57.695878  7657 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:57.640281  9826 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.669006  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 144172us
I20260709 12:25:57.673357  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:57.688759  9825 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:57.672408  9640 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:57.515810  9856 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:57.702371  7730 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:57.447621  9918 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 3, Time since last communication: 0.253s
I20260709 12:25:57.671697  9833 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:57.732070  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 210589us
W20260709 12:25:57.734278  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61036us
I20260709 12:25:57.739300  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:25:57.740000  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.742408  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.743155  7808 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:57.743800  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.705224  9733 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.709239  9864 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:57.744819  7804 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:57.700695  9911 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:57.746016  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:25:57.747238  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 230213us
I20260709 12:25:57.425520  7653 rpcz_store.cc:276] 0709 12:25:56.169492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.174154 (+  4662us) service_pool.cc:224] Handling call
0709 12:25:57.006198 (+832044us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdf8c 00007fbffedfdef6 000000000036eb39 00007fc006e711e0 00007fc006f3a0c4 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:57.386453 (+380255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8365696}
I20260709 12:25:57.748646  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.733754  9726 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.754101  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:25:57.738389  9872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.013s	sys 0.009s
W20260709 12:25:57.757158  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.758054  7805 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:57.733155  8406 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:25:57.740227  9704 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:57.735814  9856 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:57.759550  9825 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:57.737389  9826 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.467525  9919 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.769886  7809 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:25:57.732833  9827 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.763963  9640 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:57.765413  9872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.252s	user 0.014s	sys 0.010s
I20260709 12:25:57.752558  9733 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:57.814309  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60793.8us
I20260709 12:25:57.814761  9856 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:57.817312  9856 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:57.763587  9726 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:57.838229  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 99071.8us
I20260709 12:25:57.793515  9640 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.763448  9928 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:57.852383  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:57.854430  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.857183  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:57.847694  9904 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:57.858482  9734 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:57.800884  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:57.863566  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.861246  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:57.872079  9933 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:57.873572  9936 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:25:57.881875  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.882702  7735 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:57.883564  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:57.883788  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:57.884517  7736 raft_consensus.cc:2376] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3. Re-sending same reply.
I20260709 12:25:57.890105  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.891136  7805 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:25:57.861460  9640 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:57.847961  9704 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:57.860152  9617 raft_consensus.cc:2764] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:25:57.899768  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:57.883004  9797 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:57.869665  9825 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:57.860834  9928 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.878134  9905 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:57.889287  9904 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:57.895525  9789 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:57.864106  9918 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 12:25:57.908782  9844 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.005s
I20260709 12:25:57.804981  9778 raft_consensus.cc:359] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:57.894694  7653 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:57.926916  7805 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:57.905715  9936 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.932263  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 183739us
I20260709 12:25:57.914871  9928 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:57.914387  9899 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:57.913938  9864 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:25:57.997923  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:57.906492  9929 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:25:58.005180  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:57.917363  9797 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:58.034606  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 173050us
I20260709 12:25:57.992394  9907 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:57.995708  9933 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:58.056486  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:58.004802  9899 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.066532  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:58.027109  9883 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:57.990571  9918 consensus_queue.cc:1243] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:58.056656  9939 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.040402  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:58.003612  9778 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
W20260709 12:25:58.066778  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75220us
I20260709 12:25:58.036689  9804 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.035910  9789 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.990597  9825 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.056936  9905 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:58.057057  9617 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:25:58.074805  9899 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:58.139422  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:58.139920  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:58.070247  9937 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.132196  8406 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
I20260709 12:25:58.176137  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:58.217221  7805 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:25:58.078502  9929 consensus_queue.cc:1243] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:58.197647  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:58.080440  9921 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.014504  9936 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.102221  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:58.238936  7728 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.000s
I20260709 12:25:58.243996  7728 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:25:58.057401  9704 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.189469  9902 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.203540  9905 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.247766  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1577) took 2075 ms. Trace:
I20260709 12:25:58.203598  9932 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:58.106415  9918 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.208381  9789 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.235692  9804 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.256536  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 216209us
I20260709 12:25:58.251772  7662 rpcz_store.cc:276] 0709 12:25:56.172584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.173863 (+  1279us) service_pool.cc:224] Handling call
0709 12:25:58.197363 (+2023500us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:58.247694 (+ 50331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15205632}
I20260709 12:25:58.271998  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:25:58.037891  9927 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.182s
W20260709 12:25:58.263147  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.134254  9926 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:58.266039  7736 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:58.267570  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:58.106326  9883 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.208555  9825 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.252187  9726 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.323148  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:58.324265  7728 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:58.256126  9905 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:58.247853  9929 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.320703  7735 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:58.334331  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:58.081087  9904 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.345293  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:25:58.278851  9911 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.002s
I20260709 12:25:58.229939  9939 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.297252  9944 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:58.269824  9932 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.353339  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:58.362493  7728 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:58.281443  9919 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.356158  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:25:58.142038  9864 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.223711  9937 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.261547  9902 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.344343  9747 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:58.375808  7736 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:25:58.366034  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:25:58.358697  9836 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.002s
I20260709 12:25:58.383330  9941 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.348961  9617 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:58.299621  9804 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.384022  9941 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.385946  9804 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:58.386409  9757 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.980s	user 0.004s	sys 0.007s
I20260709 12:25:58.377617  9797 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.343885  9726 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:25:58.387472  9935 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:25:58.387863  9726 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.388389  9726 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.389253  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2101) took 1017 ms. Trace:
I20260709 12:25:58.382998  9836 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.253098  9778 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:25:58.393438  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 121580us
I20260709 12:25:58.298955  9918 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.389669  7734 rpcz_store.cc:276] 0709 12:25:57.371909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.372163 (+   254us) service_pool.cc:224] Handling call
0709 12:25:57.372778 (+   615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.707350 (+334572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.707383 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.707925 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.807188 (+ 99263us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:57.913921 (+106733us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.914232 (+   311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.914390 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.388991 (+474601us) raft_consensus.cc:1685] finished
0709 12:25:58.389016 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.389176 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24555,"prepare.queue_time_us":262066,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":33128,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":18511,"thread_start_us":492665,"threads_started":3,"wal-append.queue_time_us":284484}
I20260709 12:25:58.383929  9927 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.401403  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:25:58.439761  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:58.440229  7736 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:58.440550  7808 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
W20260709 12:25:58.386865  9942 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.281s	user 0.000s	sys 0.000s
I20260709 12:25:58.367558  9919 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.418401  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:58.372545  9947 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:58.378285  9747 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:58.367296  9932 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:58.461169  7659 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:58.391213  9726 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.386407  9941 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.462702  7728 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:58.460536  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:25:58.437297  9831 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.299s	user 0.010s	sys 0.005s
I20260709 12:25:58.472355  9831 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.475875  8414 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:58.419973  9917 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.485896  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:25:58.387039  9757 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.361990  9911 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.347354  9926 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.351552  9929 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.384074  9617 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.489266  9941 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:58.509778  7656 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:25:58.458703  9927 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.511324  9836 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.511643  9948 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:58.470700  9919 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.502345  9832 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.376242  9944 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:58.529987  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 354015us
I20260709 12:25:58.531023  9919 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.531394  9944 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.514364  9941 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:58.618132  9937 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.627743  9846 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.628783  9846 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:58.629878  9833 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.883s	user 0.011s	sys 0.000s
I20260709 12:25:58.630381  9833 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:58.525143  9948 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:58.631384  9640 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.003s
W20260709 12:25:58.634109  9856 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.816s	user 0.016s	sys 0.005s
I20260709 12:25:58.634495  9939 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.594916  9883 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.645587  9640 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:58.621130  9828 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.200s	user 0.018s	sys 0.000s
I20260709 12:25:58.634944  9952 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:58.639550  7735 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:25:58.639590  9831 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.621241  9941 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:58.522944  9747 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.651465  9856 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.652427  9957 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:58.652694  9640 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.493515  9917 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.623646  9789 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.636039  9936 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.702170  9617 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.647156  9828 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.640266  9825 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.641673  9948 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:58.682068  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1576) took 2505 ms. Trace:
I20260709 12:25:58.711342  7661 rpcz_store.cc:276] 0709 12:25:56.172230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.172375 (+   145us) service_pool.cc:224] Handling call
0709 12:25:56.936539 (+764164us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:25:56.936911 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.035784 (+ 98873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.035813 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.035854 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:57.035883 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.677215 (+1641332us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54004362d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:58.677283 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.677450 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251015296}
W20260709 12:25:58.640470  9734 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.001s
I20260709 12:25:58.685061  9831 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:58.695112  9877 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.342s	user 0.009s	sys 0.010s
I20260709 12:25:58.685830  9911 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.683197  9856 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.686651  9957 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:25:58.705755  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.715516  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.029s	sys 0.052s
W20260709 12:25:58.719144  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.030s	sys 0.052s
W20260709 12:25:58.720005  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 904279us
I20260709 12:25:58.721423  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:25:58.722528  9757 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.723832  7654 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:58.694420  9902 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.717001  7653 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:58.738173  7777 connection.cc:256] RPC call timeout handler was delayed by -1.1122730570000385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 12:25:58.718266  9831 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.721619  9877 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:58.680737  9952 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.726644  9939 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:58.783828  9953 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.764817  9786 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.783818  9902 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.785205  7700 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 691f0d885861432f91375d5f288af312. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:58.785072  9805 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.101s	user 0.009s	sys 0.005s
I20260709 12:25:58.784761  9786 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:58.792506  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 2.55619e+06us
I20260709 12:25:58.792639  9805 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.793975  7803 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:58.794382  9786 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.795331  9902 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.796813  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 526) took 1689 ms. Trace:
I20260709 12:25:58.796989  9805 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:58.768281  9831 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:58.797278  8411 rpcz_store.cc:276] 0709 12:25:57.107709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.107982 (+   273us) service_pool.cc:224] Handling call
0709 12:25:58.796519 (+1688537us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54009f0028. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006ec6ebc 00007fc006efc08c 00007fc006ee693b 00007fc006ee63d2 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c
0709 12:25:58.796740 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478816000}
I20260709 12:25:58.802467  7660 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:25:58.803541  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2002) took 1924 ms. Trace:
I20260709 12:25:58.765977  9944 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.804000  7660 rpcz_store.cc:276] 0709 12:25:56.879688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.317041 (+437353us) service_pool.cc:224] Handling call
0709 12:25:57.317439 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.802152 (+1484713us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b540019dfa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f241c0 00007fc001f25231 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c
0709 12:25:58.803143 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.803462 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120844672}
I20260709 12:25:58.813870  9926 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.786235  9939 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.820551  9952 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:58.767436  9828 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:58.790583  9861 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:58.811991  9789 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:58.824029  9939 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:58.850777  9949 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:58.851143  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 439) took 2626 ms. Trace:
I20260709 12:25:58.851552  7655 rpcz_store.cc:276] 0709 12:25:56.224749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.234220 (+  9471us) service_pool.cc:224] Handling call
0709 12:25:58.850160 (+2615940us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:25:58.850463 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.850699 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.850734 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.850789 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.850832 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.850902 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.851064 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17713920}
I20260709 12:25:58.826221  9861 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:58.857734  9861 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.851617  9949 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.816632  9944 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.859094  9861 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:58.865584  9899 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.634s	user 0.017s	sys 0.001s
I20260709 12:25:58.784607  9953 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:58.888814  9856 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:58.904130  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 438) took 2673 ms (client timeout 2275 ms). Trace:
W20260709 12:25:58.904544  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2112) took 1519 ms (client timeout 1905 ms). Trace:
W20260709 12:25:58.905282  7730 rpcz_store.cc:269] 0709 12:25:57.377457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.418186 (+ 40729us) service_pool.cc:224] Handling call
0709 12:25:58.896494 (+1478308us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002d7dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:58.896652 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651135872}
W20260709 12:25:58.910471  9959 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:25:58.854360  9789 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:58.904812  7659 rpcz_store.cc:269] 0709 12:25:56.221697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.224918 (+  3221us) service_pool.cc:224] Handling call
0709 12:25:58.139090 (+1914172us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:58.895066 (+755976us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:58.895303 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554643968}
I20260709 12:25:58.911588  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:58.939112  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.015s	sys 0.009s
W20260709 12:25:58.943045  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.015s	sys 0.010s
W20260709 12:25:58.939742  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 408854us
I20260709 12:25:58.890455  9899 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:58.842357  9957 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:25:59.001893  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.036s
W20260709 12:25:59.004684  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.000s	sys 0.037s
W20260709 12:25:59.005160  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 544711us
W20260709 12:25:59.010311  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 344901us
I20260709 12:25:58.997706  7731 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:25:59.013998  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.262416  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1819) took 1880 ms. Trace:
I20260709 12:25:59.257222  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1820) took 1874 ms. Trace:
I20260709 12:25:59.355950  7812 rpcz_store.cc:276] 0709 12:25:57.382445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.382965 (+   520us) service_pool.cc:224] Handling call
0709 12:25:57.396674 (+ 13709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.446441 (+ 49767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.262022 (+1815581us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:59.262322 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79629696}
W20260709 12:25:59.264242  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1574) took 3092 ms (client timeout 1600 ms). Trace:
W20260709 12:25:59.357395  7656 rpcz_store.cc:269] 0709 12:25:56.171547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.173341 (+  1794us) service_pool.cc:224] Handling call
0709 12:25:57.447491 (+1274150us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:59.263921 (+1816430us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54003980d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:59.264128 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950627072}
W20260709 12:25:59.039367  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.006s	sys 0.043s
W20260709 12:25:59.358553  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.016s	sys 0.053s
I20260709 12:25:59.257884  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:25:59.359340  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 941001us
I20260709 12:25:59.360309  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:25:59.260913  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:59.477082  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.000s	sys 0.028s
I20260709 12:25:59.102190  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.356218  7803 rpcz_store.cc:276] 0709 12:25:57.382809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.383759 (+   950us) service_pool.cc:224] Handling call
0709 12:25:57.384092 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.794399 (+1410307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.256729 (+462330us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400296654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:59.257054 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3241962880}
W20260709 12:25:59.279014  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.478492  7731 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:59.462394  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.004s	sys 0.028s
W20260709 12:25:59.480026  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.363171  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:25:59.377454  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:25:59.481863  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2243) took 2447 ms (client timeout 1905 ms). Trace:
W20260709 12:25:59.462538  7805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.011s	sys 0.016s
I20260709 12:25:59.368613  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1584) took 2830 ms. Trace:
W20260709 12:25:59.478704  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.000s	sys 0.029s
W20260709 12:25:59.009889  9778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.011s
I20260709 12:25:59.477465  9918 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:25:59.484539  9778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.028s	sys 0.025s
W20260709 12:25:59.120695  9904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.009s	sys 0.016s
W20260709 12:25:59.482510  7807 rpcz_store.cc:269] 0709 12:25:57.034641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.035790 (+  1149us) service_pool.cc:224] Handling call
0709 12:25:59.257658 (+2221868us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:59.481756 (+224098us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12221184}
W20260709 12:25:59.150990  9885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.019s	sys 0.034s
I20260709 12:25:59.501806  7775 leader_election.cc:400] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
W20260709 12:25:59.462191  9810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.045s	user 0.019s	sys 0.033s
I20260709 12:25:59.504688  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:59.354957  9846 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.725s	user 0.020s	sys 0.017s
W20260709 12:25:59.369239  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 518) took 2993 ms (client timeout 2144 ms). Trace:
I20260709 12:25:59.267207  9933 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.210819  9924 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.551s	user 0.006s	sys 0.002s
I20260709 12:25:59.357777  7655 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:25:59.480094  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.005s	sys 0.029s
W20260709 12:25:59.485028  9928 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.447s	user 0.010s	sys 0.010s
W20260709 12:25:59.522891  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 574753us
W20260709 12:25:59.516237  9864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.015s	sys 0.017s
W20260709 12:25:59.483157  7805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.012s	sys 0.016s
I20260709 12:25:59.014573  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.569752  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.570042  7660 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:59.571403  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:59.499832  9885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.249s	user 0.019s	sys 0.042s
I20260709 12:25:59.491829  9927 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.217948  9848 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:59.495837  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.101257  9884 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.497217  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1589) took 2535 ms (client timeout 3260 ms). Trace:
W20260709 12:25:59.218014  9941 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
W20260709 12:25:59.248657  9932 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.793s	user 0.005s	sys 0.011s
I20260709 12:25:59.374786  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.376129  9747 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:59.488905  9904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.017s	sys 0.023s
I20260709 12:25:59.379019  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.184195  9789 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:59.521816  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.378078  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1593) took 1692 ms. Trace:
I20260709 12:25:59.625895  9856 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.625706  7653 rpcz_store.cc:276] 0709 12:25:57.685648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.071092 (+385444us) service_pool.cc:224] Handling call
0709 12:25:59.276806 (+1205714us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b540028c654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:59.377974 (+101168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978382592}
I20260709 12:25:59.626912  9829 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.626644  9856 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.376020  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.627492  9829 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.480538  9828 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:59.628901  9828 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.485705  7657 rpcz_store.cc:276] 0709 12:25:56.538475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.154998 (+616523us) service_pool.cc:224] Handling call
0709 12:25:57.155423 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.730315 (+574892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.215651 (+1485336us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540002fb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:59.368508 (+152857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226964864}
I20260709 12:25:59.165683  9944 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.059468  9949 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.629285  9856 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.684828  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.683904  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.690515  8412 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:59.691610  9827 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.692946  9885 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:59.693403  9885 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:25:59.693825  9885 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.696694  9899 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.697683  9829 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.698349  9829 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:25:59.698927  9829 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.701211  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:25:59.706753  9948 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.990s	user 0.014s	sys 0.000s
W20260709 12:25:59.511315  9810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.105s	user 0.021s	sys 0.033s
I20260709 12:25:59.376293  9917 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.701308  9829 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.213603  9894 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:59.572299  7660 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
W20260709 12:25:59.599710  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73680.4us
W20260709 12:25:59.600040  7654 rpcz_store.cc:269] 0709 12:25:56.961474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.051954 (+ 90480us) service_pool.cc:224] Handling call
0709 12:25:59.496946 (+2444992us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540033dbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:59.497127 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3841757056}
I20260709 12:25:59.601500  7661 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:25:59.372949  9733 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:25:59.569242  9864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.015s	sys 0.017s
I20260709 12:25:59.720916  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:59.721832  8406 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.724321  9967 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:59.513629  8414 rpcz_store.cc:269] 0709 12:25:56.375880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.376104 (+   224us) service_pool.cc:224] Handling call
0709 12:25:57.031556 (+655452us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:59.270233 (+2238677us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54004691d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:59.369083 (+ 98850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3326706304}
I20260709 12:25:59.606220  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:59.729547  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:25:59.607308  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:59.603212  9848 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:59.731374  7809 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:59.515154  9911 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.746598  9767 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:59.273958  9939 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.523275  9933 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:59.521883  9846 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.621307  7812 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:25:59.621618  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 371) took 1171 ms. Trace:
W20260709 12:25:59.622005  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99353.1us
W20260709 12:25:59.602975  9704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.033s	sys 0.062s
I20260709 12:25:59.622890  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 403) took 1242 ms. Trace:
I20260709 12:25:59.623322  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:25:59.624181  9826 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:59.660952  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.219106  9831 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.596786  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:25:59.682094  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:59.765292  8409 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:59.376332  9953 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.612967  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:25:59.708161  9905 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.360s	user 0.010s	sys 0.007s
I20260709 12:25:59.772356  7656 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:25:59.629380  9828 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:59.747152  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:59.604247  9902 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.757943  7734 rpcz_store.cc:276] 0709 12:25:58.380823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.427721 (+ 46898us) service_pool.cc:224] Handling call
0709 12:25:58.428250 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.522450 (+ 94200us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:58.594256 (+ 71806us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.985186 (+390930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.985257 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.985318 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.622574 (+637256us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:59.622647 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.622819 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72141,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":651,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":2730,"spinlock_wait_cycles":216008960,"thread_start_us":389789,"threads_started":1}
W20260709 12:25:59.657150  9924 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:25:59.779906  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:59.766032  7655 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:59.757507  7810 rpcz_store.cc:276] 0709 12:25:58.450151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.450789 (+   638us) service_pool.cc:224] Handling call
0709 12:25:58.462555 (+ 11766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.543665 (+ 81110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.543728 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.543794 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.543843 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.621328 (+1077485us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b54003e9ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:25:59.621383 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.621535 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185679232}
I20260709 12:25:59.633359  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:25:59.707137  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:25:59.770818  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:25:59.789062  8411 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.790879  7811 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:25:59.774338  7700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.024s	sys 0.089s
W20260709 12:25:59.792488  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.792794  7700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.024s	sys 0.089s
I20260709 12:25:59.774730  7657 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:59.793237  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.734866  7731 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 12:25:59.673506  9789 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.794018  7731 pending_rounds.cc:85] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 4
W20260709 12:25:59.758934  9704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.033s	sys 0.062s
I20260709 12:25:59.611304  9932 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.788170  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:25:59.609455  9941 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.795998  7731 pending_rounds.cc:107] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:25:59.808962  7731 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.576871  9928 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:25:59.761248  9650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.012s	sys 0.033s
I20260709 12:25:59.719185  9894 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:25:59.611447  9884 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.764200  9731 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.802093  7804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.008s	sys 0.016s
I20260709 12:25:59.807478  8415 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:25:59.784578  9929 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.810741  9834 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:59.852648  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2242) took 2818 ms (client timeout 1634 ms). Trace:
W20260709 12:25:59.841476  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53603us
I20260709 12:25:59.764853  9911 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.838150  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2107) took 2436 ms. Trace:
I20260709 12:25:59.783466  9905 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.801187  9789 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:25:59.716784  9948 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.804982  7653 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
W20260709 12:25:59.858989  7700 connection.cc:256] RPC call timeout handler was delayed by -1.3729647819995989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260709 12:25:59.920089  8411 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
W20260709 12:25:59.921878  7804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.008s	sys 0.017s
I20260709 12:25:59.865494  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 368) took 1419 ms. Trace:
W20260709 12:25:59.911170  7778 leader_election.cc:336] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.527s (SENT)
W20260709 12:25:59.869103  9962 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:59.811015  9953 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:59.920156  7731 rpcz_store.cc:276] 0709 12:25:57.375844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.418460 (+ 42616us) service_pool.cc:224] Handling call
0709 12:25:57.423541 (+  5081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.995545 (+1572004us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400330ca0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001ea006f 00007fc001f18f60 00007fc001f1a7de 00007fc001f24b49 00007fc001f25416 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62
0709 12:25:59.811100 (+815555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.811730 (+   630us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244543872}
I20260709 12:26:00.012615  7806 rpcz_store.cc:276] 0709 12:25:58.445844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.446075 (+   231us) service_pool.cc:224] Handling call
0709 12:25:58.446591 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.447068 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.447109 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.800345 (+353236us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:25:58.800875 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.483741 (+682866us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:59.483792 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.484021 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.865120 (+381099us) raft_consensus.cc:1685] finished
0709 12:25:59.865148 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.865416 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51153,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":157,"spinlock_wait_cycles":30657920,"thread_start_us":682413,"threads_started":1,"wal-append.queue_time_us":1036795}
W20260709 12:25:59.988112  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68488.6us
I20260709 12:25:59.869478  9803 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:59.979318  7700 connection.cc:256] RPC call timeout handler was delayed by -1.2505964759998278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
I20260709 12:26:00.021206  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.022130  7734 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:00.022293  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:00.028260  9971 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:26:00.030265  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.031005  9948 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:00.032735  9939 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:00.034168  7737 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:59.865206  9928 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:25:59.869325  9846 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.987018  9932 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:25:59.921993  9958 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.908414  9960 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:25:59.858839  9977 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:00.040374  9977 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:00.040498  9831 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:00.040798  9861 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.155s	user 0.011s	sys 0.003s
I20260709 12:25:59.865556  9810 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:00.042567  9810 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.044404  9905 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.045296  7730 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:26:00.045697  7730 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 1
I20260709 12:26:00.045940  7730 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:26:00.046187  7730 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.046998  9810 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:25:59.891917  9650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.015s	sys 0.036s
W20260709 12:26:00.011346  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100230us
I20260709 12:25:59.810420  9961 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:59.918102  7811 rpcz_store.cc:269] 0709 12:25:57.034319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.035561 (+  1242us) service_pool.cc:224] Handling call
0709 12:25:58.911361 (+1875800us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:25:59.852242 (+940881us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54000686d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:25:59.852509 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001702016}
I20260709 12:26:00.081135  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.081777  7732 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:00.058408  7778 leader_election.cc:336] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.673s (SENT)
I20260709 12:26:00.082533  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:00.062790  7730 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:26:00.072019  9969 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:26:00.082935  7655 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:25:59.873252  9778 ts_tablet_manager.cc:1434] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 2.863s	user 0.060s	sys 0.064s
I20260709 12:26:00.091710  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 364) took 1668 ms. Trace:
I20260709 12:26:00.092453  7805 rpcz_store.cc:276] 0709 12:25:58.423083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.423369 (+   286us) service_pool.cc:224] Handling call
0709 12:25:58.433497 (+ 10128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.433901 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.433941 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.802516 (+368575us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:25:58.803004 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.569926 (+766922us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:59.569971 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.570039 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.075418 (+505379us) raft_consensus.cc:1685] finished
0709 12:26:00.075450 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.091600 (+ 16150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36194,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":10320,"spinlock_wait_cycles":60193536,"thread_start_us":765899,"threads_started":1,"wal-append.queue_time_us":1212583}
I20260709 12:26:00.089588  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1605) took 1033 ms. Trace:
I20260709 12:26:00.096741  7655 rpcz_store.cc:276] 0709 12:25:59.055667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.596306 (+540639us) service_pool.cc:224] Handling call
0709 12:26:00.089475 (+493169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.098352  7730 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:00.100338  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:00.101285  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.037613  9958 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:00.131531  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73149us
W20260709 12:26:00.076952  9964 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:00.136106  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:00.137316  8411 raft_consensus.cc:2368] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
I20260709 12:26:00.042826  9911 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.161322  9958 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.143673  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.162420  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1592) took 2483 ms. Trace:
I20260709 12:26:00.163425  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.164183  7728 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:00.164683  9789 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:25:59.855116  9848 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.151015  9941 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.165729  9848 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:00.162835  7662 rpcz_store.cc:276] 0709 12:25:57.685306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.484432 (+799126us) service_pool.cc:224] Handling call
0709 12:25:58.501981 (+ 17549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.504295 (+  2314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.504351 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.510195 (+  5844us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.931749 (+421554us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:58.931912 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.931995 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.066389 (+1134394us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:00.162004 (+ 95615us) raft_consensus.cc:1685] finished
0709 12:26:00.162036 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.162328 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1746,"raft.queue_time_us":5199,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":11510,"spinlock_wait_cycles":32140160,"thread_start_us":329204,"threads_started":1,"wal-append.queue_time_us":1281080}
I20260709 12:26:00.152640  9884 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.153445  7775 leader_election.cc:400] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
I20260709 12:26:00.168232  7775 leader_election.cc:403] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:26:00.169046  9928 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:00.039592  9894 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.139494  9673 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:00.172215  9894 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:26:00.173856  9747 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.001s
I20260709 12:26:00.138671  9931 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:00.039608  9960 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:25:59.991026  9731 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.078630  9961 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:00.057327  9953 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.139640  9932 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:00.142179  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.092451  9778 ts_tablet_manager.cc:1467] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Tablet startup took 3087ms
I20260709 12:26:00.041627  9977 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.180056  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.199429  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
W20260709 12:26:00.182204  7702 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.013s	sys 0.029s
I20260709 12:26:00.184306  7732 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:00.089596  9803 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:00.189361  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.147s	user 0.017s	sys 0.023s
W20260709 12:26:00.184161  9957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.014s	sys 0.033s
I20260709 12:26:00.207756  7736 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:00.197808  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.016s	sys 0.009s
W20260709 12:26:00.202225  7702 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.013s	sys 0.029s
W20260709 12:26:00.188462  9876 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.016s	sys 0.040s
W20260709 12:26:00.205251  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.018s	sys 0.023s
I20260709 12:26:00.191872  9961 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:26:00.212267  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 933939us
W20260709 12:26:00.215377  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.026s	sys 0.066s
I20260709 12:26:00.187079  9933 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:00.212167  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.017s	sys 0.009s
W20260709 12:26:00.239471  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.206883  9957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.015s	sys 0.033s
W20260709 12:26:00.232158  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.017s	sys 0.021s
I20260709 12:26:00.197573  9939 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:00.235343  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.19287e+06us
I20260709 12:26:00.232985  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:00.202970  9916 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.021s	sys 0.030s
I20260709 12:26:00.216001  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.255728  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:00.237514  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.026s	sys 0.066s
I20260709 12:26:00.256697  7808 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
W20260709 12:26:00.240741  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.017s	sys 0.021s
W20260709 12:26:00.214881  9934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.019s	sys 0.042s
I20260709 12:26:00.242427  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.259949  7728 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:00.243240  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:00.262146  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:00.262684  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:00.263456  7658 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:00.243883  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.264430  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.239718  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 865186us
I20260709 12:26:00.249959  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.249819  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:00.266094  7811 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:26:00.256009  7736 raft_consensus.cc:2376] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3. Re-sending same reply.
W20260709 12:26:00.269124  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.155s	user 0.013s	sys 0.038s
W20260709 12:26:00.269618  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.013s	sys 0.038s
I20260709 12:26:00.265239  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:00.273653  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.274549  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:26:00.238425  9876 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.017s	sys 0.040s
I20260709 12:26:00.203632  9932 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:00.280292  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.244232  9972 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.259552  9934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.020s	sys 0.043s
I20260709 12:26:00.298382  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.305244  7732 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:00.241164  9747 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.296135  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:00.314230  7805 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:00.300902  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.254106  9673 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:00.315435  7803 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
W20260709 12:26:00.316419  7807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.016s
I20260709 12:26:00.299571  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.329541  7736 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:00.330924  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.331956  7736 raft_consensus.cc:2368] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
W20260709 12:26:00.334328  7807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.000s	sys 0.017s
I20260709 12:26:00.338860  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.270605  9948 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:00.257733  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 894575us
I20260709 12:26:00.307706  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.310426  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:00.312207  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:00.318646  7777 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.013s	sys 0.005s
I20260709 12:26:00.317260  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.301928  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.234234  9828 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.317698  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:00.271674  9905 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:00.324445  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.259289  9941 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:00.327399  7733 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
W20260709 12:26:00.328281  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.021s	sys 0.042s
I20260709 12:26:00.482527  7733 pending_rounds.cc:85] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 2
W20260709 12:26:00.482640  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.021s	sys 0.042s
I20260709 12:26:00.482944  7733 pending_rounds.cc:107] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.3
W20260709 12:26:00.328436  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.295977  9846 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:00.257225  9916 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.022s	sys 0.030s
W20260709 12:26:00.306609  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48169.6us
I20260709 12:26:00.420542  7660 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:26:00.473013  7732 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:00.474151  7810 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:00.476576  9932 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:00.478400  9933 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:00.479475  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:00.311458  8412 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:00.483446  7624 connection.cc:256] RPC call timeout handler was delayed by -1.3152772299999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 12:26:00.483899  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.540277  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:00.541046  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.541240  7732 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:00.542436  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.484185  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.473284  7661 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:00.546664  7732 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:00.548547  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.550453  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.552273  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.486971  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:00.487808  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:00.488622  7777 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.016s	sys 0.012s
I20260709 12:26:00.483271  7733 raft_consensus.cc:2917] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Aborting config change with OpId 2.3: Aborted: Op aborted by new leader
I20260709 12:26:00.558411  7733 raft_consensus.cc:2889] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.500917  7656 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:00.506562  9905 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.564494  9827 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.565398  9827 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.566792  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:00.515975  9846 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:00.568955  9976 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.264s	user 0.000s	sys 0.000s
I20260709 12:26:00.568629  9846 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.569612  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.570574  7803 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:00.571344  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 365) took 2147 ms. Trace:
I20260709 12:26:00.571987  7804 rpcz_store.cc:276] 0709 12:25:58.423501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.423675 (+   174us) service_pool.cc:224] Handling call
0709 12:25:58.424185 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.424714 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.424758 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.858432 (+433674us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b18002cc8c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:25:58.859203 (+   771us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:25:58.859927 (+   724us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.011767 (+1151840us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:00.268271 (+256504us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.268627 (+   356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.268745 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.571003 (+302258us) raft_consensus.cc:1685] finished
0709 12:26:00.571026 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.571202 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":588,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":53378,"spinlock_wait_cycles":69819648,"thread_start_us":1360058,"threads_started":2,"wal-append.queue_time_us":1355331}
I20260709 12:26:00.574724  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.576092  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.635418  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.502327  7658 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:00.645332  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.640635  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.502944  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.317615  9673 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:00.343902  9885 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.002s
I20260709 12:26:00.505252  7811 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:00.510783  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:00.509784  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:00.647647  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136948us
I20260709 12:26:00.648973  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.650602  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.318473  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.516119  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.651614  7660 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:26:00.543960  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:00.546638  7657 raft_consensus.cc:2368] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:00.654218  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:00.635483  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.645450  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.646358  7655 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:00.253679  9939 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:00.669843  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.670567  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.656342  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:00.511996  9958 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.658028  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.668190  7660 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:00.311725  9972 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.669452  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:00.673521  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53562 (request call id 8) took 5161 ms. Trace:
I20260709 12:26:00.669037  9673 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:00.302479  9803 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.682106  9902 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:00.715306  9902 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:00.546751  9911 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.191s
I20260709 12:26:00.715938  9902 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:00.716930  9911 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:00.717489  9911 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:00.719131  9911 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.710933  9958 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.760769  7655 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:00.722697  9986 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:00.682410  9810 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.725534  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1826) took 1121 ms. Trace:
I20260709 12:26:00.774327  7807 rpcz_store.cc:276] 0709 12:25:59.604093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.605304 (+  1211us) service_pool.cc:224] Handling call
0709 12:25:59.605667 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.630945 (+ 25278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.630980 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.683189 (+ 52209us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.335279 (+652090us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:00.335344 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.335452 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.725261 (+389809us) raft_consensus.cc:1685] finished
0709 12:26:00.725293 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.725454 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24856,"raft.queue_time_us":49151,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":30072,"thread_start_us":633132,"threads_started":1,"wal-append.queue_time_us":994734}
I20260709 12:26:00.726060  7736 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:00.706882  7733 rpcz_store.cc:276] 0709 12:25:55.421801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.421961 (+   160us) service_pool.cc:224] Handling call
0709 12:25:55.670449 (+248488us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400165e20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e9f520 00007fc001f28b42 00007fc001f27800 00007fc006ee8d29 00007fc0004671d4 00007fc000467162 00007fc000467091 00007fc00046700c 00007fc0004662c2 00007fc00025d48c
0709 12:25:55.757148 (+ 86699us) log.cc:844] Serialized 499 byte log entry
0709 12:25:56.033727 (+276579us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b18003cf220. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001e7bb0b 00007fc001f00b63 00007fc001f28752 00007fc006ee8d29
0709 12:25:57.508394 (+1474667us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400165e20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e9b062 00007fc001e7beaa 00007fc001e82d53 00007fc001e82cd9 00007fc001e82c69 00007fc001e82c31 00007fc001e81dfd 0000000000388634 0000000000388479 00007fbffee35475
0709 12:26:00.583526 (+3075132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170203,"raft.queue_time_us":180323,"raft.run_cpu_time_us":10024,"raft.run_wall_time_us":1504782,"spinlock_wait_cycles":680037632,"thread_start_us":90687,"threads_started":1,"wal-append.queue_time_us":152883,"wal-append.run_cpu_time_us":2811,"wal-append.run_wall_time_us":1046841}
I20260709 12:26:00.684921  9933 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:00.344676  9747 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:00.705739  9884 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:00.786725  7735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:26:00.787668  7657 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:00.787694  7735 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:00.789804  7655 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:00.782485  7653 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:00.799347  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.684088  9885 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.775406  9953 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:00.791976  9650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.023s	sys 0.051s
W20260709 12:26:00.840188  9650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.023s	sys 0.052s
W20260709 12:26:00.840281  7607 auto_rebalancer.cc:629] Failed to schedule move for tablet 691f0d885861432f91375d5f288af312: Aborted: Op aborted by new leader
I20260709 12:26:00.789937  7735 raft_consensus.cc:614] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:00.846364  7804 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:00.826918  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.799248  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:00.548254  8406 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.826s	user 0.015s	sys 0.000s
W20260709 12:26:00.855804  7737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.014s	sys 0.013s
W20260709 12:26:00.863554  7737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.014s	sys 0.013s
I20260709 12:26:00.657429  7732 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:00.345407  9940 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.011s
I20260709 12:26:00.501683  7662 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:00.560145  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.20539e+06us
W20260709 12:26:00.820149  7734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.017s	sys 0.010s
I20260709 12:26:00.793025  9988 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.880800  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.890920  9983 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:00.892443  7736 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:26:00.897398  7736 pending_rounds.cc:85] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:26:00.897758  7736 pending_rounds.cc:107] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:26:00.898030  7736 raft_consensus.cc:2889] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.797950  9704 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.899693  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 420306us
I20260709 12:26:00.899693  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:00.900797  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:00.776378  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:00.904239  7728 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:26:00.905066  7660 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:00.906044  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 419129us
W20260709 12:26:00.890592  7734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.018s	sys 0.010s
I20260709 12:26:00.784416  9986 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.910897  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.910991  7736 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:00.911772  7807 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:00.893761  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.878535  7656 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:26:00.913884  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.917085  7656 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:00.783004  9810 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.918663  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:00.946028  7807 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:00.823644  9987 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.786787  9933 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:00.894785  9940 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.013s	sys 0.018s
I20260709 12:26:00.854578  9731 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:00.964874  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:00.798759  9884 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.909297  9704 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:00.958899  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:00.514094  9941 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.971094  7736 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:26:00.981925  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.946s (SENT) [suppressed 36 similar messages]
I20260709 12:26:00.882757  8406 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:00.910211  9911 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:00.902534  9953 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:00.992594  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:00.511142  9948 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:00.998057  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2098) took 3633 ms. Trace:
I20260709 12:26:01.014520  7733 rpcz_store.cc:276] 0709 12:25:57.365034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.366144 (+  1110us) service_pool.cc:224] Handling call
0709 12:25:57.366624 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.793200 (+3426576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.997739 (+204539us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b5400211e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:00.997972 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6807844096}
I20260709 12:26:01.006423  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:01.006671  9731 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:01.007079  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 535) took 1306 ms. Trace:
I20260709 12:26:00.990378  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.030926  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:01.032716  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.033026  9932 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:01.033924  7653 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:01.034657  9960 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:01.035617  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.038496  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:00.902889  9988 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:01.070246  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:01.079059  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.049s	sys 0.066s
W20260709 12:26:01.084789  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.050s	sys 0.066s
I20260709 12:26:01.095395  7658 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:01.115592  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.972867  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.117612  7658 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:01.266121  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.265163  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:01.221344  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 151198us
I20260709 12:26:01.267750  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:01.256613  7661 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.676s	user 0.027s	sys 0.058s
I20260709 12:26:01.268990  7661 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
W20260709 12:26:01.269490  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 360571us
W20260709 12:26:01.223703  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 322965us
I20260709 12:26:01.119045  7655 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:26:01.234755  7810 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.725s	user 0.023s	sys 0.076s
I20260709 12:26:01.272095  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:01.222713  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:01.226948  7805 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.912s	user 0.036s	sys 0.016s
I20260709 12:26:01.273663  7805 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:01.274459  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 373) took 1026 ms. Trace:
I20260709 12:26:01.274959  7805 rpcz_store.cc:276] 0709 12:26:00.248277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.248422 (+   145us) service_pool.cc:224] Handling call
0709 12:26:01.274369 (+1025947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.271056  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:01.200161  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.281034  7656 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:01.170557  7660 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.284235  7700 leader_election.cc:400] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:01.285512  7810 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:01.241755  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:01.058518  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:01.323307  7700 leader_election.cc:403] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.323486  7807 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:01.323102  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52131.7us
I20260709 12:26:01.325554  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.326642  7810 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:01.326866  7656 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:01.287824  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.330627  7809 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:01.265843  9979 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.412s	user 0.001s	sys 0.001s
W20260709 12:26:01.333688  7657 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.008s
W20260709 12:26:01.329932  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.023s	sys 0.051s
I20260709 12:26:01.082824  9828 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:01.279093  9876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.004s	sys 0.034s
I20260709 12:26:01.360325  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.391482  7807 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:01.288084  9831 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:01.285936  9826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.018s	sys 0.022s
I20260709 12:26:01.261824  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:00.197300  9778 ts_tablet_manager.cc:1469] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Trace:
0709 12:25:50.980119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.980859 (+   740us) service_pool.cc:224] Handling call
0709 12:25:51.668810 (+687951us) ts_tablet_manager.cc:934] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
0709 12:25:53.143040 (+1474230us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:53.567770 (+424730us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:53.568009 (+   239us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:54.377286 (+809277us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:54.456559 (+ 79273us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:54.493059 (+ 36500us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:54.509946 (+ 16887us) inbound_call.cc:177] Queueing success response
0709 12:25:55.082725 (+572779us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:25:57.010541 (+1927816us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:25:57.297774 (+287233us) tablet_replica.cc:261] Starting instrumentation
0709 12:25:57.339986 (+ 42212us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":40344,"mutex_wait_us":13000,"raft.queue_time_us":257387,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":1598,"tablet-copy.queue_time_us":603076,"tablet-download-pool.queue_time_us":2537,"tablet-download-pool.run_cpu_time_us":2016,"tablet-download-pool.run_wall_time_us":41842,"thread_start_us":1641303,"threads_started":3}
I20260709 12:26:01.214316  9827 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.337934  7728 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:01.339987  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:01.254428  9950 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:26:01.381546  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.285351  9972 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.385707  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:01.288239  9916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.015s	sys 0.025s
I20260709 12:26:01.266288  9939 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:01.285701  9905 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:01.400861  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:01.021977  8406 rpcz_store.cc:276] 0709 12:25:59.700215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.720063 (+ 19848us) service_pool.cc:224] Handling call
0709 12:26:01.006963 (+1286900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.509179  7662 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:01.220589  9932 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:01.364466  9953 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:01.229661  9731 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:01.336927  9885 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.530629  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1807) took 1220 ms. Trace:
I20260709 12:26:01.562083  7728 rpcz_store.cc:276] 0709 12:26:00.310481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.701051 (+390570us) service_pool.cc:224] Handling call
0709 12:26:01.530552 (+829501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.506361  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1594) took 2494 ms. Trace:
W20260709 12:26:01.498042  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.453380  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.531864  7809 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.487637  9948 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:01.488299  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.022s	sys 0.022s
I20260709 12:26:01.532461  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1811) took 1053 ms. Trace:
I20260709 12:26:01.534037  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:01.451845  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:01.588182  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.024s	sys 0.022s
I20260709 12:26:01.589351  7810 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:01.365126  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.024s	sys 0.051s
I20260709 12:26:01.588830  7807 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:01.408125  9826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.021s	sys 0.025s
I20260709 12:26:01.525465  9810 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:01.538826  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:01.544724  9986 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.524086  7660 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:01.391175  9876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.009s	sys 0.036s
I20260709 12:26:01.363986  9987 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.180s
I20260709 12:26:01.338124  9989 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:01.549140  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.454427  7653 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:01.552680  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.510421  9928 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.341s	user 0.020s	sys 0.020s
I20260709 12:26:01.663128  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.404263  9831 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:01.555346  7806 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.254s	user 0.024s	sys 0.039s
W20260709 12:26:01.542865  9916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.020s	sys 0.051s
I20260709 12:26:01.404852  9960 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.533077  7805 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:01.499560  9933 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.563409  7659 rpcz_store.cc:276] 0709 12:25:59.011852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.012026 (+   174us) service_pool.cc:224] Handling call
0709 12:25:59.012556 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.013018 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.013051 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.013567 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.270134 (+1256567us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:00.320491 (+ 50357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.320627 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.505976 (+1185349us) raft_consensus.cc:1685] finished
0709 12:26:01.505995 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.506249 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":36909,"thread_start_us":1156450,"threads_started":1,"wal-append.queue_time_us":1614384}
I20260709 12:26:01.681779  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.682726  7659 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:01.408161  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:01.569707  9998 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:26:01.687923  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.571619  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1805) took 1261 ms. Trace:
W20260709 12:26:01.563786  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.573637  7809 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:01.690593  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.574987  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.691464  7811 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:01.604827  9989 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:01.581529  7733 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:01.288074  9789 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:26:01.548017  9941 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:01.588423  7732 rpcz_store.cc:276] 0709 12:26:00.479036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.635082 (+156046us) service_pool.cc:224] Handling call
0709 12:26:01.532388 (+897306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.591009  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 406) took 1527 ms. Trace:
I20260709 12:26:01.531082  9950 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:01.592103  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53398.4us
I20260709 12:26:01.590225  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:01.591604  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.008s	sys 0.008s
I20260709 12:26:01.597007  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2015) took 1162 ms. Trace:
W20260709 12:26:01.595674  7737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.009s	sys 0.026s
W20260709 12:26:01.585960  7734 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.011s	sys 0.039s
W20260709 12:26:01.600080  7804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.000s	sys 0.015s
I20260709 12:26:01.510772  9828 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:26:01.608513  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:01.563743  9704 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.550362  9939 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:26:01.443277  9993 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 12:26:01.666699  7806 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.366s	user 0.025s	sys 0.039s
I20260709 12:26:01.666751  7660 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
W20260709 12:26:01.660036  9940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.005s	sys 0.046s
I20260709 12:26:01.666699  7807 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:01.666818  9957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.025s	sys 0.046s
W20260709 12:26:01.591132  9981 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.011s	sys 0.004s
I20260709 12:26:01.683843  7812 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:01.594710  9834 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:01.598490  9948 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:01.602715  9884 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:01.570315  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 410) took 1490 ms. Trace:
I20260709 12:26:01.594756  9988 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.575557  7661 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:01.694156  7659 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:01.690253  7728 rpcz_store.cc:276] 0709 12:26:00.309946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.570694 (+1260748us) service_pool.cc:224] Handling call
0709 12:26:01.571040 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.571286 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.571314 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.571362 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.571394 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.571448 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.571561 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.695590  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:01.696607  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.697926  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1806) took 1387 ms. Trace:
W20260709 12:26:01.734620  9940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.005s	sys 0.047s
I20260709 12:26:01.668991  9831 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.666177  9958 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:01.670329  9928 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4.
W20260709 12:26:01.704300  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.712033  7810 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:01.718585  7656 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:01.719554  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.009s	sys 0.008s
W20260709 12:26:01.719874  7737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.010s	sys 0.026s
I20260709 12:26:01.744546  9939 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:01.714264  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.723381  7662 rpcz_store.cc:276] 0709 12:26:00.557178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.914359 (+357181us) service_pool.cc:224] Handling call
0709 12:26:01.507066 (+592707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.547922 (+ 40856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.596666 (+ 48744us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54000d07d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:01.596911 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180232576}
W20260709 12:26:01.724891  7734 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.011s	sys 0.040s
W20260709 12:26:01.719138  7804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.001s	sys 0.017s
I20260709 12:26:01.723126  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:01.779268  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:01.780089  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57054.3us
I20260709 12:26:01.780532  7732 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:01.781718  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:01.738452  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:01.741112  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 151008us
I20260709 12:26:01.743532  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.702483  9989 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.754401  7731 rpcz_store.cc:276] 0709 12:26:00.080078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.122145 (+ 42067us) service_pool.cc:224] Handling call
0709 12:26:00.123107 (+   962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.140241 (+ 17134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.140291 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.150335 (+ 10044us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.085541 (+935206us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:01.085603 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.085714 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.570054 (+484340us) raft_consensus.cc:1685] finished
0709 12:26:01.570076 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.570250 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10881,"raft.queue_time_us":27292,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":934689,"threads_started":1,"wal-append.queue_time_us":1404281}
I20260709 12:26:01.712873  9950 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:01.759552  7733 rpcz_store.cc:276] 0709 12:26:00.310220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.580851 (+1270631us) service_pool.cc:224] Handling call
0709 12:26:01.581273 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.697653 (+116380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.697863 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.699585  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.799836  7805 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:26:01.757548 10003 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:01.690215  9960 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:01.739200  9981 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.012s	sys 0.004s
I20260709 12:26:01.735030  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:01.739506  9957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.025s	sys 0.047s
W20260709 12:26:01.771466  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 442673us
I20260709 12:26:01.774463  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2010) took 1341 ms. Trace:
I20260709 12:26:01.775629  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 408) took 1697 ms. Trace:
I20260709 12:26:01.704582  7729 rpcz_store.cc:276] 0709 12:26:00.063957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.064159 (+   202us) service_pool.cc:224] Handling call
0709 12:26:00.064497 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.064719 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:00.304371 (+239652us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.590630 (+1286259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.590689 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:01.590743 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.590826 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.590953 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":971800,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":15869,"raft.queue_time_us":801,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":3697,"thread_start_us":1525111,"threads_started":2,"wal-append.queue_time_us":1261378}
W20260709 12:26:01.776094  7654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.003s	sys 0.023s
W20260709 12:26:01.830579  7654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.003s	sys 0.023s
I20260709 12:26:01.825392  7657 rpcz_store.cc:276] 0709 12:26:00.432972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.764244 (+331272us) service_pool.cc:224] Handling call
0709 12:26:00.787482 (+ 23238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.770603 (+983121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.770665 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.770756 (+    91us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:01.770809 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.770922 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.774399 (+  3477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28733,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":17479,"thread_start_us":386667,"threads_started":1}
I20260709 12:26:01.800666  9987 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:01.781782  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.841389  7732 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:01.841640  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.842368  7729 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:01.842553  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.843210  7732 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:01.843547  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:01.783744  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:01.801398  9933 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:01.847046  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:01.848110  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:01.799125  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.848959  7733 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:01.789139  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.789547  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.849753  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:01.849990  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:01.850468  7728 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:01.850710  7662 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.792042  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 407) took 1714 ms. Trace:
I20260709 12:26:01.809314  9928 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:01.851387  7734 rpcz_store.cc:276] 0709 12:26:00.077544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.077735 (+   191us) service_pool.cc:224] Handling call
0709 12:26:00.078109 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.078505 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.078533 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.079032 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.159856 (+ 80824us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b1800323fe0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:00.909298 (+749442us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:01.569153 (+659855us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:01.790608 (+221455us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.791640 (+  1032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.791755 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.791823 (+    68us) raft_consensus.cc:1685] finished
0709 12:26:01.791839 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.791981 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":607,"prepare.queue_time_us":352874,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":10284,"raft.queue_time_us":15452,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":43884,"spinlock_wait_cycles":265812992,"thread_start_us":1574867,"threads_started":2,"wal-append.queue_time_us":1158313}
I20260709 12:26:01.794546  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:01.854714  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.801951  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.855491  7807 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:01.803499  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.744438  9948 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.829414  7737 rpcz_store.cc:276] 0709 12:26:00.078227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.078402 (+   175us) service_pool.cc:224] Handling call
0709 12:26:00.144732 (+ 66330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.145022 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:00.864237 (+719215us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.775211 (+910974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.775307 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:01.775362 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.775443 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.775572 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1043786,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":15778,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10528,"thread_start_us":1588451,"threads_started":2,"wal-append.queue_time_us":878302}
I20260709 12:26:01.773578  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.824692  9747 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:01.791045  9846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.044s	sys 0.036s
I20260709 12:26:01.863638  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.839563  7658 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:01.866717  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:01.783209  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:01.844159  7729 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.818133  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.844167  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:01.785478  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 505255us
I20260709 12:26:01.870546  7732 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:01.873426  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:01.876524  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:01.877418  7732 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.857825  7808 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:01.850081  7731 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:01.859102  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:01.880180  7733 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:01.860462  7810 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:01.837242  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.887388  7805 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:01.861305  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:01.857964  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:26:01.862807  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:01.888813  7734 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:01.828243  7653 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:01.902451  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.903438  7731 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:01.903700  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109242us
I20260709 12:26:01.904676  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.906216  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.906829  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:01.908236  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.908710  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:01.911703  7656 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:01.913148  9827 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:01.913862  9827 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.914387  9827 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:01.907033  7731 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:01.914939  7733 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:01.899135  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:01.899892  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:01.919211  7728 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:01.919524  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:01.854105  9894 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.681s	user 0.017s	sys 0.000s
I20260709 12:26:01.905035  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:01.921080  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:01.937788  7733 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:26:01.937567  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.937721  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.940537  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1810) took 1629 ms. Trace:
I20260709 12:26:01.936985  7730 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:01.941129  7733 rpcz_store.cc:276] 0709 12:26:00.337216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.937047 (+1599831us) service_pool.cc:224] Handling call
0709 12:26:01.937412 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.940234 (+  2822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.940476 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.940567  7656 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:01.858067  9987 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:01.940881  7809 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:01.862264  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:01.861816  9948 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.853360  9941 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:01.897631  9846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.044s	sys 0.037s
I20260709 12:26:01.912174  9905 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:01.922595  9986 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:01.946272  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.945209  9958 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.945567  9828 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:01.939543  9972 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:01.967708  9731 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:01.968387  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 418) took 1004 ms. Trace:
I20260709 12:26:01.916610  9876 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:01.936825  9848 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.771s	user 0.012s	sys 0.006s
W20260709 12:26:01.971511  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 193943us
I20260709 12:26:01.971973  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:02.018107  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77533.3us
I20260709 12:26:02.019042  7737 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:01.916787  9933 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:01.938331  9704 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:01.990586  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:01.922245  9885 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:26:01.973863  9977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.009s	sys 0.027s
I20260709 12:26:01.862200  9928 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.001477  7732 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:02.003485  7728 rpcz_store.cc:276] 0709 12:26:00.964130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.942259 (+978129us) service_pool.cc:224] Handling call
0709 12:26:01.967793 (+ 25534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.968004 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.968030 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.968080 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:01.968124 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.968189 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.968319 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.970750  9941 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:02.009743  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 154112us
I20260709 12:26:02.033721 10011 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:02.034098  9966 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.020s	sys 0.069s
I20260709 12:26:02.034461 10011 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.035555 10004 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:02.036202 10004 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:02.036599 10004 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.036670  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 375) took 1787 ms. Trace:
I20260709 12:26:02.037140  7806 rpcz_store.cc:276] 0709 12:26:00.249088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.249210 (+   122us) service_pool.cc:224] Handling call
0709 12:26:00.249562 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.249809 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:01.727901 (+1478092us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.036222 (+308321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.036293 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:02.036350 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.036429 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.036594 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":649382,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":1734716,"threads_started":3,"wal-append.queue_time_us":289096}
I20260709 12:26:02.039532 10017 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:02.042958  7803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.011s
W20260709 12:26:02.043802  7803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.011s
I20260709 12:26:02.033102  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:02.046043  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:01.999923  9986 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:26:02.048146  9977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.009s	sys 0.029s
I20260709 12:26:02.055631 10012 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:02.017820  9953 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.065099  9960 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.067279  9928 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:02.002336  9958 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.068545  9933 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.052300  9941 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.104597  9940 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:02.105067  9940 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:01.999608  9828 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:02.075093  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:02.109141  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:02.016008  9905 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:02.026115  9885 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:02.016362  9876 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:02.094022 10022 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:26:02.120183  9905 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:02.093729  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:02.120594  9876 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.073068 10011 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:02.121390  9905 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:02.097682 10005 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
W20260709 12:26:02.125622 10008 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 12:26:02.077858  9986 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:02.125849  9885 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.154207  9932 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:02.107630  9953 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.084239  9987 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.087826  9916 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:02.183643  9916 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:02.184856  9916 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.198758  9953 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.202023  9950 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.144145  9958 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.203789  9977 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:26:02.258534  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 337250us
I20260709 12:26:02.261535  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:02.264087  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 480823us
I20260709 12:26:02.276077  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.278155  7653 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 2: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:02.280218  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:01.910717  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:02.267211  7557 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 4 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:02.264696  7559 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 4 to -1, leader changed from <none> to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194), NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) evicted, now has a pending config: VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194), VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195), NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:26:02.282549  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 371783us
I20260709 12:26:02.283383  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:02.284255  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:02.283625  7559 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 4 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.285423  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.287568  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.288062  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:02.288404  7557 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.289664  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:02.290040  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:02.286702  7559 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 3 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.274379  7555 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 282) took 9109 ms. Trace:
I20260709 12:26:02.291173  7557 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.293061  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.296566  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.296469  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.306200  7657 raft_consensus.cc:2368] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:02.292263  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.296480  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.307610  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.308166  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.309551  7659 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:26:02.307485  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.311151  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.308728  7656 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:02.312950  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.314376  7656 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:02.295507  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1613) took 1394 ms. Trace:
I20260709 12:26:02.295451  7555 rpcz_store.cc:276] 0709 12:25:53.164739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.166377 (+  1638us) service_pool.cc:224] Handling call
0709 12:25:53.399005 (+232628us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001f2af3a 00007fc007ce0e9c 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c 00007fc00025c8c6
0709 12:26:02.274032 (+8875027us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:55.933504 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:56.148786 (+215282us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:56.152188 (+  3402us) write_op.cc:435] Acquired schema lock
0709 12:25:56.155016 (+  2828us) tablet.cc:662] Decoding operations
0709 12:25:56.253679 (+ 98663us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:56.256062 (+  2383us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:56.258963 (+  2901us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:56.333329 (+ 74366us) tablet.cc:701] Row locks acquired
0709 12:25:56.337124 (+  3795us) write_op.cc:260] PREPARE: finished
0709 12:25:56.363302 (+ 26178us) write_op.cc:270] Start()
0709 12:25:56.390784 (+ 27482us) write_op.cc:276] Timestamp: P: 1783599956363233 usec, L: 0
0709 12:25:56.395080 (+  4296us) op_driver.cc:348] REPLICATION: starting
0709 12:25:56.491855 (+ 96775us) log.cc:844] Serialized 942 byte log entry
0709 12:25:57.693198 (+1201343us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:25:59.321779 (+1628581us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400235120. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e97dbe 00007fc001ea2930 00007fc001ea28d8 00007fc001ea2858 00007fc001ea2813 00007fc001ea1a9c 00007fc007d8d2f6 00007fc007d87aa8 00007fc001ede7cb 00007fc001ee12c0
0709 12:26:00.855912 (+1534133us) op_driver.cc:464] REPLICATION: finished
0709 12:26:01.898130 (+1042218us) write_op.cc:301] APPLY: starting
0709 12:26:01.917679 (+ 19549us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:01.955815 (+ 38136us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:01.960971 (+  5156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:02.083665 (+122694us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:02.107432 (+ 23767us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:26:02.137063 (+ 29631us) write_op.cc:312] APPLY: finished
0709 12:26:02.160326 (+ 23263us) log.cc:844] Serialized 29 byte log entry
0709 12:26:02.209091 (+ 48765us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:02.209746 (+   655us) write_op.cc:454] Released schema lock
0709 12:26:02.210004 (+   258us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:02.210509 (+   505us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:02.210548 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":53471488,"child_traces":[["op",{"apply.queue_time_us":951748,"apply.run_cpu_time_us":9128,"apply.run_wall_time_us":348558,"num_ops":2,"prepare.queue_time_us":118314,"prepare.run_cpu_time_us":42808,"prepare.run_wall_time_us":1934429,"raft.queue_time_us":1388397,"raft.run_cpu_time_us":24266,"raft.run_wall_time_us":1074620,"replication_time_us":4387290,"spinlock_wait_cycles":49421952,"thread_start_us":3248104,"threads_started":5,"wal-append.queue_time_us":1298266,"wal-append.run_cpu_time_us":72750,"wal-append.run_wall_time_us":3722985}]]}
I20260709 12:26:02.316920  7659 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:02.315800  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.315857  7658 rpcz_store.cc:276] 0709 12:26:00.901087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.444709 (+543622us) service_pool.cc:224] Handling call
0709 12:26:01.445166 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.865262 (+420096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.294746 (+429484us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54001a40d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:02.295203 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905861376}
I20260709 12:26:02.305162  7557 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.323266  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.324299  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:02.325613 10042 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:26:02.010927  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.371330  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.081090  7737 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:02.072705  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53349.1us
I20260709 12:26:02.330719  9826 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:02.375738  9826 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:02.376410  9826 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.377070  9731 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:02.331509  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 419) took 1310 ms. Trace:
I20260709 12:26:02.374909  7737 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:26:02.379452  7733 rpcz_store.cc:276] 0709 12:26:01.020388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.942764 (+922376us) service_pool.cc:224] Handling call
0709 12:26:01.943276 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.943537 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.943578 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.947076 (+  3498us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.072662 (+125586us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:02.072715 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.072801 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.330559 (+257758us) raft_consensus.cc:1685] finished
0709 12:26:02.330612 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.331025 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":70609,"threads_started":1,"wal-append.queue_time_us":222564}
I20260709 12:26:02.380321  9731 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:02.334056  7736 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:02.384085  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1796) took 2120 ms. Trace:
I20260709 12:26:02.384671  7736 rpcz_store.cc:276] 0709 12:26:00.263820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.995380 (+731560us) service_pool.cc:224] Handling call
0709 12:26:01.023401 (+ 28021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.382802 (+1359401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.383703 (+   901us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:02.383985 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010437376}
I20260709 12:26:02.385454  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:02.136446  9828 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:02.387439  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:02.338233  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 416) took 2161 ms. Trace:
I20260709 12:26:02.390158 10042 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:26:02.389307  7735 rpcz_store.cc:276] 0709 12:26:00.176334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.924779 (+748445us) service_pool.cc:224] Handling call
0709 12:26:00.925157 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.266636 (+341479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.266668 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.322502 (+ 55834us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.819282 (+496780us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.819331 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.819409 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.337562 (+518153us) raft_consensus.cc:1685] finished
0709 12:26:02.337607 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.337980 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42370,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":46199,"thread_start_us":410671,"threads_started":1,"wal-append.queue_time_us":676597}
I20260709 12:26:02.391283 10042 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:02.391881 10042 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:02.392331 10042 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.392544  7735 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:26:02.340274  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 231046us
I20260709 12:26:02.341820  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2254) took 1799 ms. Trace:
W20260709 12:26:02.344033  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 325350us
I20260709 12:26:02.348477  9644 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:02.351099  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:02.353269  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1783) took 2252 ms. Trace:
W20260709 12:26:02.353615  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 266731us
W20260709 12:26:02.335988  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 445089us
I20260709 12:26:02.356560  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:02.359200  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:02.361446  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:02.362460  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:02.362823  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:02.365715  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 378) took 2104 ms (client timeout 2111 ms). Trace:
I20260709 12:26:02.396332  7730 rpcz_store.cc:276] 0709 12:26:00.100449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.131670 (+ 31221us) service_pool.cc:224] Handling call
0709 12:26:00.132099 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.941784 (+1809685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.338942 (+397158us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b5400121e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:02.352972 (+ 14030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4149945728}
W20260709 12:26:02.398430  7808 rpcz_store.cc:269] 0709 12:26:00.261590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.261829 (+   239us) service_pool.cc:224] Handling call
0709 12:26:02.347108 (+2085279us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:02.365482 (+ 18374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3533496192}
I20260709 12:26:02.290796  7653 raft_consensus.cc:2376] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4. Re-sending same reply.
I20260709 12:26:02.401051  7557 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:02.423327  9673 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.617s	user 0.022s	sys 0.028s
I20260709 12:26:02.424104  9673 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.423879  7557 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:02.428778  7729 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.005s
W20260709 12:26:02.430469  9827 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.010s
I20260709 12:26:02.435531  9960 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:02.436242  9960 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:02.435851  9827 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.436782  9960 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.438019  7656 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:02.434350  9644 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:02.442202 10030 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.440816  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:02.464516 10030 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.441023  7737 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:26:02.452775  7729 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:02.211577  9989 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.468504 10031 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.464912 10036 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:02.393774  7804 rpcz_store.cc:276] 0709 12:26:00.897954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.951381 (+ 53427us) service_pool.cc:224] Handling call
0709 12:26:00.988227 (+ 36846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.779014 (+790787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.779069 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.801212 (+ 22143us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.025454 (+224242us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:02.033694 (+  8240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.033884 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.340875 (+306991us) raft_consensus.cc:1685] finished
0709 12:26:02.340932 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.341406 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6818,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":12054,"thread_start_us":973260,"threads_started":2,"wal-append.queue_time_us":366494}
I20260709 12:26:02.451020  9673 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.440222  9644 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.203047  9941 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:02.517799  9977 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:02.523494  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 64601.6us
I20260709 12:26:02.405468  9932 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.108s
W20260709 12:26:02.556644  7662 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.009s
I20260709 12:26:02.452462 10011 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:26:02.203399  9940 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:02.530294  7654 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:02.604286  7731 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:02.700145  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 247616us
I20260709 12:26:02.609675  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:02.604646  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 106459us
I20260709 12:26:03.043009  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.045316  7732 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.036125  7662 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:03.047284  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2027) took 1224 ms. Trace:
I20260709 12:26:02.597923  7655 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:02.526705  9977 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.050571  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1612) took 2163 ms. Trace:
W20260709 12:26:02.523137  9911 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.502s	user 0.001s	sys 0.015s
I20260709 12:26:03.023021  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.047780  7662 rpcz_store.cc:276] 0709 12:26:01.822928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.824976 (+  2048us) service_pool.cc:224] Handling call
0709 12:26:03.047127 (+1222151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:02.556876  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116365us
W20260709 12:26:02.800326  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.598510 10012 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:02.935478  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.051019  7655 rpcz_store.cc:276] 0709 12:26:00.887431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.579179 (+691748us) service_pool.cc:224] Handling call
0709 12:26:01.579760 (+   581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.049164 (+1469404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.050186 (+  1022us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54004371d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:03.050471 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278426368}
I20260709 12:26:02.782505  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:03.066118  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.517815  8406 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:02.696341  9831 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.921s	user 0.019s	sys 0.024s
W20260709 12:26:02.529508  9961 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 2.292s	user 0.015s	sys 0.019s
I20260709 12:26:02.557387 10036 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:02.692781  9986 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.003s
I20260709 12:26:02.523774  9928 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:02.557834  9950 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.038735  9939 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:26:03.023445  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:03.004307  9902 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 2.287s	user 0.025s	sys 0.015s
W20260709 12:26:03.102691  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 539423us
I20260709 12:26:03.033321 10030 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.610621 10024 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:26:03.113485  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55577.7us
I20260709 12:26:03.114276  9928 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:03.121322  9986 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.122076  9941 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:02.907536  9826 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.128203  7735 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:02.688692 10004 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.145219  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:02.523774 10031 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.162720  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:03.152199  7778 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.561s (SENT)
I20260709 12:26:03.116539 10012 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:03.172847  9826 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.174320  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.185083  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72033.6us
I20260709 12:26:03.186305  7803 raft_consensus.cc:2410] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:03.195758  9933 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.196489  7729 raft_consensus.cc:2376] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3. Re-sending same reply.
I20260709 12:26:03.116585  9831 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.201853  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2028) took 1339 ms. Trace:
I20260709 12:26:03.168605 10024 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:03.124979  9961 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.210872  7732 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:03.206822  7654 rpcz_store.cc:276] 0709 12:26:01.862696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.863353 (+   657us) service_pool.cc:224] Handling call
0709 12:26:03.188046 (+1324693us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b540002ec54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.201749 (+ 13703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530147584}
I20260709 12:26:03.220057  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:03.140093  9902 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.223987  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.225189  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:03.227084  9960 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.789s	user 0.016s	sys 0.001s
I20260709 12:26:03.237018  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:03.227543 10036 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.245860  7734 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:03.231611  8412 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.247462  7729 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:03.229708 10011 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.251763  7804 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:26:03.259364  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126282us
I20260709 12:26:03.260097  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2140) took 1414 ms. Trace:
I20260709 12:26:03.262140  9988 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:03.262290  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.263188  9988 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:03.263960 10045 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.002s
I20260709 12:26:03.258642  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.265693 10011 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.266371  7730 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:03.266532  9988 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.246587 10036 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.226655  9933 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.274567  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1808) took 2963 ms. Trace:
W20260709 12:26:03.283152  9948 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.247s	user 0.030s	sys 0.013s
I20260709 12:26:03.279170  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.243517  9960 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.282172  7731 rpcz_store.cc:276] 0709 12:26:00.310730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.936547 (+1625817us) service_pool.cc:224] Handling call
0709 12:26:02.594192 (+657645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.094760 (+500568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.267982 (+173222us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400282b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.274501 (+  6519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229793024}
W20260709 12:26:03.288194  7778 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.590s (SENT)
I20260709 12:26:03.273869 10036 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:03.291922  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 94077.2us
I20260709 12:26:03.288692  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.260710  7733 rpcz_store.cc:276] 0709 12:26:01.845166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.381306 (+536140us) service_pool.cc:224] Handling call
0709 12:26:02.381925 (+   619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.439479 (+ 57554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.439519 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.440335 (+   816us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.565956 (+125621us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:03.200371 (+634415us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.200718 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.200835 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.259579 (+ 58744us) raft_consensus.cc:1685] finished
0709 12:26:03.259644 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.259935 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":500013,"prepare.queue_time_us":635705,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":95,"raft.queue_time_us":67050,"raft.run_cpu_time_us":597,"raft.run_wall_time_us":22060,"thread_start_us":238706,"threads_started":2,"wal-append.queue_time_us":676666}
I20260709 12:26:03.296437  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.289746 10012 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.300846  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.302881 10011 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.300860 10024 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:03.297685 10048 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 12:26:03.305701  9827 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.307096  8411 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:03.309671  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.312983  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.315522  9876 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.194s	user 0.034s	sys 0.014s
I20260709 12:26:03.316157  9876 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.317615  8405 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.317884  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:03.318930  8408 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:03.318990  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 417) took 2738 ms. Trace:
I20260709 12:26:03.271636 10028 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:26:03.254675  9928 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.322719  9731 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:03.323382  9826 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.323668  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 718) took 1534 ms. Trace:
I20260709 12:26:03.320600  7734 rpcz_store.cc:276] 0709 12:26:00.580708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.943352 (+1362644us) service_pool.cc:224] Handling call
0709 12:26:01.943826 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.268010 (+1324184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.318648 (+ 50638us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400212d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:03.318913 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2980087552}
I20260709 12:26:03.289969 10031 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.311892  8409 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEARNER]: Advancing to term 5
I20260709 12:26:03.322000  7728 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:03.324558  9986 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.332845  9961 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:03.330790  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 94608.5us
W20260709 12:26:03.367389  9885 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.200s	user 0.009s	sys 0.027s
I20260709 12:26:03.368120  9885 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.342569  9831 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.341223  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.327906  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.334820 10055 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:26:03.344058 10051 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:26:03.345095  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49052.2us
I20260709 12:26:03.343106  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.352418  8411 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:03.287782  9948 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.356950  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.358852  9933 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.340287 10004 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.346336  9876 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:03.325958  9988 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:26:03.360164  7811 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:03.331790  9902 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.360492  9928 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.360781  7730 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:03.393397  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109414us
W20260709 12:26:03.393816  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53757.3us
I20260709 12:26:03.362954  8408 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:03.363862  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:03.375303  7661 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:03.374486  9885 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:03.382637  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 215373us
I20260709 12:26:03.347167  8406 rpcz_store.cc:276] 0709 12:26:01.788882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.806800 (+ 17918us) service_pool.cc:224] Handling call
0709 12:26:01.853807 (+ 47007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.139067 (+1285260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.323269 (+184202us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:03.323574 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352979328}
W20260709 12:26:03.388496 10042 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.014s
I20260709 12:26:03.416392  7660 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:03.414759  8379 connection.cc:256] RPC call timeout handler was delayed by -1.4413037519998397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260709 12:26:03.428977  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:03.446990  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.448738  7778 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.656s (SENT)
W20260709 12:26:03.453763  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 219372us
W20260709 12:26:03.456176  9828 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.971s	user 0.012s	sys 0.010s
I20260709 12:26:03.456805  9828 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.457981 10051 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.458525 10051 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.458911 10051 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.418639 10042 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.460685  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.461073 10031 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.462031  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.462092 10031 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:03.463341 10058 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:26:03.463443  9885 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.465188 10039 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:03.465317  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.465842  9827 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.466253 10039 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:03.466645  9827 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.467911  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.468195  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 538) took 1119 ms. Trace:
I20260709 12:26:03.468690  8405 rpcz_store.cc:276] 0709 12:26:02.349318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.351439 (+  2121us) service_pool.cc:224] Handling call
0709 12:26:02.352058 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.318066 (+966008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.467856 (+149790us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:03.468105 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219636096}
I20260709 12:26:03.469393  9939 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:03.470045  9939 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.470136  7729 raft_consensus.cc:2376] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
I20260709 12:26:03.470695  9939 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.463249  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.465041  9885 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.481148  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2139) took 1648 ms. Trace:
I20260709 12:26:03.473603  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.482681  7737 rpcz_store.cc:276] 0709 12:26:01.832608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.380495 (+547887us) service_pool.cc:224] Handling call
0709 12:26:02.381207 (+   712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.440728 (+ 59521us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:02.778241 (+337513us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.416520 (+638279us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.443893 (+ 27373us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.444197 (+   304us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:03.480339 (+ 36142us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.480704 (+   365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.480816 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.480871 (+    55us) raft_consensus.cc:1685] finished
0709 12:26:03.480889 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.481068 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58287,"prepare.queue_time_us":35932,"raft.queue_time_us":81866,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":6488,"thread_start_us":463026,"threads_started":2,"wal-append.queue_time_us":379832}
I20260709 12:26:03.437731  9989 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.490175  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:03.490397  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:03.487260  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2285020760000407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 12:26:03.495541  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2123020760000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 12:26:03.496272  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2123020760000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 12:26:03.491374 10004 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:03.497210 10004 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:03.497434  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.498173  7809 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.499701  9828 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.500658  9828 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.501389  9828 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.503806  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.504312  9948 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.505152  9948 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.505640  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.505795  9948 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.506235  9932 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:03.498379 10004 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.469806  9827 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.490847  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.509449  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:03.520092  7809 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
W20260709 12:26:03.519889  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138235us
I20260709 12:26:03.517481  8408 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:26:03.524360  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49426.1us
I20260709 12:26:03.526170  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.530205  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.531278  7658 raft_consensus.cc:2376] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4. Re-sending same reply.
I20260709 12:26:03.533011  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.533345 10059 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:03.509865  9932 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.507462  9933 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.525159  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:03.525645  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70136.6us
I20260709 12:26:03.509317  8412 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:03.549563  7811 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:26:03.550719  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51361us
W20260709 12:26:03.588817  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.590659  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 109424us
W20260709 12:26:03.591511  8379 connection.cc:256] RPC call timeout handler was delayed by -1.3576037519997044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 12:26:03.591892  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.588418  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:03.593645  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.515264  9986 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:03.543097  9932 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.597601  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:03.597970  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45363.4us
I20260709 12:26:03.542224 10042 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.601157  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:03.597002  7778 leader_election.cc:336] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.770s (SENT)
I20260709 12:26:03.605567  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.605823  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.611202  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.535830  7654 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40860756ca234260bb6147416168cd6a"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:39088
I20260709 12:26:03.609608  9885 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:26:03.625126  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.620060 10035 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:03.637658  9948 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.638366  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.638950  7654 raft_consensus.cc:606] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:26:03.639456  7654 raft_consensus.cc:614] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:03.624015 10031 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.635759  9885 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.640753  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.640969  9885 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.620517  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.642187  7654 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:03.619678 10033 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:03.620357  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:26:03.595837  9989 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.629838  7805 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:03.623623  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:03.643734 10064 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.162s	user 0.002s	sys 0.000s
I20260709 12:26:03.637740 10035 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.620625  9987 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 12:26:03.620862  9650 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:03.631670  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.628904  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.639698 10004 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:03.619696  7812 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:03.653980  7736 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:03.655439  7656 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:03.613140  8379 connection.cc:256] RPC call timeout handler was delayed by -1.09840375199974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22530000000000003s for a response
W20260709 12:26:03.666082  8379 connection.cc:256] RPC call timeout handler was delayed by -1.078603751999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2275s for a response
I20260709 12:26:03.552548  9988 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:26:03.557974 10011 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:26:03.659291  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8 on tablet 59f766c3d13d4360bcdeb1d5f0f318a8 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:03.668488  7809 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:03.690608  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.645870 10033 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.631395  9826 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:03.694103  7659 raft_consensus.cc:2368] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:03.696621  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:03.663805 10035 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.624843  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.558353 10059 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.703933  9939 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.707600  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.628131 10036 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.708972  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.717641  7736 raft_consensus.cc:2368] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:03.604754  9986 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.601604 10012 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:26:03.726745  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.671268 10039 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:03.603418  9933 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.610663  9828 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:03.719559  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 214081us
I20260709 12:26:03.740049  7653 raft_consensus.cc:2368] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:03.695611  9826 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:03.750077  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 206607us
I20260709 12:26:03.752701  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.753060  9902 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.709988  9948 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.753717  9902 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.753839  9948 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.754702  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.756274  9902 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.756682  7659 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:03.758003  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.758793 10028 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.759305  7653 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:03.759438  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.760303  7806 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:03.698949 10033 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.728665  7811 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:26:03.729485  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.698338  9977 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:26:03.756338  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.699601 10042 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.811437  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:03.815456  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.803876  7729 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:03.814203  7661 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2034) took 1556 ms (client timeout 1524 ms). Trace:
I20260709 12:26:03.816291  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:03.819758  7661 rpcz_store.cc:269] 0709 12:26:02.257352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.295736 (+ 38384us) service_pool.cc:224] Handling call
0709 12:26:03.813946 (+1518210us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400116554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.814103 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445352832}
W20260709 12:26:03.820602  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55997.6us
I20260709 12:26:03.821116  9828 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.821719  9828 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.821616  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:03.822359  9828 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.796962  7812 raft_consensus.cc:2368] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5.
I20260709 12:26:03.813369  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2035) took 1555 ms. Trace:
I20260709 12:26:03.810786 10036 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:03.784590  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:03.827281  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.825538  7660 rpcz_store.cc:276] 0709 12:26:02.257969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.295764 (+ 37795us) service_pool.cc:224] Handling call
0709 12:26:03.813069 (+1517305us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400397bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.813248 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2553002112}
I20260709 12:26:03.756747  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:03.769132  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:03.832085  8380 leader_election.cc:400] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:03.832854  8380 leader_election.cc:403] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:26:03.833078  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.833614  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0134020760001476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260709 12:26:03.834031  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0134020760001476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
I20260709 12:26:03.836031  9950 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:03.839524 10059 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.841354  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.841933  9939 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:03.844959  9985 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:03.845252  9940 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.845758  9985 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:03.846347  9985 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.847453  9961 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:03.848151  9961 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:03.847653  9826 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.849478  9961 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.850325 10051 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.852720  7809 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:03.856570 10078 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:03.800961 10074 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:03.872977  9933 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.851153  9961 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.875582  9986 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.876566  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 542) took 1518 ms. Trace:
I20260709 12:26:03.877451  8412 rpcz_store.cc:276] 0709 12:26:02.358061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.358398 (+   337us) service_pool.cc:224] Handling call
0709 12:26:03.876204 (+1517806us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.876480 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878308096}
I20260709 12:26:03.823199  9988 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:03.879868  9988 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:03.881433  9988 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.851411  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:03.884145  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68425.8us
I20260709 12:26:03.885452  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.885730  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:03.887187  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.888024  7731 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.888571 10012 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.889640  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.889305 10012 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.889189 10051 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:03.891047 10051 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:03.891294  9826 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.756372  9948 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.723404  9650 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:03.892397  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.893121  9650 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:03.894120  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.894507  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.895529 10036 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:03.896580 10051 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:03.723122  9989 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.897992  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.857048  9960 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:03.858363 10039 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.903242 10039 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.904565  9932 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.911535 10059 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:03.804296  7656 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 2.
I20260709 12:26:03.857892  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.859679  8415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 539) took 1475 ms (client timeout 1704 ms). Trace:
W20260709 12:26:03.866060  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 309061us
I20260709 12:26:03.867209  7654 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:03.924535  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 364958us
I20260709 12:26:03.922739  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.915794  8415 rpcz_store.cc:269] 0709 12:26:02.349943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.350321 (+   378us) service_pool.cc:224] Handling call
0709 12:26:03.814569 (+1464248us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54007b3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:03.825232 (+ 10663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425548032}
I20260709 12:26:03.930188  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.931516  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:03.928776  7730 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:03.936437  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.937263  7730 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:03.928762  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82765.1us
I20260709 12:26:03.941188  7729 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:03.942379  7736 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.946623  7807 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.946885  7735 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:03.948318  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.949705  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.950342  7735 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:03.950433  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 128955us
I20260709 12:26:03.946631  7728 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:03.952605  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:03.954257  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:03.959758  7778 leader_election.cc:336] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.804s (SENT)
I20260709 12:26:03.964671  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.967943  7728 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:03.968228  9828 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:03.952486 10035 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:03.969558  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82339.2us
I20260709 12:26:03.899067  9989 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:03.969548  7658 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:26:03.962510  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:03.970733  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:03.979303  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47910.7us
I20260709 12:26:03.982829  7809 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:03.984580  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.986804  7805 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:03.975917  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:03.991847  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:03.993500  7804 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.000039  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:03.897773 10051 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.000186  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:04.004711  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:04.006449  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:04.009521  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79903.2us
I20260709 12:26:04.010576  9828 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:04.012995 10035 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:04.010723  9989 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:04.013635 10077 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:04.013764 10035 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:03.984565 10036 raft_consensus.cc:2764] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:26:04.016080  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:04.012802 10011 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:04.018292  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:04.019282  7803 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:04.019239  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 184458us
I20260709 12:26:04.003738  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:04.021196  7778 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.838s (SENT)
I20260709 12:26:04.022905  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:03.904605 10033 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.025058  8408 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:26:04.026381  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.992799  9902 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:26:03.988871  9960 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:04.030675  7805 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:04.001562 10030 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:04.038273  7778 leader_election.cc:336] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.605s (SENT)
I20260709 12:26:04.039633  7626 leader_election.cc:400] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5.
I20260709 12:26:04.040123  7626 leader_election.cc:403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.044724  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:04.034677  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:04.031098  9932 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:04.050637  7809 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:04.033313  9939 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:04.051546  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:04.041548  9986 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:04.038931 10004 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:04.061309  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66359.6us
I20260709 12:26:04.063587  7731 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:04.043759  9828 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:04.051945 10081 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:04.039739  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:04.075713 10044 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:04.027648  9989 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:26:04.068843  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:04.077286  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.052088  9933 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:26:04.075765  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:04.082257  7731 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.059695 10042 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:04.054576  9960 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:04.076457 10044 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:04.094343  7808 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:04.059192  9939 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:04.065958 10030 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.070353  9986 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.117091  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:04.067922  9644 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:04.119366 10057 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.068238 10004 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:04.067812 10081 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:04.126574  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91447.2us
I20260709 12:26:04.048645  9932 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:26:04.128307  7807 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:04.143042  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.160434  7730 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:04.170061  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.099685  9827 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:04.174177  7659 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:04.087531  9902 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:04.178236  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:04.180677  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 93047.8us
I20260709 12:26:04.158627  9986 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:04.188244  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.888s (SENT)
I20260709 12:26:04.164137 10004 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.167693 10081 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.167539 10031 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:04.190712  7808 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:04.180853  7731 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:04.194985  7730 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:04.197950  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:04.198295  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:04.199929  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 171960us
I20260709 12:26:04.200650  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:04.199416  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:04.176450  9827 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:04.179361  9644 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:04.165591  9885 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.116874 10044 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:04.206063  7778 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.910s (SENT)
I20260709 12:26:04.173293 10030 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.171885  9948 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:26:04.178676  9902 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:04.206681  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:04.206992  7730 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:04.201323 10031 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:04.212975 10067 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 12:26:04.220659  7731 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:04.221681  7730 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:04.213282 10033 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:04.214545 10093 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:04.205482  9827 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:04.233490  9885 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:04.233060 10031 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.234807  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:04.239473  9928 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:04.268169 10024 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:04.268730 10024 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:04.269222  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.269793 10024 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.240320 10062 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:04.228396  9939 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:04.242753  7778 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.022s (SENT)
I20260709 12:26:04.263195  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.269387  7735 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:04.289783  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:04.290766  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.270913  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.289069  7731 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:04.227931  9960 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.295702  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:04.299074  7736 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:04.297116  7804 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:04.312134  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.313058  7729 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.315842  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:04.316672  7735 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:04.319944  7732 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:04.264868 10030 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:04.268157  9928 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:04.322459  9928 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:04.323167  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 266) took 9191 ms. Trace:
I20260709 12:26:04.318823 10039 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.310341 10081 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.331151  7728 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:04.396152  7729 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
W20260709 12:26:04.615271  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 372545us
I20260709 12:26:04.393251  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:04.616238  7729 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:26:04.392047  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.392485  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.390254  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.369164  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
W20260709 12:26:04.424233  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.452957  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.655112  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.615273  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.656875  7658 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:04.581418  7734 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.576s	user 0.013s	sys 0.025s
W20260709 12:26:04.389310  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.551365  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 343325us
W20260709 12:26:04.562309  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 397025us
I20260709 12:26:04.620757  7731 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:04.659812  8379 leader_election.cc:400] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:04.621629  7659 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:04.326249 10091 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.221s	user 0.000s	sys 0.003s
W20260709 12:26:04.511178  9940 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.007s
I20260709 12:26:04.637770  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.324107 10024 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.467192  9644 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.674847 10088 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:04.675539  7810 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:26:04.676033  7810 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 5
I20260709 12:26:04.676317  7810 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:26:04.676651  7810 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.330998 10031 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:04.536182  9731 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:04.509732  9939 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.616776  7811 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:04.653337  7654 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:04.325764 10033 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:04.653025  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.062s (SENT)
I20260709 12:26:04.427824 10057 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5.
I20260709 12:26:04.661237  7734 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:04.663818  7662 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.666520  9826 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.330667  7559 rpcz_store.cc:276] 0709 12:25:55.131676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.132468 (+   792us) service_pool.cc:224] Handling call
0709 12:26:04.323059 (+9190591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:02.599276 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:03.038095 (+438819us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:03.038165 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:26:03.038197 (+    32us) tablet.cc:662] Decoding operations
0709 12:26:03.039076 (+   879us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:03.039171 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:03.039211 (+    40us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:26:03.221196 (+181985us) tablet.cc:701] Row locks acquired
0709 12:26:03.231974 (+ 10778us) write_op.cc:260] PREPARE: finished
0709 12:26:03.247667 (+ 15693us) write_op.cc:270] Start()
0709 12:26:03.270013 (+ 22346us) write_op.cc:276] Timestamp: P: 1783599963238533 usec, L: 0
0709 12:26:03.272047 (+  2034us) op_driver.cc:348] REPLICATION: starting
0709 12:26:03.330947 (+ 58900us) log.cc:844] Serialized 1550 byte log entry
0709 12:26:03.968483 (+637536us) op_driver.cc:464] REPLICATION: finished
0709 12:26:04.170216 (+201733us) write_op.cc:301] APPLY: starting
0709 12:26:04.184095 (+ 13879us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:04.207799 (+ 23704us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:04.218488 (+ 10689us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:04.231835 (+ 13347us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:04.232289 (+   454us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:26:04.232494 (+   205us) write_op.cc:312] APPLY: finished
0709 12:26:04.232949 (+   455us) log.cc:844] Serialized 35 byte log entry
0709 12:26:04.233343 (+   394us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:04.233464 (+   121us) write_op.cc:454] Released schema lock
0709 12:26:04.233530 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:04.303412 (+ 69882us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:04.303462 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":266,"mutex_wait_us":629,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":119373,"apply.run_cpu_time_us":5003,"apply.run_wall_time_us":146760,"num_ops":3,"prepare.queue_time_us":287936,"prepare.run_cpu_time_us":44353,"prepare.run_wall_time_us":1003845,"raft.queue_time_us":98266,"raft.run_cpu_time_us":15614,"raft.run_wall_time_us":39985,"replication_time_us":696202,"thread_start_us":485929,"threads_started":4,"wal-append.queue_time_us":387684}]]}
I20260709 12:26:04.685952  7661 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:04.391144  9827 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.427680  9902 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.736311  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.742199  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.830610  7658 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.323781  9928 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:04.749234  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.675599 10088 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:04.657768 10044 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.749444  9985 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.903s	user 0.012s	sys 0.005s
I20260709 12:26:04.837467  8379 leader_election.cc:400] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3.
I20260709 12:26:04.829519  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.850394  7654 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:04.853467  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 112623us
W20260709 12:26:04.854497  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 201723us
W20260709 12:26:04.865077  7778 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.084s (SENT)
W20260709 12:26:04.862438  9986 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.659s	user 0.015s	sys 0.000s
I20260709 12:26:04.867121  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.866915 10044 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.867774  9986 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.868041  7661 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:04.867957  7778 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.091s (SENT)
I20260709 12:26:04.870018 10044 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:04.870592  9986 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:04.872157  9885 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:04.872875  9885 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.849092  9826 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.877329  9885 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:04.879057 10051 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:04.880256 10051 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:04.877324  7778 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.174s (SENT)
W20260709 12:26:04.885685  7778 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.206s (SENT)
I20260709 12:26:04.862387  9985 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:04.902486  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73154.5us
I20260709 12:26:04.902901 10051 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:04.904592 10100 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:04.905812  8408 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:04.907073 10088 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:04.908692 10088 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:04.910127 10004 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.000s
I20260709 12:26:04.908726  7661 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:04.912966 10012 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:04.904713  9985 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:04.927944  7657 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:04.931384  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.937649  7737 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:04.939605  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:04.956215  7657 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:04.878304  9826 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:04.962847  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 388) took 1590 ms. Trace:
I20260709 12:26:04.969506  7662 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:04.964177  7810 rpcz_store.cc:276] 0709 12:26:03.372457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.376749 (+  4292us) service_pool.cc:224] Handling call
0709 12:26:03.377243 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.746264 (+1369021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.962546 (+216282us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:04.962772 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2984427776}
I20260709 12:26:04.945878  9902 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:05.020196 10102 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:26:05.028471  7660 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
W20260709 12:26:05.032578  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.971592 10101 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.974483  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88825.6us
W20260709 12:26:04.978761  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.980312  7656 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:26:04.981571  7543 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.019s	sys 0.023s
I20260709 12:26:04.956348  9928 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.910642 10004 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:04.983677 10081 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:04.986891  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:04.987746 10035 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:26:05.075620  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88904.9us
I20260709 12:26:04.996106 10039 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:04.997077  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1836) took 1135 ms. Trace:
I20260709 12:26:05.012789  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:05.079929  7655 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:05.016239  9827 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:05.018592 10099 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:26:05.042807  7654 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:05.044858  7661 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:26:05.085024  7732 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:05.047160  7659 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:05.087083  7557 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 178) took 8191 ms. Trace:
I20260709 12:26:05.068019  7658 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:05.088629  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:04.987360  7662 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:05.089514  7656 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:05.047425  7543 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.020s	sys 0.023s
I20260709 12:26:05.078955  7730 rpcz_store.cc:276] 0709 12:26:03.861497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.259874 (+398377us) service_pool.cc:224] Handling call
0709 12:26:04.260313 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.299060 (+ 38747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.299116 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.307786 (+  8670us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.738426 (+430640us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:04.738485 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.738582 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.996793 (+258211us) raft_consensus.cc:1685] finished
0709 12:26:04.996817 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.997007 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11230,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":1361,"thread_start_us":430131,"threads_started":1,"wal-append.queue_time_us":651507}
I20260709 12:26:05.081944  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:05.082993  7778 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.621s (SENT)
I20260709 12:26:05.074232 10035 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:05.086205 10088 raft_consensus.cc:2764] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:05.088215  7557 rpcz_store.cc:276] 0709 12:25:56.895676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.895894 (+   218us) service_pool.cc:224] Handling call
0709 12:25:58.897284 (+2001390us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001f2af3a 00007fc007ce0e9c 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c 00007fc00025c8c6
0709 12:26:05.086997 (+6189713us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:03.693733 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:03.706964 (+ 13231us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:03.707013 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:26:03.707042 (+    29us) tablet.cc:662] Decoding operations
0709 12:26:03.707928 (+   886us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:03.707986 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:03.708020 (+    34us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:26:03.708581 (+   561us) tablet.cc:701] Row locks acquired
0709 12:26:03.708600 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:26:03.708767 (+   167us) write_op.cc:270] Start()
0709 12:26:03.708889 (+   122us) write_op.cc:276] Timestamp: P: 1783599963708709 usec, L: 0
0709 12:26:03.708915 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:26:03.743601 (+ 34686us) log.cc:844] Serialized 2434 byte log entry
0709 12:26:04.324977 (+581376us) op_driver.cc:464] REPLICATION: finished
0709 12:26:04.325588 (+   611us) write_op.cc:301] APPLY: starting
0709 12:26:04.325678 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:04.325875 (+   197us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:04.325983 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:04.388372 (+ 62389us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:04.388735 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:26:04.388816 (+    81us) write_op.cc:312] APPLY: finished
0709 12:26:04.389268 (+   452us) log.cc:844] Serialized 53 byte log entry
0709 12:26:04.389546 (+   278us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:04.389749 (+   203us) write_op.cc:454] Released schema lock
0709 12:26:04.389817 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:04.390345 (+   528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:04.390397 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":2156027264,"child_traces":[["op",{"apply.queue_time_us":331,"apply.run_cpu_time_us":3658,"apply.run_wall_time_us":64670,"num_ops":6,"prepare.queue_time_us":1221425,"prepare.run_cpu_time_us":5007,"prepare.run_wall_time_us":105728,"replication_time_us":615901}]]}
W20260709 12:26:05.104080  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.112574  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.113389  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.099762  7558 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.046487 10101 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.118799  7558 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.125108  7558 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.126987  7658 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:05.127100  7558 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:05.122857  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.129637  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:05.145917  7558 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.158080  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:05.167068  7733 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:26:05.167716  7733 pending_rounds.cc:85] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:26:05.168078  7733 pending_rounds.cc:107] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:26:05.168402  7733 raft_consensus.cc:2889] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.172323  7658 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:05.171017  7558 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.191347  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:05.193672  7658 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:05.174175 10110 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:05.197577  7558 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.203847  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2149) took 1342 ms. Trace:
I20260709 12:26:05.209676  7733 rpcz_store.cc:276] 0709 12:26:03.861497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.220901 (+359404us) service_pool.cc:224] Handling call
0709 12:26:04.246485 (+ 25584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.169058 (+922573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.203475 (+ 34417us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003b56d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:05.203756 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115836800}
I20260709 12:26:05.187332 10031 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.222661  7702 leader_election.cc:400] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:05.222671  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:05.225270  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:05.227370  7702 leader_election.cc:403] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:05.242316  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.244788  7626 leader_election.cc:400] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:05.245203  7626 leader_election.cc:403] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:05.249747  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.254491  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:05.261423  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:05.262409  7812 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:05.262990  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.263933  7804 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:05.266534  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:05.269312  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44215.3us
I20260709 12:26:05.269232  7558 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:05.271126  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:05.273178  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:05.273907  7808 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:26:05.267302  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:05.272938  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.221252 10004 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:05.279552  7558 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.174856 10035 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.266458 10039 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:05.295049  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50426.6us
I20260709 12:26:05.296535  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:05.300966  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.289372  7558 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:05.339311  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68355.2us
I20260709 12:26:05.340608  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:05.302076  8415 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:05.318058  7558 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:05.255928 10031 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:05.353328  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54588.5us
I20260709 12:26:05.286245  9932 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:05.367722  7656 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:05.368248  7812 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:05.356274  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:05.300752  9960 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:05.302953  8412 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:05.351843 10108 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.210s	user 0.000s	sys 0.003s
I20260709 12:26:05.337703 10035 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:05.316237 10004 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:05.403959 10031 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:05.337241 10101 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:05.338119 10110 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:05.388073  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:05.386828  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46402.7us
I20260709 12:26:05.354377 10109 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:05.387964  7658 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:05.419549  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:05.380698 10036 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:05.427502  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 205107us
W20260709 12:26:05.434813  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 496046us
I20260709 12:26:05.384078  9988 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:05.438678  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.432425  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.081s	sys 0.145s
W20260709 12:26:05.437692  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49751.9us
I20260709 12:26:05.440668  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:05.439162  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.081s	sys 0.145s
I20260709 12:26:05.434394  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:05.444298  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 528535us
I20260709 12:26:05.445649  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:05.441829 10004 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:05.472401  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:05.444111 10101 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.448601  9960 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:05.447458 10012 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:05.475323  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:05.478508  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 378443us
W20260709 12:26:05.471280 10024 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.801s	user 0.016s	sys 0.000s
I20260709 12:26:05.486579  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.481541  9960 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.437224 10109 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.480610 10012 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:05.488149 10101 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:05.543720 10111 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 12:26:05.606933  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2305) took 1279 ms. Trace:
W20260709 12:26:05.564653  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 563355us
I20260709 12:26:05.607465  7803 rpcz_store.cc:276] 0709 12:26:04.327806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.390425 (+ 62619us) service_pool.cc:224] Handling call
0709 12:26:04.390905 (+   480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.391503 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.391538 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.621659 (+230121us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.047494 (+425835us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:05.443206 (+395712us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.444428 (+  1222us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.447172 (+  2744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.447315 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.606669 (+159354us) raft_consensus.cc:1685] finished
0709 12:26:05.606697 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.606863 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3109,"raft.queue_time_us":17650,"raft.run_cpu_time_us":1425,"raft.run_wall_time_us":16227,"spinlock_wait_cycles":11304192,"thread_start_us":574090,"threads_started":2,"wal-append.queue_time_us":460922}
I20260709 12:26:05.611651  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:05.481015 10039 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:26:05.614318 10039 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:05.614790 10039 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:05.615495 10031 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:05.509557 10113 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:26:05.486593  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:05.618409  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.619520  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 150168us
I20260709 12:26:05.619831 10004 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.622339  7702 leader_election.cc:400] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:05.622773  7702 leader_election.cc:403] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:05.462751  9731 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:05.596904  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 124561us
I20260709 12:26:05.594105 10088 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:05.625216 10120 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:05.626117 10120 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:05.577390  7808 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:05.599448  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.599956  7733 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:05.602218 10012 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:05.627393 10057 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.627964 10057 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.618343 10110 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:05.622648 10109 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:05.513911 10024 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:26:05.584471  9988 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:05.642678  9988 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.643425 10057 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:05.614705  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:05.614969  9928 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 12:26:05.646974  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:05.638980  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.647727  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:05.629348  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:05.595713  9960 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:05.594692 10081 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:05.594709 10051 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:05.596375 10044 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.658674 10081 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.667258  9939 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:05.668268  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.685783  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:05.686268  8409 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:05.669039  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.685783  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:05.671520  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1841) took 1467 ms. Trace:
I20260709 12:26:05.627144 10120 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.634943 10110 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:05.657876  9828 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:05.680413  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.691386  7733 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:05.694546  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.688223  7736 rpcz_store.cc:276] 0709 12:26:04.203902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.577743 (+373841us) service_pool.cc:224] Handling call
0709 12:26:04.612565 (+ 34822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.089402 (+476837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.089431 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.099817 (+ 10386us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.537581 (+437764us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.537648 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.537801 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.671256 (+133455us) raft_consensus.cc:1685] finished
0709 12:26:05.671289 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.671444 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2464,"raft.queue_time_us":300943,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":152,"thread_start_us":824331,"threads_started":2,"wal-append.queue_time_us":569166}
I20260709 12:26:05.616408 10039 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.698764  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:05.698899  7732 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:05.684995 10051 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:05.684247  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 562) took 1458 ms. Trace:
I20260709 12:26:05.685921 10081 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:05.696115 10110 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:05.685539  9939 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:05.706857  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 261350us
W20260709 12:26:05.710153  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.751873  7778 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.960s (SENT)
I20260709 12:26:05.752720  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:05.700690 10101 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:05.685657 10044 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.773626  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.774772  8409 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 1.
W20260709 12:26:05.775825  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 153672us
W20260709 12:26:05.780073  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.780793  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.790385  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 305490us
I20260709 12:26:05.794121  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:05.802615  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.770164  9731 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:05.806667  7627 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.000s	sys 0.021s
W20260709 12:26:05.807251  7627 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.000s	sys 0.021s
W20260709 12:26:05.809137  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 556965us
I20260709 12:26:05.770866 10101 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.817520  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:05.816536  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.825960  8408 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:05.826745  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:05.803974 10039 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:05.834554  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.802336 10109 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:05.837898  9828 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:05.844684  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 146025us
I20260709 12:26:05.841444  9988 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:05.842947  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.706280  8412 rpcz_store.cc:276] 0709 12:26:04.221914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.233875 (+ 11961us) service_pool.cc:224] Handling call
0709 12:26:04.234344 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.386386 (+1152042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.645297 (+258911us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:05.679830 (+ 34533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407451392}
I20260709 12:26:05.867821  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:05.869208  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 105159us
I20260709 12:26:05.826293  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1627) took 1314 ms. Trace:
W20260709 12:26:05.878383  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60940.7us
I20260709 12:26:05.850375 10136 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:05.859899  9988 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.898068  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.894903  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:05.867802  9731 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:05.829151 10101 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.815951  9939 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.832970 10031 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:05.901163  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78313.9us
I20260709 12:26:05.914305  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.861394  9928 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:26:05.853130 10132 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:26:05.852725 10109 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.902988  9988 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:05.848748 10051 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:05.910804 10113 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.976609  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:05.981976  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:05.971310 10131 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:26:05.927387  9731 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:05.929509 10004 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:05.908134 10134 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:26:05.960332  9939 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:05.910082 10012 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:05.999893  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:06.066377  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.614s (SENT) [suppressed 45 similar messages]
I20260709 12:26:06.086107 10156 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.089298  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.118074  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51763.5us
W20260709 12:26:06.119352  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.127648  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 447) took 1091 ms. Trace:
I20260709 12:26:06.000712  9826 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.931655 10136 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
W20260709 12:26:06.087968 10142 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:26:06.128640  7735 rpcz_store.cc:276] 0709 12:26:05.036333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.036757 (+   424us) service_pool.cc:224] Handling call
0709 12:26:05.037256 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.037572 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:05.457390 (+419818us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.127032 (+669642us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.127100 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:06.127178 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.127424 (+   246us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.127577 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":633222,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":49774,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":4098,"thread_start_us":1035650,"threads_started":3,"wal-append.queue_time_us":409343}
W20260709 12:26:06.133165  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 228756us
W20260709 12:26:06.134032  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.972s (SENT)
W20260709 12:26:06.115792 10036 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.662s	user 0.014s	sys 0.000s
I20260709 12:26:06.136036  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:06.137382  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1639) took 1041 ms. Trace:
I20260709 12:26:05.931874  9828 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.155335  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:06.162322  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.983410 10113 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.128062  9902 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4.
I20260709 12:26:06.087344 10004 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.171743  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.159632  7662 rpcz_store.cc:276] 0709 12:26:05.096468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.096671 (+   203us) service_pool.cc:224] Handling call
0709 12:26:05.097082 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.103406 (+  6324us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:05.570203 (+466797us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.136888 (+566685us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.136958 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:06.137019 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.137108 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.137292 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6389,"prepare.queue_time_us":564888,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":78313,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5244,"thread_start_us":981949,"threads_started":3,"wal-append.queue_time_us":322122}
W20260709 12:26:06.185717  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.189628  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 294999us
I20260709 12:26:06.190603  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:06.199369  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65378.7us
I20260709 12:26:06.204342  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:06.170579 10138 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:26:06.212822  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.214766  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.139640  9928 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260709 12:26:06.176782 10035 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.004s
I20260709 12:26:06.171764 10036 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:06.139317  9826 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.175887 10153 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:06.125473 10031 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.172851 10155 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:06.135839 10156 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:06.241912  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.254400  8409 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:26:06.260310  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 288959us
W20260709 12:26:06.258970  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111011us
I20260709 12:26:06.244421 10035 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.281713 10113 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.281320 10035 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.240871 10153 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:06.251904 10134 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.284590  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:06.299933  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 95685.6us
I20260709 12:26:06.251991 10088 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:06.273041 10130 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.361s	user 0.000s	sys 0.002s
I20260709 12:26:06.313501  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:05.881769  7653 rpcz_store.cc:276] 0709 12:26:04.509270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.529358 (+ 20088us) service_pool.cc:224] Handling call
0709 12:26:04.989025 (+459667us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:04.989339 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.989877 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.989921 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.990735 (+   814us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.461711 (+470976us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:05.469572 (+  7861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.484889 (+ 15317us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.764875 (+279986us) raft_consensus.cc:1685] finished
0709 12:26:05.767814 (+  2939us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.823908 (+ 56094us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2918,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":152,"spinlock_wait_cycles":8830336,"thread_start_us":452114,"threads_started":1,"wal-append.queue_time_us":495982}
I20260709 12:26:06.319432  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:06.320407  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.323040  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:06.323829  7806 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:06.226167  9731 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.283020 10160 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:06.330528  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:06.315646 10088 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:06.338557  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:06.292557 10153 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:06.339448  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:06.317049 10109 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:06.329746 10145 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.360s	user 0.000s	sys 0.002s
I20260709 12:26:06.322815  8412 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:06.291946 10136 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:26:06.318261 10165 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:06.344457  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.251462 10155 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:06.352751 10152 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:26:06.290693 10164 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 12:26:06.316509 10134 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.383881  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:06.387521  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 236850us
I20260709 12:26:06.388583  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:06.347613 10012 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.356819 10160 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:06.356833 10165 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:06.377849 10109 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:06.373369 10146 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:26:06.400090 10178 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.428865  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:06.431559  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1643) took 1232 ms. Trace:
I20260709 12:26:06.436744  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:06.439707  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.390282 10036 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:06.400485 10164 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:06.431977  7655 rpcz_store.cc:276] 0709 12:26:05.204830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.207359 (+  2529us) service_pool.cc:224] Handling call
0709 12:26:05.255400 (+ 48041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.284815 (+ 29415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.284855 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.291506 (+  6651us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.699042 (+407536us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:05.699084 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.699158 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.431330 (+732172us) raft_consensus.cc:1685] finished
0709 12:26:06.431353 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.431497 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8423,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":22547,"thread_start_us":316212,"threads_started":1,"wal-append.queue_time_us":522354}
I20260709 12:26:06.401412  9928 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:06.400177 10171 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:06.429683  9939 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:06.430068 10165 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:06.449338 10139 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:26:06.400097 10136 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:26:06.422034 10174 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:06.402590  9932 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260709 12:26:06.467833  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 135942us
I20260709 12:26:06.445482 10164 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:06.455698  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.452257 10171 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:06.464206  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.473416  8409 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3. Re-sending same reply.
W20260709 12:26:06.462982  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 155538us
I20260709 12:26:06.486788  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.486928  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:06.453230 10166 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.454589 10118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.014s	sys 0.009s
I20260709 12:26:06.467502  9939 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:06.495631  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:06.498440  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59373.3us
I20260709 12:26:06.469012 10170 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:06.504139 10166 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.504901  7803 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:06.437968 10160 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.481815 10157 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:06.493597 10118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.015s	sys 0.010s
W20260709 12:26:06.584111  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.470069 10179 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:06.442562 10178 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.601172 10155 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.602608 10178 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:06.471992 10012 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:26:06.576009  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:06.578027  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.481699 10035 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.488826 10159 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:06.588339  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101496us
W20260709 12:26:06.587680  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 140288us
I20260709 12:26:06.503281 10112 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.581076  9731 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:06.629086  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.629426  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:06.582947 10109 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.619480  9928 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:06.654853  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.635264 10112 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:06.662432 10112 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.664158  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:06.615272 10157 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:06.656324  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:06.673107  7803 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:06.657089  7736 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:06.673977  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:06.676772  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:06.678258  7735 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:06.630160  7806 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:06.587002 10170 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:06.682304  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:06.688726  7731 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:06.688571  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.613744 10156 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:06.627851 10171 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.758694  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:06.614523 10179 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:06.806129  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.814726  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.749266  7729 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:06.671499 10109 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.800292  7806 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:06.884778  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.770879 10110 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.006s	user 0.012s	sys 0.000s
W20260709 12:26:06.879036  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.012s	sys 0.004s
W20260709 12:26:06.890652  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.004s
I20260709 12:26:06.708392 10036 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:06.765961 10167 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:06.742816 10154 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:06.916695 10179 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:06.765206 10160 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:06.902153  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 246568us
I20260709 12:26:06.771332  9928 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.857407  9902 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:06.839581 10039 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.983s	user 0.015s	sys 0.002s
I20260709 12:26:06.900490 10110 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:06.916184 10181 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.162s	user 0.002s	sys 0.000s
I20260709 12:26:06.915762 10036 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.758721 10191 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:06.806450 10166 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:06.919801  8408 raft_consensus.cc:1217] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 12:26:06.761085 10134 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:06.919821  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:06.940225  8408 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:26:06.710100  9731 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.208s
I20260709 12:26:06.828563 10159 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:06.934208  7808 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:06.870813 10136 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:06.931787 10153 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.007s
W20260709 12:26:06.946722  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 332949us
I20260709 12:26:06.947930  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:06.965406  7808 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:26:06.968907  7658 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:06.969373  7659 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:06.978420  7658 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:07.009809  7659 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:07.010205  7653 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:26:07.013671 10188 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:26:06.979538  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:07.017907  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 574) took 1131 ms. Trace:
I20260709 12:26:06.744346  9939 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.030596  7657 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:06.939028  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:06.940807  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 569) took 1324 ms. Trace:
I20260709 12:26:07.065361  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:07.018627  8407 rpcz_store.cc:276] 0709 12:26:05.886363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.893409 (+  7046us) service_pool.cc:224] Handling call
0709 12:26:05.907442 (+ 14033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.907784 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.907828 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.912769 (+  4941us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.136479 (+223710us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.136559 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.136660 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.017592 (+880932us) raft_consensus.cc:1685] finished
0709 12:26:07.017616 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.017820 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":156817,"threads_started":1,"wal-append.queue_time_us":1003384}
I20260709 12:26:07.065382  7658 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
W20260709 12:26:07.071321  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120362us
I20260709 12:26:07.123631  7808 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:07.120666  7654 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:06.955952  9902 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:07.157029  7808 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:26:07.157671  7808 pending_rounds.cc:85] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 4
I20260709 12:26:07.157961  7808 pending_rounds.cc:107] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:26:07.158248  7808 raft_consensus.cc:2889] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:07.165133  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.139899  7806 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:07.194758  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 462) took 1052 ms. Trace:
I20260709 12:26:07.195851 10110 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.200071  9902 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.182704 10176 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:07.205073 10081 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.449s	user 0.017s	sys 0.000s
I20260709 12:26:07.205283 10176 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.205735 10081 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.207442  7656 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:07.195578  7737 rpcz_store.cc:276] 0709 12:26:06.141972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.142128 (+   156us) service_pool.cc:224] Handling call
0709 12:26:06.168287 (+ 26159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.211994 (+ 43707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.212043 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.241341 (+ 29298us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.376758 (+135417us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:06.376824 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.376964 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.194505 (+817541us) raft_consensus.cc:1685] finished
0709 12:26:07.194527 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.194685 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":243762,"spinlock_wait_cycles":22589696,"thread_start_us":129230,"threads_started":2,"wal-append.queue_time_us":907959}
I20260709 12:26:07.210307  9826 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.157406 10157 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.224730  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:07.225936  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:07.228011  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.229224  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:26:07.230685  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.231869  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.232915  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.229579 10161 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:07.243136  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:07.137866  8412 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3. Re-sending same reply.
I20260709 12:26:07.240056  9828 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.261318  9828 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.261708 10174 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:07.261948  9828 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:06.932222 10167 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.269587 10167 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.261219  7653 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:07.123073 10191 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:07.272960 10191 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:06.961429 10153 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.278369 10153 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:07.262261 10174 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.279805 10174 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:07.240319 10161 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.284497 10161 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:07.278439  7812 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:07.240530 10031 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:07.288216 10031 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:07.288635 10031 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.240816 10057 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.293649 10057 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.294216 10057 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:07.300207  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 473) took 1309 ms. Trace:
I20260709 12:26:07.300719  8408 rpcz_store.cc:276] 0709 12:26:05.990920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.991150 (+   230us) service_pool.cc:224] Handling call
0709 12:26:06.919690 (+928540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.944682 (+ 24992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.299973 (+355291us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54004ba850. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:07.300127 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915163392}
I20260709 12:26:07.305451 10164 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.306895 10164 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:07.305454 10109 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.309556  9827 raft_consensus.cc:1064] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:26:07.312563  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:07.313555 10176 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:06.960125 10159 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:07.332743  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.252338  9988 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.279s	user 0.010s	sys 0.014s
W20260709 12:26:07.058416  9960 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.365s	user 0.007s	sys 0.011s
I20260709 12:26:07.274746 10044 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:07.356105 10165 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.883s	user 0.013s	sys 0.003s
I20260709 12:26:07.159314 10177 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.536s
I20260709 12:26:07.133092 10136 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:07.144074  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 183528us
I20260709 12:26:07.064160 10170 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:07.386062  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49460.2us
I20260709 12:26:07.348891 10110 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.404582  8379 connection.cc:256] RPC call timeout handler was delayed by -2.1579641280000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 12:26:07.403033  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:07.358151 10200 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.413340  8407 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:07.406038  8379 connection.cc:256] RPC call timeout handler was delayed by -1.7439641279997886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 12:26:07.404668  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:07.440202  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:07.450181 10203 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:26:07.454237 10176 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.455020 10176 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.458676 10176 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:07.460716  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:07.462560  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 316481us
I20260709 12:26:07.465878  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:07.467191  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:07.468218  7805 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:07.468534 10196 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:07.469869  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:07.471455 10192 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:07.472373 10192 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.361852  9988 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.362231 10202 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:26:07.482249  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:07.495836  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.496186  7734 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:26:07.502913  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 62833.4us
I20260709 12:26:07.386746  9960 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.508635  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:07.509506  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:07.510475  7658 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:07.511931  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:07.512781  7657 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:07.515807  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.514971  7657 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:07.517378  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:07.412842 10110 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.521735  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:07.386518 10165 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:07.529122  7662 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5.
I20260709 12:26:07.471009 10196 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.529554  7658 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:07.528332  7811 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:26:07.536659  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 132064us
I20260709 12:26:07.476284 10199 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.475044 10156 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.529572 10153 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:26:07.476995 10159 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.425515 10109 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:26:07.551019  7808 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:07.522444  9985 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:07.529176 10176 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:07.562628  7732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.018s	sys 0.069s
I20260709 12:26:07.529274 10039 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:07.563944  7732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.018s	sys 0.069s
W20260709 12:26:07.564291  7733 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.037s	sys 0.089s
W20260709 12:26:07.573905  7733 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.037s	sys 0.089s
W20260709 12:26:07.590991  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 52299.8us
I20260709 12:26:07.552048 10166 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:07.543159 10168 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:07.599294  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 164705us
I20260709 12:26:07.447757 10200 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.523874 10057 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.638693 10200 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.639205 10057 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.618546  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:07.640131  7734 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:07.641985  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:07.642732  7728 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:07.628919 10193 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:26:07.631373  9827 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.647953  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:07.568327 10176 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.532213 10148 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:26:07.655270 10176 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:07.595078 10182 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:07.616402 10088 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.252s	user 0.015s	sys 0.005s
I20260709 12:26:07.658300 10088 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.660190 10148 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.660629 10088 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.635399  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:07.660794 10148 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.643817  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:07.662052  9828 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.556582 10156 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.665459 10156 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.662053  7728 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:07.586541 10039 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.667707 10039 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.566666  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:07.671679  7653 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:07.672871 10165 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:07.675633  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:07.075332  8405 rpcz_store.cc:276] 0709 12:26:05.610523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.613059 (+  2536us) service_pool.cc:224] Handling call
0709 12:26:05.613436 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.613709 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:05.815340 (+201631us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.859125 (+1043785us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18001d77c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:26:06.879920 (+ 20795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.890265 (+ 10345us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:06.895873 (+  5608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.908576 (+ 12703us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.934827 (+ 26251us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":291784,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":74,"raft.queue_time_us":13518,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":8939,"spinlock_wait_cycles":7435136,"thread_start_us":280678,"threads_started":2,"wal-append.queue_time_us":275474}
I20260709 12:26:07.654222 10161 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.636305  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:07.663255 10148 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.550904 10199 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:07.685690  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:07.686339 10110 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:07.681921 10200 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:07.679456  7731 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:07.687613 10200 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.687124 10199 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:07.677026  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:07.681018  7735 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
W20260709 12:26:07.679144  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 43840.6us
I20260709 12:26:07.689533  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:07.690871  7735 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:07.687094  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.691046  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:07.692409  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:07.692704  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:07.693351 10109 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:07.693826  7735 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:07.689872 10200 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.593808 10178 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:07.545581 10196 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:07.738742  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1859) took 1107 ms. Trace:
W20260709 12:26:08.022687  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.737030  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.722302  7728 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:08.053025  7732 rpcz_store.cc:276] 0709 12:26:06.630825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.653648 (+ 22823us) service_pool.cc:224] Handling call
0709 12:26:06.673281 (+ 19633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.007583 (+334302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.007636 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.008510 (+   874us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.575988 (+567478us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:07.576090 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.576233 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.738443 (+162210us) raft_consensus.cc:1685] finished
0709 12:26:07.738464 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.738663 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117891,"raft.queue_time_us":229149,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":159,"thread_start_us":708543,"threads_started":2,"wal-append.queue_time_us":727620}
W20260709 12:26:08.074919  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.000s
W20260709 12:26:08.078203  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.003s	sys 0.000s
I20260709 12:26:08.072722  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 484) took 1104 ms. Trace:
W20260709 12:26:08.078816  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 559831us
W20260709 12:26:08.054661  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.078955  7803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.014s	sys 0.025s
W20260709 12:26:08.081780  7803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.014s	sys 0.025s
I20260709 12:26:08.071255  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.082780  7730 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
W20260709 12:26:08.033080 10157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.019s	sys 0.015s
I20260709 12:26:08.079851  7656 rpcz_store.cc:276] 0709 12:26:06.968200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.968356 (+   156us) service_pool.cc:224] Handling call
0709 12:26:06.968706 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.207880 (+239174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.072327 (+864447us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54003980d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:08.072619 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548561280}
W20260709 12:26:08.111701  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 419343us
W20260709 12:26:07.957141 10214 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.211s	user 0.000s	sys 0.005s
I20260709 12:26:07.648869 10081 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:07.826683 10164 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
W20260709 12:26:08.119899  7660 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.004s	sys 0.010s
W20260709 12:26:08.078732  9939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.014s	sys 0.027s
I20260709 12:26:07.695838 10192 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:08.136857  9939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.014s	sys 0.031s
I20260709 12:26:08.137645  9939 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.797998 10193 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.138554  7660 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.004s	sys 0.012s
I20260709 12:26:07.840135  9988 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.684583 10165 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:08.132951  7658 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.018s	sys 0.033s
W20260709 12:26:08.156378  7658 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.019s	sys 0.033s
I20260709 12:26:08.078223  9960 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:08.118410  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1858) took 1452 ms (client timeout 1924 ms). Trace:
W20260709 12:26:08.159412  7730 rpcz_store.cc:269] 0709 12:26:06.630503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.654249 (+ 23746us) service_pool.cc:224] Handling call
0709 12:26:08.071002 (+1416753us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:08.083166 (+ 12164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3847552}
W20260709 12:26:08.135669  7655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.000s	sys 0.048s
W20260709 12:26:08.160878  7655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.000s	sys 0.049s
I20260709 12:26:07.745528 10136 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:07.734737 10199 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:08.026858 10110 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.716287 10109 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:08.146927  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:08.111742 10205 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.017s	sys 0.027s
I20260709 12:26:07.697714 10156 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:08.093806  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:07.936842 10203 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:08.215816 10203 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.216935 10196 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.217803 10196 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:08.218252 10196 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:08.182643 10205 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.017s	sys 0.031s
I20260709 12:26:08.218585 10203 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.218755 10196 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:08.169269  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 480) took 1231 ms. Trace:
I20260709 12:26:08.160490 10165 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.241719  7655 rpcz_store.cc:276] 0709 12:26:06.938305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.938920 (+   615us) service_pool.cc:224] Handling call
0709 12:26:06.958169 (+ 19249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.011974 (+ 53805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.012004 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.012736 (+   732us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.352361 (+339625us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:08.161422 (+809061us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.168833 (+  7411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.168958 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.169010 (+    52us) raft_consensus.cc:1685] finished
0709 12:26:08.169021 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.169185 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35487,"raft.queue_time_us":11556,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":42134,"thread_start_us":1111270,"threads_started":2,"wal-append.queue_time_us":544203}
I20260709 12:26:07.741750 10176 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:08.129866 10160 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.020s	sys 0.040s
I20260709 12:26:08.178244  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2328) took 1342 ms. Trace:
I20260709 12:26:08.117184 10133 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:26:08.213587  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:08.134709 10200 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:08.265389 10160 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.020s	sys 0.040s
I20260709 12:26:08.221053  7653 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:08.111827 10157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.019s	sys 0.016s
I20260709 12:26:07.720649 10031 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:08.276362 10208 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:26:08.281397 10031 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.224174  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 490) took 1245 ms. Trace:
I20260709 12:26:08.214359 10110 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.282558  7660 rpcz_store.cc:276] 0709 12:26:06.978457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.978623 (+   166us) service_pool.cc:224] Handling call
0709 12:26:07.072676 (+ 94053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.073016 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.073057 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.130607 (+ 57550us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.565038 (+434431us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.139364 (+574326us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.139719 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.139826 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.223876 (+ 84050us) raft_consensus.cc:1685] finished
0709 12:26:08.223908 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.224080 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13775,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":9110,"thread_start_us":876584,"threads_started":2,"wal-append.queue_time_us":1053865}
W20260709 12:26:08.252935  9902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.012s	sys 0.035s
I20260709 12:26:08.211730  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:08.271166  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.177409 10199 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:08.167018 10164 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.285652 10199 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:08.286103 10110 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:08.285843 10164 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.214720 10156 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:08.284994  9902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.012s	sys 0.035s
I20260709 12:26:08.153523  9939 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.288581 10156 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:08.288224 10039 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.005s
I20260709 12:26:08.289201 10039 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.290306  9939 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.288291 10164 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.291556  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.291656 10039 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.265529  7803 rpcz_store.cc:276] 0709 12:26:06.836385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.836632 (+   247us) service_pool.cc:224] Handling call
0709 12:26:06.871458 (+ 34826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.871712 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:07.390289 (+518577us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.177667 (+787378us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.177761 (+    94us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:08.177838 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.177939 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.178151 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":742297,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":73995,"spinlock_wait_cycles":13708544,"thread_start_us":1280760,"threads_started":3,"wal-append.queue_time_us":764766}
W20260709 12:26:08.294235  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.295789 10164 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.297546 10164 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:08.310261  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 132275us
I20260709 12:26:08.311910  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.314333  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.315749  9902 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:08.315908 10157 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:08.316319 10200 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:08.315500 10199 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.321446  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:08.322781 10166 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.679s	user 0.005s	sys 0.002s
I20260709 12:26:08.323433 10200 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:08.323923 10200 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.324290 10200 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:08.324748 10176 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.330649 10035 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:08.336701 10164 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.340142 10035 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.246639 10165 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:08.332294  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1863) took 1700 ms (client timeout 2248 ms). Trace:
W20260709 12:26:08.335706 10174 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.055s	user 0.012s	sys 0.003s
W20260709 12:26:08.344527  7729 rpcz_store.cc:269] 0709 12:26:06.632031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.655196 (+ 23165us) service_pool.cc:224] Handling call
0709 12:26:08.331968 (+1676772us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b54002d7dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:08.332195 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111420160}
I20260709 12:26:08.344787 10164 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:08.333379  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.346813 10174 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.158800 10193 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.349979 10174 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.349979 10193 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.349067 10035 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.351679 10193 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:08.352941 10088 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:08.355518  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.292531 10039 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:08.290926  9939 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.355603  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:08.387763  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.388417  7655 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:26:08.388650  7653 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:08.389099  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.389616  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.390283  7655 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:08.366869  7659 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:08.392198  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.392109 10156 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.395520 10164 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.397841  9939 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.398348 10164 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.399981 10088 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:08.400517 10088 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:08.400002 10210 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:08.401014 10088 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:08.401937  9939 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.400527 10199 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.403594 10222 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:08.404568  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.405536 10210 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:08.391533 10200 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.409890 10200 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:08.353987 10193 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.402973  7653 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:08.413807  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.414065  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.415055  7659 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:08.416759 10166 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:08.394891 10156 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.418306 10134 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:08.419037 10134 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:08.420300 10134 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.422465 10088 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.412564 10193 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.406946  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:08.437428  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 317462us
I20260709 12:26:08.438284  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.439476  7659 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:08.243563 10161 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.435043  7655 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:08.444768  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.451828  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.459666  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:08.471956  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.424820 10035 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:08.473388  7733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.030s	sys 0.079s
W20260709 12:26:08.475577  7733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.030s	sys 0.079s
I20260709 12:26:08.446954 10166 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:08.465214 10210 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:08.502159  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.467666 10039 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:08.450932 10200 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:26:08.481818 10168 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:08.508598  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.515673  7729 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:08.504560  7656 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:08.503686  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 288400us
I20260709 12:26:08.494153  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.504803  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.531731  7661 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:08.506477  7655 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:08.474627 10110 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:26:08.537249  7658 raft_consensus.cc:2410] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:08.503618 10191 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.000s	sys 0.014s
I20260709 12:26:08.541854  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:08.542740  7811 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:08.540211  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.540577  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.544337  7735 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
I20260709 12:26:08.545289  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.547065  7655 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:08.546208  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.541652  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.548583  7805 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:08.547817  7730 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:08.549583  7737 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:08.551683  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.551944  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.552387  7737 raft_consensus.cc:2376] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
W20260709 12:26:08.539249 10149 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.309s	user 0.009s	sys 0.028s
I20260709 12:26:08.483695 10035 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.558012  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.554212  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:08.542344 10191 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.015s
I20260709 12:26:08.555678  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.555955  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.561348  7811 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:08.528585 10237 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:08.561966  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.562366 10237 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.565433 10237 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.566510  7729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691f0d885861432f91375d5f288af312"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:26:08.567471  7729 raft_consensus.cc:606] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:26:08.567929  7729 raft_consensus.cc:614] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:08.556875  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.570596  7656 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:08.571802  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.559638  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.554845  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.558885  7661 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:08.572592  7804 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:08.546805 10133 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:08.579464 10133 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:08.581084 10101 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:08.587247  7607 auto_rebalancer.cc:629] Failed to schedule move for tablet 0a2203ee13714c8bacacb6f83ec31034: Illegal state: Replica fc963afbbf4c4b34b161448ef79bd2ed is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:08.568553  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.578035 10196 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.561848 10088 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.594534  9828 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:08.573261  7658 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:08.658165  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.659337  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.659906  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.660480  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.661011  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:08.647147  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 400) took 1630 ms. Trace:
I20260709 12:26:08.647580  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.651872 10237 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:26:08.586061 10039 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.668053  7655 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:26:08.649102  7700 leader_election.cc:336] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.914s (SENT)
W20260709 12:26:08.558393 10149 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.332s	user 0.010s	sys 0.028s
I20260709 12:26:08.650211  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.653736  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.670763  7734 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:08.664748  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.652901  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:08.654875 10235 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:26:08.648247  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2329) took 1439 ms. Trace:
I20260709 12:26:08.667587  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.673888  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.647856 10184 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:08.673943  7808 rpcz_store.cc:276] 0709 12:26:07.208510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.208641 (+   131us) service_pool.cc:224] Handling call
0709 12:26:07.208998 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.551532 (+342534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.596907 (+1045375us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400373cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:08.648167 (+ 51260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786057472}
I20260709 12:26:08.676234  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:08.667801  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 228431us
I20260709 12:26:08.677561  7655 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:08.647542 10182 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:08.671945  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.681893  7661 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:08.683413  7661 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:26:08.651571 10226 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.093s	user 0.001s	sys 0.006s
I20260709 12:26:08.664080  7658 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:08.687733  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.689002  7656 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:08.672770  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.680130  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.685266  7661 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:08.691301  7730 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:08.686323  7655 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:08.691324  7804 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:08.663870  7809 rpcz_store.cc:276] 0709 12:26:07.016141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.058275 (+ 42134us) service_pool.cc:224] Handling call
0709 12:26:07.139036 (+ 80761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.408333 (+269297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.408389 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:07.408464 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:07.408535 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.582960 (+1174425us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54001338d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:08.646335 (+ 63375us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.646491 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618409216}
I20260709 12:26:08.665118 10171 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:08.708103  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.657460  7653 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:08.712316 10246 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:08.712682  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.688565 10182 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:08.713500  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:08.713595 10004 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.707383 10196 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:08.714581  7807 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:08.716042 10035 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.743623  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.718675  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.744755  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.707051 10115 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.723193  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.751516  7656 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:08.751004  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:08.700037  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47201.7us
I20260709 12:26:08.700914  7658 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:08.753952  7808 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:08.754514  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.696851  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.756109  7734 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:08.680132 10184 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:08.694933 10193 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:08.733719 10036 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.760602  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.758325  7805 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:08.765851  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2323) took 2274 ms. Trace:
I20260709 12:26:08.768641  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:08.746522 10194 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.766587  7812 rpcz_store.cc:276] 0709 12:26:06.484699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.484882 (+   183us) service_pool.cc:224] Handling call
0709 12:26:06.507676 (+ 22794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.286818 (+779142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.759197 (+1472379us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54003c9e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:08.759427 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772029312}
I20260709 12:26:08.781179 10248 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:26:08.681227 10133 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.758529 10115 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.789543 10194 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.783447 10036 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:08.761822 10193 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:08.790143  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1860) took 2159 ms. Trace:
I20260709 12:26:08.806351  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.800302  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.807042  7806 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:08.810523 10036 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:08.816072  7811 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:08.813102  7733 rpcz_store.cc:276] 0709 12:26:06.631139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.631295 (+   156us) service_pool.cc:224] Handling call
0709 12:26:06.664117 (+ 32822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.574601 (+910484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.574648 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.575349 (+   701us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.476219 (+900870us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:08.476286 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.476460 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.788874 (+312414us) raft_consensus.cc:1685] finished
0709 12:26:08.788898 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.790074 (+  1176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9569,"raft.queue_time_us":236928,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":1610777,"threads_started":2,"wal-append.queue_time_us":896828}
I20260709 12:26:08.798496 10115 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:08.824316  7812 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:08.826422  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.748075 10246 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:08.829612  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90295.5us
I20260709 12:26:08.829638  7806 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:08.827294  7730 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:08.833302  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:08.834051  7658 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:08.817517 10193 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:08.836194  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.837846  7729 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:08.789760 10253 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:08.846326  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:08.847642  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.848366  7729 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:08.851748  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:08.830679 10184 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.853233 10184 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:08.853320  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.853668  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.855020  7735 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:08.880467  7731 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:08.870756  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:08.989197  7729 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
W20260709 12:26:09.090858  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.082943  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.100308  7736 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:26:09.100131  7735 raft_consensus.cc:2376] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
I20260709 12:26:08.914219  7730 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:09.165334  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.166421  7729 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:09.167877  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.170011  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.873692  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.192148  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.907616  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2308) took 3699 ms. Trace:
I20260709 12:26:09.194526  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:09.195657  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 352459us
I20260709 12:26:09.197500  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:09.197947  7735 raft_consensus.cc:2410] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:08.868081  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:09.189726  7733 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:09.203778  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.205263  7730 raft_consensus.cc:2376] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3. Re-sending same reply.
I20260709 12:26:09.209477  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.210660  7736 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:09.193607  7810 rpcz_store.cc:276] 0709 12:26:05.207885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.208036 (+   151us) service_pool.cc:224] Handling call
0709 12:26:05.256194 (+ 48158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.309567 (+1053373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.309615 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.309672 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:06.309708 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.907338 (+2597630us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54003c50d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:08.907385 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.907545 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45889,"raft.queue_time_us":116743,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":8467,"spinlock_wait_cycles":2218008832,"thread_start_us":64677,"threads_started":1}
I20260709 12:26:09.214674  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.216043  7729 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:09.218930  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.219792  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.871903 10255 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:09.221354  7624 connection.cc:256] RPC call timeout handler was delayed by -1.4454882380000527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
I20260709 12:26:08.871177  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:09.234269  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.220453  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:08.925098 10051 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.237778  7810 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:09.239261  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.249294  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:09.260597  7806 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:26:08.842638 10165 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:08.887569  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:09.221370  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.192643 10035 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.271130  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.272289  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.218066 10246 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:09.278421  7731 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:09.289435  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.290318  7730 raft_consensus.cc:2376] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3. Re-sending same reply.
I20260709 12:26:09.292783  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.293998  7736 raft_consensus.cc:2376] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
I20260709 12:26:09.295573  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:08.876019 10249 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.404s
I20260709 12:26:08.874727 10182 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:09.299975  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:08.919670 10169 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:09.274338 10051 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:09.304847  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.325533  7732 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:09.224193  7624 connection.cc:256] RPC call timeout handler was delayed by -1.2447882380001829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 12:26:09.307472  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 459794us
I20260709 12:26:09.308542  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.313385  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:09.337035  7624 connection.cc:256] RPC call timeout handler was delayed by -1.233088238000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:26:09.337962  7624 connection.cc:256] RPC call timeout handler was delayed by -1.2285882380001567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
I20260709 12:26:09.316138  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.237425 10255 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:09.314747  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.337316  7731 raft_consensus.cc:2376] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
I20260709 12:26:09.338878  7729 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
W20260709 12:26:09.338423  7624 connection.cc:256] RPC call timeout handler was delayed by -1.0341882379998424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 12:26:09.345142  7624 connection.cc:256] RPC call timeout handler was delayed by -1.026088238000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
I20260709 12:26:09.346120  7736 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:09.332319  7733 raft_consensus.cc:2376] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
I20260709 12:26:09.298022 10035 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:09.312366 10246 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:09.373492  7729 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:26:09.562885  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.453704  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:09.436017  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 239393us
I20260709 12:26:09.386940  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.817714  7730 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:09.373656  7807 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:09.820639  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.821727  7733 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 2.
W20260709 12:26:09.629359  9985 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.005s	user 0.038s	sys 0.081s
W20260709 12:26:09.703742 10193 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.860s	user 0.011s	sys 0.002s
W20260709 12:26:09.835110  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.832638  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:09.840809  7777 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.003s	sys 0.020s
I20260709 12:26:09.839468  7806 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:26:09.865238  7777 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.026s
W20260709 12:26:09.865940  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 616883us
I20260709 12:26:09.866258  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:09.864672  9985 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.265s	user 0.041s	sys 0.090s
I20260709 12:26:09.355327 10182 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:09.871634  7812 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:09.457813 10171 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:09.873337  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:09.391171 10253 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:09.414198  8405 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:26:09.870496 10257 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:26:09.836567 10024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.009s	sys 0.013s
I20260709 12:26:09.877152  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:09.877280  7729 raft_consensus.cc:2376] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
I20260709 12:26:09.897279  7658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:09.323158 10169 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:09.413602 10035 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:09.908295  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.887599  7806 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:09.869252 10193 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:09.756821 10249 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 12:26:09.886407  7653 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:09.762567 10258 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:26:09.920183  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2168) took 1136 ms. Trace:
I20260709 12:26:09.873827  9985 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:09.889460 10253 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:09.928457  7734 rpcz_store.cc:276] 0709 12:26:08.783484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.783644 (+   160us) service_pool.cc:224] Handling call
0709 12:26:08.784029 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.785819 (+  1790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.785882 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.805961 (+ 20079us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.066900 (+260939us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:09.214041 (+147141us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.214295 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.214381 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.919932 (+705551us) raft_consensus.cc:1685] finished
0709 12:26:09.919953 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.920115 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":154593,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":4242,"raft.queue_time_us":339,"spinlock_wait_cycles":53097088,"thread_start_us":235662,"threads_started":1,"wal-append.queue_time_us":572298}
W20260709 12:26:09.907827 10024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.010s	sys 0.013s
I20260709 12:26:09.898892 10165 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:09.457527 10263 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.473s
I20260709 12:26:09.942103 10115 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:09.935315 10255 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:09.942850 10115 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:09.943872 10115 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:09.867715 10211 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:26:09.948952 10211 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:26:09.950659 10184 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.097s	user 0.015s	sys 0.000s
I20260709 12:26:09.895533 10235 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:26:09.951390  7735 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:09.938524  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.924168  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.952921  7729 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:09.953310  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:09.954352  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:09.893537 10171 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:09.940233 10264 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:26:09.942287 10263 consensus_queue.cc:1243] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:09.938699  7804 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:09.923544 10035 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:09.941191  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.011s	sys 0.044s
W20260709 12:26:09.968955  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.011s	sys 0.044s
I20260709 12:26:09.968815  7734 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
W20260709 12:26:09.969714  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 620812us
I20260709 12:26:09.966595  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:09.911841 10169 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:09.976814  7734 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:26:09.974346 10165 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.998860 10169 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:10.005255  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:10.034322  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:10.039131  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.976316 10035 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.034024 10182 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:26:10.070180  7700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.008s	sys 0.010s
W20260709 12:26:10.070655  7700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.008s	sys 0.010s
I20260709 12:26:10.006549 10165 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.001489 10253 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:10.079247  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 105749us
I20260709 12:26:10.073465 10184 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:10.082207  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.086143  7658 raft_consensus.cc:2376] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
W20260709 12:26:10.070953  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 1.01823e+06us
I20260709 12:26:10.046809  9985 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.076622 10273 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:10.089432 10184 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:10.091952 10193 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.090193 10182 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.103965  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69634.9us
I20260709 12:26:10.134778  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:10.109282 10035 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.140318  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.110056  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.141527  7657 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:10.111713  8404 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 with delete_type TABLET_DATA_TOMBSTONED (TS 992d6df95ef74f7f8b12534534a45501 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:57906
I20260709 12:26:10.143093  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.113030 10200 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:10.123654 10268 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:26:10.125473 10165 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.122872 10254 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:10.106295 10184 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.094657 10253 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.109925  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.104902  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.112440  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:10.145875 10269 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:10.147303  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:10.130637 10171 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:10.163014  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:10.166731  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:10.169348  7658 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:10.177485  7734 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:10.138695  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:10.179349  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:10.182771  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44203.7us
I20260709 12:26:10.184219  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:10.100418 10273 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.164512 10254 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:10.141029 10136 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:26:10.170405 10088 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:26:10.184844  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 352382us
I20260709 12:26:10.140476 10182 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.195017  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:10.197230 10035 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:10.197729 10035 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.198140 10035 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:10.197947 10263 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:10.198777 10263 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:10.199482 10088 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:10.167658 10165 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 12:26:10.196199  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.167814 10200 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.201928  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:10.206144  7734 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:10.189316 10282 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:10.211017  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.200224 10088 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:26:10.214399  7734 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:10.218340  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:10.216976  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.224421  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 580) took 1302 ms. Trace:
I20260709 12:26:10.224953  8413 rpcz_store.cc:276] 0709 12:26:08.922032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.922167 (+   135us) service_pool.cc:224] Handling call
0709 12:26:10.218153 (+1295986us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:10.224350 (+  6197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9212160}
I20260709 12:26:10.226217  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.226874  7734 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:10.227975  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:10.228848  7734 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.234187  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:10.236985  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.237290  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.237871  7734 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:10.242262  8379 leader_election.cc:336] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.627s (SENT)
I20260709 12:26:10.201998 10283 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:10.208006 10198 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:10.245735 10088 consensus_queue.cc:1243] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:10.239166  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.212745 10136 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:10.213277 10282 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:10.268955  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.268985  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72639.7us
I20260709 12:26:10.274369  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:10.209825 10274 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
W20260709 12:26:10.288491  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77986.1us
I20260709 12:26:10.215036 10286 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:26:10.289908  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:10.267470  9985 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:26:10.290506  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:10.297407  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:10.280766  7653 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:10.280532 10283 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:10.280143 10282 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:10.305804  7658 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:10.329747 10291 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:10.330324 10291 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:10.330711 10291 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.331918 10295 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:10.332494 10295 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:10.333477 10253 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:10.333953 10253 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:10.335747  7654 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:10.254150 10254 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.350059  7810 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:10.491909  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 97537.7us
W20260709 12:26:10.424598 10287 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 12:26:10.397804  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1873) took 1637 ms (client timeout 1677 ms). Trace:
I20260709 12:26:10.449802 10253 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:10.710263  7804 rpcz_store.cc:269] 0709 12:26:08.711775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.737364 (+ 25589us) service_pool.cc:224] Handling call
0709 12:26:10.349206 (+1611842us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:10.349483 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720768768}
I20260709 12:26:10.425343  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:10.712926  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.713460  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1872) took 2019 ms (client timeout 2311 ms). Trace:
I20260709 12:26:10.344115 10192 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:10.527033 10200 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:10.695070  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.711006 10253 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:10.682504 10171 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.442866  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 128617us
I20260709 12:26:10.343573  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:10.380365  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.723717  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:10.725034  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.725344 10300 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:26:10.725399  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 434919us
I20260709 12:26:10.711642 10292 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:10.342162  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:10.714375  7809 rpcz_store.cc:269] 0709 12:26:08.694391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.742843 (+ 48452us) service_pool.cc:224] Handling call
0709 12:26:10.425142 (+1682299us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:10.713387 (+288245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9785600}
W20260709 12:26:10.349113  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.355369  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1652) took 2822 ms. Trace:
I20260709 12:26:10.729575  7654 rpcz_store.cc:276] 0709 12:26:07.514505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.542472 (+ 27967us) service_pool.cc:224] Handling call
0709 12:26:10.335200 (+2792728us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:10.335465 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.336739 (+  1274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.337058 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12471040}
I20260709 12:26:10.721772  7655 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:10.448421 10134 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:10.730155  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.323791 10198 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:10.346253 10242 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:10.526197 10184 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.732683  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:10.733704  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.719633 10192 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:10.735121 10200 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:26:10.735127  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:10.736889  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.739967  7809 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:10.341730  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.709192 10298 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.622260 10169 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.592059 10035 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.743763  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.744856 10194 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.744197 10198 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.703739 10306 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:26:10.746448  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.747146 10282 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.333504 10295 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.747931  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:10.735368 10302 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:26:10.734908 10147 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:10.749244  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 482) took 3793 ms (client timeout 1627 ms). Trace:
I20260709 12:26:10.749454  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.731999 10171 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.750188  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:10.751019  8409 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:10.774026  8409 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:10.774330  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:10.775364  7654 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:10.756812  7804 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:10.760205  7656 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:10.478124 10182 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:10.707645 10305 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:26:10.774905  8409 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.779568  8409 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:10.783247 10182 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:10.765829  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:10.784142  7728 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.783958 10182 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:10.784384 10282 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.785342  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:10.786031  7809 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:10.786397 10182 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.766511  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.771606 10306 consensus_queue.cc:1243] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:10.739902 10242 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.788218 10242 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.788489  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.438099 10291 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.789893 10310 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:10.790381 10310 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:10.792557 10247 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.987s	user 0.002s	sys 0.021s
I20260709 12:26:10.767447  7734 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:10.753830 10192 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:10.769578  7733 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.737601 10292 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:10.770941  7662 rpcz_store.cc:269] 0709 12:26:06.947380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.947563 (+   183us) service_pool.cc:224] Handling call
0709 12:26:10.740329 (+3792766us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54002198d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:10.740550 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328671488}
I20260709 12:26:10.782797  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:10.786325  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:10.792428 10291 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:10.759675 10200 consensus_queue.cc:1243] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:10.771369  7731 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:10.765502  7736 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:10.789937  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:10.792490  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:10.347128 10283 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.763177 10298 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:10.768338 10308 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.131s
W20260709 12:26:11.020576  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157351us
I20260709 12:26:10.768977 10169 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:11.121223  7627 connection.cc:256] RPC call timeout handler was delayed by -3.0694563450001624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 12:26:11.121858  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7895563450001646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 12:26:11.122049  7737 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:11.122244  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7040760570002931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31560000000000005s for a response
W20260709 12:26:11.122653  7627 connection.cc:256] RPC call timeout handler was delayed by -2.5159760570004437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3365s for a response
I20260709 12:26:10.776363 10147 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.047979 10171 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:10.810127  8409 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:26:10.825862  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.890762  7809 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:11.124918 10255 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:26:10.961237 10182 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.125954  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 577) took 2205 ms. Trace:
I20260709 12:26:10.985920 10168 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:11.127074 10298 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:10.914235 10311 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:26:11.128100  7804 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.907104 10310 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.129433 10310 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:11.031447  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.130585  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:10.854592 10305 consensus_queue.cc:1243] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:11.131376  7736 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:10.891358  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.028313  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 581) took 1832 ms. Trace:
I20260709 12:26:11.132444  8415 rpcz_store.cc:276] 0709 12:26:09.195683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.196016 (+   333us) service_pool.cc:224] Handling call
0709 12:26:09.223578 (+ 27562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.340385 (+116807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.340420 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.340473 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:09.340516 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.027904 (+1687388us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:11.027988 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.028218 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267989760}
I20260709 12:26:11.133312  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:10.799165  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:10.796142  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:10.961097 10282 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:11.130851  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 584) took 1933 ms (client timeout 2233 ms). Trace:
I20260709 12:26:11.065307  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:10.961144 10289 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:11.117552 10051 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:11.134334  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:10.804198  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.137465  7627 connection.cc:256] RPC call timeout handler was delayed by -2.3569427790002919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2515s for a response
W20260709 12:26:11.137831  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.138078  7627 connection.cc:256] RPC call timeout handler was delayed by -2.1589427790004265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2735s for a response
I20260709 12:26:11.126518  8409 rpcz_store.cc:276] 0709 12:26:08.920824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.924520 (+  3696us) service_pool.cc:224] Handling call
0709 12:26:10.750653 (+1826133us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:10.750882 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.124718 (+373836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.125805 (+  1087us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11377408}
I20260709 12:26:11.138572 10169 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.139324 10292 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:11.140307  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.133555 10147 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:10.901934  7728 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:11.141801  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1878) took 1049 ms. Trace:
I20260709 12:26:11.141855  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.139708  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.142241  7728 rpcz_store.cc:276] 0709 12:26:10.092334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.761945 (+669611us) service_pool.cc:224] Handling call
0709 12:26:11.141727 (+379782us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.138496  7627 connection.cc:256] RPC call timeout handler was delayed by -2.1244105880000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
I20260709 12:26:11.135027 10088 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:11.143749  7627 connection.cc:256] RPC call timeout handler was delayed by -2.1172105880000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 12:26:11.119153 10184 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.135552  7803 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:11.148032  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1670) took 2676 ms. Trace:
I20260709 12:26:11.134790  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:11.144147  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0857105879999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 12:26:11.148993  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0632105879999472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 12:26:11.149696  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0245105879998846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 12:26:11.150122  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0110332310000558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 12:26:11.150426  7627 connection.cc:256] RPC call timeout handler was delayed by -1.9354105880001953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
I20260709 12:26:11.127211 10168 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:11.136425  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 585) took 1924 ms (client timeout 2100 ms). Trace:
I20260709 12:26:11.145016  8409 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
W20260709 12:26:11.150744  7627 connection.cc:256] RPC call timeout handler was delayed by -1.9120105880001574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:26:11.135807  8408 rpcz_store.cc:269] 0709 12:26:09.197366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.019932 (+1822566us) service_pool.cc:224] Handling call
0709 12:26:11.130782 (+110850us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.154347  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8445105880000483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 12:26:11.154803  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1886) took 1058 ms. Trace:
I20260709 12:26:11.148646  7659 rpcz_store.cc:276] 0709 12:26:08.472102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.472296 (+   194us) service_pool.cc:224] Handling call
0709 12:26:08.472664 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.473102 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.473138 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.473700 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.657827 (+184127us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:08.657869 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.657937 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.791503 (+2133566us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:11.115959 (+324456us) raft_consensus.cc:1685] finished
0709 12:26:11.116018 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.147958 (+ 31940us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4954,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":13596,"spinlock_wait_cycles":13111040,"thread_start_us":152873,"threads_started":1,"wal-append.queue_time_us":321817}
W20260709 12:26:11.153316  8414 rpcz_store.cc:269] 0709 12:26:09.212231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.064901 (+1852670us) service_pool.cc:224] Handling call
0709 12:26:11.136334 (+ 71433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.156245  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.158092  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.159483  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.142544  8407 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:11.122745  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1890) took 1017 ms. Trace:
I20260709 12:26:11.165712  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:11.148635  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 356194us
I20260709 12:26:11.162405  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.166955  7733 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:11.125000  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.865782  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 579) took 1941 ms (client timeout 2037 ms). Trace:
W20260709 12:26:11.154942  7627 connection.cc:256] RPC call timeout handler was delayed by -1.7878105879999566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 12:26:11.167570  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:11.172919  7627 connection.cc:256] RPC call timeout handler was delayed by -1.7761105879999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 12:26:11.173295  7627 connection.cc:256] RPC call timeout handler was delayed by -1.7599105879999115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
I20260709 12:26:11.173350  7736 rpcz_store.cc:276] 0709 12:26:10.096424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.130286 (+1033862us) service_pool.cc:224] Handling call
0709 12:26:11.154732 (+ 24446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.151904  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:11.168111  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.166509  7737 rpcz_store.cc:276] 0709 12:26:10.105389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.786026 (+680637us) service_pool.cc:224] Handling call
0709 12:26:11.122675 (+336649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.125933 10182 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:11.151160 10136 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.005s	sys 0.020s
W20260709 12:26:11.173667  7627 connection.cc:256] RPC call timeout handler was delayed by -1.7356105879998722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 12:26:11.160269  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.183403  8413 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:11.174412  7728 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:11.175036  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:11.167604  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1885) took 1071 ms. Trace:
I20260709 12:26:11.181387  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.181800  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:11.188864  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.190555  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.191308  7728 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:11.192029  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1887) took 1090 ms. Trace:
I20260709 12:26:11.192595  7728 rpcz_store.cc:276] 0709 12:26:10.101651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.190249 (+1088598us) service_pool.cc:224] Handling call
0709 12:26:11.191962 (+  1713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.192891  7733 rpcz_store.cc:276] 0709 12:26:10.096145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.747640 (+651495us) service_pool.cc:224] Handling call
0709 12:26:11.167537 (+419897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.193975 10249 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:26:11.184218  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 385110us
I20260709 12:26:11.184785  7737 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:11.186731  7627 connection.cc:256] RPC call timeout handler was delayed by -1.6636105880002106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260709 12:26:11.196082  7627 connection.cc:256] RPC call timeout handler was delayed by -1.6492105880001873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 12:26:11.196492  7627 connection.cc:256] RPC call timeout handler was delayed by -1.6258105880001494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
W20260709 12:26:11.196785  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5999946320002891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 12:26:11.197050  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5925105880000956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 12:26:11.197616  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5675946320002367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:26:11.198139  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5610105880000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 12:26:11.198207  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:11.198808  7627 connection.cc:256] RPC call timeout handler was delayed by -1.543910588000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 12:26:11.199011  7659 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:11.187609  8414 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:11.192006  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.136240 10289 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:11.165009 10292 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:11.245154  7702 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.525s (SENT) [suppressed 33 similar messages]
W20260709 12:26:11.204454  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5027488320001794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.225s for a response
W20260709 12:26:11.207917  7775 connection.cc:256] RPC call timeout handler was delayed by -1.7227716370002781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 12:26:11.248241  7775 connection.cc:256] RPC call timeout handler was delayed by -1.7159360089999609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 12:26:11.203776  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1892) took 1058 ms. Trace:
I20260709 12:26:11.249133  7731 rpcz_store.cc:276] 0709 12:26:10.145433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.144427 (+998994us) service_pool.cc:224] Handling call
0709 12:26:11.203610 (+ 59183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.205909  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:11.251719  7728 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.251878  7804 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:11.243640  7736 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:11.256078  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1891) took 1150 ms. Trace:
I20260709 12:26:11.256668  7736 rpcz_store.cc:276] 0709 12:26:10.105718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.191722 (+1086004us) service_pool.cc:224] Handling call
0709 12:26:11.255947 (+ 64225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.258256  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:11.262956  7775 connection.cc:256] RPC call timeout handler was delayed by -1.4367463970002063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
I20260709 12:26:11.265537  7809 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:26:11.265995 10290 tablet_replica.cc:333] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:26:11.266084 10317 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:11.267369 10290 raft_consensus.cc:2243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEARNER]: Raft consensus shutting down.
I20260709 12:26:11.268119 10290 raft_consensus.cc:2272] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEARNER]: Raft consensus is shut down!
W20260709 12:26:11.269403 10253 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 12:26:11.266877 10317 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.135510 10282 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.188666 10296 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:11.182777 10136 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.005s	sys 0.021s
I20260709 12:26:11.275023  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.276701  7805 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:11.206377 10035 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:11.278357  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.286839  7731 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:11.235307  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.272023  8415 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:11.269261 10182 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.293242  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1889) took 1190 ms. Trace:
I20260709 12:26:11.293280  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.265581  7806 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:11.295207  7809 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:11.294893  7731 rpcz_store.cc:276] 0709 12:26:10.102378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.250127 (+1147749us) service_pool.cc:224] Handling call
0709 12:26:11.293122 (+ 42995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.289623  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:11.299391  7736 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:11.304337  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 106972us
W20260709 12:26:11.305121  7702 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.733s (SENT)
I20260709 12:26:11.305583  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.228893  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:11.307850  7733 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:11.271641  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:11.248170  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1567237590002151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22740000000000002s for a response
I20260709 12:26:11.182221 10120 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:26:11.312219 10253 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:11.316309  7812 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:26:11.321382  9827 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:11.279313 10136 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:11.250696 10249 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:26:11.324155 10136 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.324836 10249 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:11.325440 10249 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:11.326659  9827 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:11.327643  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 590) took 2077 ms (client timeout 1571 ms). Trace:
W20260709 12:26:11.328203  8412 rpcz_store.cc:269] 0709 12:26:09.250583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.322701 (+ 72118us) service_pool.cc:224] Handling call
0709 12:26:11.235127 (+1912426us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfdeee 000000000036eb39 00007fc006e711e0 00007fc006f3a0c4 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:11.327535 (+ 92408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23674752}
I20260709 12:26:11.329054 10317 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:11.172609  8406 rpcz_store.cc:269] 0709 12:26:08.921642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.961683 (+ 40041us) service_pool.cc:224] Handling call
0709 12:26:10.743553 (+1781870us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:10.862708 (+119155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21551488}
W20260709 12:26:11.330520 10134 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 12:26:11.334379  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.336958  7729 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:26:11.305765  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.270931 10275 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.341486  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 69969.3us
I20260709 12:26:11.341440  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:11.350621  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1904) took 1100 ms. Trace:
I20260709 12:26:11.345405  7737 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:11.345369  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.351040  7728 rpcz_store.cc:276] 0709 12:26:10.250647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.205446 (+954799us) service_pool.cc:224] Handling call
0709 12:26:11.350544 (+145098us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.349084  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:11.353559  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.279866 10296 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:11.356817  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51721.2us
W20260709 12:26:11.359318  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53625.5us
I20260709 12:26:11.351548  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1895) took 1164 ms. Trace:
I20260709 12:26:11.361191  7737 rpcz_store.cc:276] 0709 12:26:10.187704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.174682 (+986978us) service_pool.cc:224] Handling call
0709 12:26:11.351478 (+176796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.362273  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.370096  7728 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 1.
I20260709 12:26:11.370203  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.340169 10024 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.372423  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.371661  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.348138  9827 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:11.375880  7728 raft_consensus.cc:2376] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3. Re-sending same reply.
I20260709 12:26:11.380121  7728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7112151d65374f6690a117439cc2dc33"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:26:11.380903  7728 raft_consensus.cc:606] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.381353  7728 raft_consensus.cc:614] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:11.371109  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:11.382830  7737 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:11.326514 10249 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.382964  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:11.369966 10133 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.371706 10134 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.418226  7736 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:11.370599 10182 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:11.370028 10088 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:11.389927  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 142636us
I20260709 12:26:11.387706  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:11.425442  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:11.370301 10147 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:11.363415 10296 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.428136  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.352134 10167 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:26:11.383880 10163 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:11.417367  7733 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:11.387132 10169 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:11.434415 10290 ts_tablet_manager.cc:1916] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:26:11.434324  9827 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:11.435863 10147 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:11.353679 10289 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.429755 10035 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:11.436487 10168 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.397650  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.434181 10051 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:11.379859 10024 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.444079 10317 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:11.445750  7730 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:11.661867  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1875) took 1570 ms. Trace:
W20260709 12:26:11.662287  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 229458us
I20260709 12:26:11.662472  7730 rpcz_store.cc:276] 0709 12:26:10.091529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.444760 (+1353231us) service_pool.cc:224] Handling call
0709 12:26:11.445229 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.661410 (+216181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.661778 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.448483  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2194) took 2570 ms (client timeout 1976 ms). Trace:
W20260709 12:26:11.664381  7735 rpcz_store.cc:269] 0709 12:26:08.878204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.207424 (+329220us) service_pool.cc:224] Handling call
0709 12:26:11.448118 (+2240694us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:11.448315 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645709184}
I20260709 12:26:11.452145  7731 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:11.452924 10242 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.002s
I20260709 12:26:11.430069 10249 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:11.453802 10282 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.431605 10198 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:11.356123 10275 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.467276  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1905) took 1216 ms. Trace:
I20260709 12:26:11.685963  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.686064  7733 rpcz_store.cc:276] 0709 12:26:10.250910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.228468 (+977558us) service_pool.cc:224] Handling call
0709 12:26:11.467140 (+238672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.420940 10318 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.470708 10147 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.421389 10310 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:11.690058  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 309995us
W20260709 12:26:11.480741  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83250.2us
I20260709 12:26:11.692759  7735 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:26:11.693501  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1881) took 1598 ms. Trace:
W20260709 12:26:11.485823  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.693847  7735 rpcz_store.cc:276] 0709 12:26:10.094992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.665654 (+1570662us) service_pool.cc:224] Handling call
0709 12:26:11.692263 (+ 26609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.693241 (+   978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.693440 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.695039  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.487834  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2178) took 2635 ms (client timeout 2515 ms). Trace:
I20260709 12:26:11.694072  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:11.697643  7732 rpcz_store.cc:269] 0709 12:26:08.852822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.853022 (+   200us) service_pool.cc:224] Handling call
0709 12:26:11.487525 (+2634503us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:11.487742 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083556096}
W20260709 12:26:11.492388  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1875) took 2779 ms (client timeout 1769 ms). Trace:
I20260709 12:26:11.511355  7653 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:11.698920  7811 rpcz_store.cc:269] 0709 12:26:08.712594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.713214 (+   620us) service_pool.cc:224] Handling call
0709 12:26:11.492076 (+2778862us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b5400133dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:11.492283 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147984256}
W20260709 12:26:11.514627 10256 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 1.757s	user 0.002s	sys 0.003s
W20260709 12:26:11.521606  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.705863  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.706350  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.526903  7728 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:11.528519 10300 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:11.530987  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.712772  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.713418  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.714507  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.531831 10258 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:26:11.636093  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.678017  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.472339 10289 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:11.438742 10192 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.726569  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.727545  7809 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:11.690450  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.733095  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:11.479498 10035 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.703006  7737 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:11.709240  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.711395  7732 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:11.735265  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1882) took 1640 ms. Trace:
I20260709 12:26:11.735809  7732 rpcz_store.cc:276] 0709 12:26:10.095291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.704828 (+1609537us) service_pool.cc:224] Handling call
0709 12:26:11.705279 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.734829 (+ 29550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.735170 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.715065  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.718532  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.738592  7811 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.724488  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.739462  7812 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:11.754207  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45064.3us
I20260709 12:26:11.762040  7809 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:11.684502 10237 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:11.755458 10248 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:26:11.766317 10248 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:11.766767 10248 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:11.767395 10168 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:11.768132 10168 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:11.768659 10168 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:11.769722 10168 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.772140 10167 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:11.774726  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:11.776355  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.776648 10136 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:11.777551 10136 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:11.778273 10136 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.779529  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2209) took 1032 ms. Trace:
I20260709 12:26:11.779960  7731 rpcz_store.cc:276] 0709 12:26:10.747177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.353125 (+605948us) service_pool.cc:224] Handling call
0709 12:26:11.779351 (+426226us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b54002490d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:11.779469 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222489472}
I20260709 12:26:11.782371  7808 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:11.782516 10192 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.788829  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 480) took 1683 ms. Trace:
I20260709 12:26:11.712193 10300 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:11.787837  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.717061 10198 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:11.789369  7732 rpcz_store.cc:276] 0709 12:26:10.105782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.736898 (+1631116us) service_pool.cc:224] Handling call
0709 12:26:11.788084 (+ 51186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.788314 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.788339 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.788387 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.788419 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.788474 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.788746 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.775741  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.802968  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.781772  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 481) took 1669 ms. Trace:
I20260709 12:26:11.796806  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.797823  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.717651 10258 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:11.716545 10310 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:11.684230 10242 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.806063  7731 rpcz_store.cc:276] 0709 12:26:10.112727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.780870 (+1668143us) service_pool.cc:224] Handling call
0709 12:26:11.781242 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.781419 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.781444 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.781484 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.781515 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.781564 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.781715 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.800310  7809 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:11.841594  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.843297 10275 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:11.843524 10312 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.802771 10222 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:11.846297  7732 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:11.847105  8409 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:11.847568  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.847158  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1893) took 1675 ms. Trace:
I20260709 12:26:11.848639  7732 rpcz_store.cc:276] 0709 12:26:10.172080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.845559 (+1673479us) service_pool.cc:224] Handling call
0709 12:26:11.846010 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.846835 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.847090 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.723829 10282 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 12:26:11.851300  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.851809 10260 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:11.852602 10260 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:11.853559 10260 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.776647 10115 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:11.855078  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.855867 10198 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.856669  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.858229  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.753934  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2077) took 1021 ms. Trace:
I20260709 12:26:11.859455  7653 rpcz_store.cc:276] 0709 12:26:10.729409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.510404 (+780995us) service_pool.cc:224] Handling call
0709 12:26:11.511023 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.704587 (+193564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.751144 (+ 46557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.860518  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:11.862136  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.836229  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:11.713052 10255 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:26:11.717020 10274 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:26:11.716900 10182 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:26:11.863436  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.863933  7809 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:11.706879 10318 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:11.865074  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.851747 10282 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:11.749339 10035 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:11.750355 10200 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:11.805401  7811 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:11.854923 10260 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:11.885830  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.887059  7812 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
W20260709 12:26:11.890901  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 354291us
W20260709 12:26:11.893787  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 267044us
W20260709 12:26:11.896240  7777 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.547s (SENT)
I20260709 12:26:11.897619  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:11.897786  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.898319  7808 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.901829  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:11.912372  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 206554us
I20260709 12:26:11.917091  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:11.924840  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.925630 10169 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:11.928985  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.893445 10088 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.892777 10182 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.920487  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.934504  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:11.888026 10282 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:11.908912 10309 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:11.944854  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.922930 10283 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:11.923465 10318 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:11.936501 10325 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:11.968163  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.968564  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:11.972982  7808 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:11.973915  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.976181  7812 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:11.979815  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:11.980788  7659 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:11.981408  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:11.981865  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:11.982146  7656 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:11.982424  7662 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:11.983824  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:11.912025 10248 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:11.978015  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:11.951885 10133 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 12:26:11.939029 10163 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:11.942250 10170 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:11.987582  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:11.938768 10200 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:11.979015  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:11.966527 10169 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:11.992060  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:12.013893  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:12.014436  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:12.014742  8411 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:12.015040  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:11.994104  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:11.946950 10312 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:26:11.988984  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:12.005240  7656 raft_consensus.cc:2410] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:12.005816  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:11.967676 10175 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:12.001571  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:11.975394 10325 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:12.024055  7662 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:11.991811 10318 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:12.017938  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:12.028607  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:12.031641  7812 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6.
W20260709 12:26:12.033382  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 327091us
I20260709 12:26:12.000348 10170 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:12.025177 10296 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
I20260709 12:26:12.016624  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:11.990926 10036 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:11.960855 10310 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.015172  7660 raft_consensus.cc:2376] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3. Re-sending same reply.
I20260709 12:26:11.950881 10182 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:12.030745  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 336717us
I20260709 12:26:12.017571  8412 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:12.032864 10318 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.070439 10318 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:12.070475  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.072127  7808 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:12.050290  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:12.072474  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:12.073122  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:12.055099 10136 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.074817  7808 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:26:11.948899 10258 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.075358 10088 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.076308  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:12.069065  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1876) took 1977 ms. Trace:
I20260709 12:26:12.071978  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.028945  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:12.017232 10253 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.016345 10192 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:12.054108  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:12.107824  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.080677  7662 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:12.075781 10258 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:12.120599  7736 rpcz_store.cc:276] 0709 12:26:10.091796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.523752 (+1431956us) service_pool.cc:224] Handling call
0709 12:26:11.524248 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.571848 (+ 47600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.571902 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.670597 (+ 98695us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.891130 (+220533us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:11.891196 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.891329 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.068771 (+177442us) raft_consensus.cc:1685] finished
0709 12:26:12.068803 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.068981 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46940,"raft.queue_time_us":47844,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":23211,"thread_start_us":171629,"threads_started":1,"wal-append.queue_time_us":330575}
I20260709 12:26:12.132726  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:12.053200 10254 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:12.134686 10192 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.051175 10325 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.059379 10283 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:12.121515 10036 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:12.069959 10334 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:12.127239  7808 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:12.027595  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:12.063740 10163 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.075981 10088 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.128355  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:12.128696 10336 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:12.032774 10200 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.048380 10328 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:26:12.250293 10200 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.175964 10192 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.255350 10200 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.255548 10192 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:12.257534 10325 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.131465 10253 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.136656 10182 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:12.069554 10242 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:12.060153 10248 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.066777 10282 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.124104 10296 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.133569  7736 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:12.139982 10340 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.144042  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.261807  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:12.262557  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.144270  7803 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:26:12.264901  7803 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:12.144894 10318 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:12.169638  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 283838us
I20260709 12:26:12.029812 10175 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:12.027756 10169 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:12.060148 10312 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:12.223915  7808 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:12.180604  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.129326 10289 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:12.235812  7624 leader_election.cc:336] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.100s (SENT)
W20260709 12:26:12.249851  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.131316 10051 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 12:26:12.252179 10088 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:12.268545  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 409) took 3545 ms. Trace:
I20260709 12:26:12.269863  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:12.276119  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.303781  7810 rpcz_store.cc:276] 0709 12:26:08.723114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.249113 (+525999us) service_pool.cc:224] Handling call
0709 12:26:10.349750 (+1100637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.709530 (+359780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.268342 (+1558812us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400067cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:12.268480 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2530188032}
W20260709 12:26:12.304030  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 298321us
W20260709 12:26:12.269871  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 197951us
I20260709 12:26:12.221906 10036 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:12.067615 10330 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.202s
I20260709 12:26:12.227190 10336 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:12.272765 10341 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:12.275151 10175 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:12.295815  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.013s	sys 0.009s
I20260709 12:26:12.296355  7808 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:26:12.298957  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63218.8us
W20260709 12:26:12.299465  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.301055  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 272517us
W20260709 12:26:12.304669  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.274519  7803 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:26:12.314077  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.014s	sys 0.009s
W20260709 12:26:12.315882  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.309039  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.309952  7702 leader_election.cc:400] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6.
I20260709 12:26:12.227432 10133 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.311368  8414 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:12.312744  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:12.295112  7736 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:26:12.315062  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.323576  7702 leader_election.cc:403] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:26:12.305843  8412 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:12.328199  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.328246  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:12.332746  7808 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:12.324970  7810 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:12.332944  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:12.329202  7734 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:12.310714 10331 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:26:12.334378  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.333137  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:12.334976  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.271656  8405 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:12.335337  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:12.307108 10289 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:12.310755 10147 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:26:12.337244  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.303138 10342 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:12.275368 10340 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.372545  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:12.373670  7736 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:12.441619  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:12.334851 10133 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.375836  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:12.375208 10270 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:26:12.419235  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45165.8us
I20260709 12:26:12.448412  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:12.451452  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75728.7us
I20260709 12:26:12.453229  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.330746 10309 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:12.313784 10341 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.376398  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:12.460888  7808 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:26:12.331161 10312 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:12.336395 10051 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:12.442726  7736 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:12.342787  7624 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:12.461894  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 146888us
I20260709 12:26:12.312904 10336 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:12.378228 10308 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 12:26:12.440305 10342 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:12.464700  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:12.478636  7736 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:12.458920  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119972us
I20260709 12:26:12.332573 10182 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.455418 10148 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:12.458071 10165 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:12.451303 10346 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:12.432597 10168 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 12:26:12.587980 10344 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:26:12.588512 10051 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:12.555943 10342 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.589043 10345 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:12.590116 10242 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:12.591408 10282 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.487355 10330 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:26:12.591898  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 112034us
I20260709 12:26:12.616814  7655 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:12.617722  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2078) took 1883 ms. Trace:
I20260709 12:26:12.592700  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1897) took 2386 ms. Trace:
I20260709 12:26:12.455771 10192 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 12:26:12.488142 10349 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:12.594620 10133 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.595935 10292 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.596074 10182 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6.
W20260709 12:26:12.597146  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2206) took 2867 ms (client timeout 2274 ms). Trace:
W20260709 12:26:12.599956  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 742) took 2180 ms (client timeout 2075 ms). Trace:
I20260709 12:26:12.602236  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1894) took 2418 ms. Trace:
I20260709 12:26:12.461483 10270 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:12.618662  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 170378us
W20260709 12:26:12.619763  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 284844us
I20260709 12:26:12.662232  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:12.662814  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:12.663183  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.618141  7655 rpcz_store.cc:276] 0709 12:26:10.734567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.616197 (+1881630us) service_pool.cc:224] Handling call
0709 12:26:12.616582 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.617430 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.617646 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.618927  7734 rpcz_store.cc:276] 0709 12:26:10.206251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.959197 (+1752946us) service_pool.cc:224] Handling call
0709 12:26:11.959939 (+   742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.334070 (+374131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.592378 (+258308us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:12.592611 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848750336}
I20260709 12:26:12.664593  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.664815  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:12.665470  7734 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:12.618445  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 165315us
I20260709 12:26:12.445529 10169 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:26:12.630151  7729 rpcz_store.cc:269] 0709 12:26:09.732968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.953956 (+220988us) service_pool.cc:224] Handling call
0709 12:26:12.596860 (+2642904us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400071cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:12.597050 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3176891648}
I20260709 12:26:12.630250  7736 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:12.660223  7732 rpcz_store.cc:276] 0709 12:26:10.184221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.849431 (+1665210us) service_pool.cc:224] Handling call
0709 12:26:11.849852 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.130375 (+280523us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:12.130419 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.130479 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:12.130523 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.601964 (+471441us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400072bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:12.602020 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.602166 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2550,"raft.queue_time_us":17559,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":638231424}
I20260709 12:26:12.668090  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.668402  7729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:45404
I20260709 12:26:12.669414  7729 raft_consensus.cc:606] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.581056 10334 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:12.583705 10353 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:12.608951  8406 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:26:12.618310 10296 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:12.610335 10345 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:12.697839  7732 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:26:12.698700  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81122.2us
I20260709 12:26:12.702245  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:12.704448  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1901) took 2454 ms. Trace:
I20260709 12:26:12.704561  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:12.704878  7732 rpcz_store.cc:276] 0709 12:26:10.249742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.696951 (+2447209us) service_pool.cc:224] Handling call
0709 12:26:12.697437 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.704079 (+  6642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.704368 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.611016 10351 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:26:12.706321  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 43586.7us
I20260709 12:26:12.699066  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1902) took 2449 ms. Trace:
I20260709 12:26:12.708616  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.708284  7729 rpcz_store.cc:276] 0709 12:26:10.250044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.698074 (+2448030us) service_pool.cc:224] Handling call
0709 12:26:12.698510 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.698726 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.698753 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.698802 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:12.698837 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.698891 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.699010 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84864}
I20260709 12:26:12.716518  7732 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:12.623198 10168 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:12.717530  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1907) took 2446 ms. Trace:
I20260709 12:26:12.723273  7729 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:12.724325  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.629920 10147 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.729277  7732 rpcz_store.cc:276] 0709 12:26:10.271469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.706004 (+2434535us) service_pool.cc:224] Handling call
0709 12:26:12.706554 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.717177 (+ 10623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.717436 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.736043  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.630131 10292 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:12.618759 10242 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:12.675685 10318 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:12.753063  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.741147  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:12.758505  7732 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:12.759828  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.700502 10330 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:12.734031 10051 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.622514 10282 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.718853 10349 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.609651 10289 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.716823 10182 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:12.729894 10133 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.729242 10168 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:12.769042 10351 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:12.603685 10341 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:12.661284  8405 rpcz_store.cc:269] 0709 12:26:10.419841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.794211 (+374370us) service_pool.cc:224] Handling call
0709 12:26:12.599699 (+1805488us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540013fbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:12.599864 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2602268544}
I20260709 12:26:12.776460 10165 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:12.781112 10330 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.791771 10168 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:12.800365 10358 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:26:12.812014 10348 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:26:12.812579 10351 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.813632 10357 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 12:26:12.812783 10347 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:12.815323 10354 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.191s
I20260709 12:26:12.811645 10355 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:12.817262 10355 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:12.817909 10355 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:12.818015 10349 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:12.817458 10352 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:26:12.814049 10368 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:12.804945 10340 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.792166 10133 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.810631 10356 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:12.814599 10147 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.842514 10356 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.842948 10356 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.827783 10136 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:12.818718 10170 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.850625 10170 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:12.814218 10351 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.852955 10290 ts_tablet_manager.cc:1929] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:26:12.853571 10290 log.cc:1199] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-3-root/wals/59f766c3d13d4360bcdeb1d5f0f318a8
I20260709 12:26:12.802661 10242 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.825564 10358 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.831761 10368 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.798545 10270 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.841205 10330 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.790741 10345 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.830746 10341 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:12.866297 10350 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:26:12.875887 10345 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.870440 10290 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57906 (request call id 0) took 2790 ms. Trace:
I20260709 12:26:12.888582 10282 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:12.860633 10334 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.848640 10355 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.899711 10368 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.903437 10358 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.907581 10336 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:12.910022 10336 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:12.910281 10342 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.916306 10342 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.918081 10356 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:12.920792 10356 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:12.963683  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 297650us
I20260709 12:26:12.968075  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 483) took 2633 ms. Trace:
W20260709 12:26:12.969182  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 260462us
W20260709 12:26:12.969467  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 264822us
I20260709 12:26:12.970992  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.973887  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:12.969024  7732 rpcz_store.cc:276] 0709 12:26:10.334116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.757711 (+2423595us) service_pool.cc:224] Handling call
0709 12:26:12.758312 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.966911 (+208599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.967605 (+   694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.979652  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:12.975445  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:12.981022  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 485) took 2557 ms. Trace:
I20260709 12:26:12.981875  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2341) took 1302 ms. Trace:
I20260709 12:26:12.837708 10133 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:12.981689  7732 rpcz_store.cc:276] 0709 12:26:10.424090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.978156 (+2554066us) service_pool.cc:224] Handling call
0709 12:26:12.979070 (+   914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.980572 (+  1502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.980920 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.996744  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.985205  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2347) took 1288 ms. Trace:
W20260709 12:26:12.985268  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.987807  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.988251  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:13.024678 10246 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.130s	user 0.023s	sys 0.021s
I20260709 12:26:13.025107 10339 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:12.993304 10136 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:13.025296 10371 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:26:12.925114 10353 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:12.888640 10290 rpcz_store.cc:276] 0709 12:26:10.093480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.111458 (+ 17978us) service_pool.cc:224] Handling call
0709 12:26:10.346685 (+235227us) ts_tablet_manager.cc:1151] Deleting tablet 59f766c3d13d4360bcdeb1d5f0f318a8
0709 12:26:11.249152 (+902467us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:26:12.256376 (+1007224us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:12.852790 (+596414us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:12.864956 (+ 12166us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":165148,"thread_start_us":66497,"threads_started":1}
I20260709 12:26:12.893049 10346 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:13.031744 10353 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.033061 10346 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:13.037791  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 335497us
W20260709 12:26:13.025415 10246 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.147s	user 0.023s	sys 0.021s
W20260709 12:26:13.039678  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73759.4us
I20260709 12:26:13.040349 10163 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:13.041903 10163 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:13.044027 10336 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.043918 10372 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:13.044740 10336 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.046054 10341 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:13.049537 10334 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.051587 10345 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.052047 10370 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:13.054687  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66871.5us
I20260709 12:26:12.983425  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:13.024173 10375 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:13.079610 10136 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:13.081914 10136 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.085897 10345 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.065230 10336 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.090674 10165 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.097678  7656 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:13.099892 10372 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:13.100733 10372 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:13.102631 10325 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.844s	user 0.009s	sys 0.012s
I20260709 12:26:13.105003 10372 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.108247 10246 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:13.108839 10341 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:13.089412  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:13.123546  7736 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:12.989387 10242 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.089529 10355 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.121196 10341 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:12.982637  7806 rpcz_store.cc:276] 0709 12:26:11.679175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.681048 (+  1873us) service_pool.cc:224] Handling call
0709 12:26:11.681562 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.688074 (+  6512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.688142 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.701304 (+ 13162us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.316161 (+614857us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:12.316203 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.316283 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.981050 (+664767us) raft_consensus.cc:1685] finished
0709 12:26:12.981097 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.981552 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6921,"raft.queue_time_us":6818,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":19699,"thread_start_us":526349,"threads_started":1,"wal-append.queue_time_us":707160}
I20260709 12:26:13.020694  7812 rpcz_store.cc:276] 0709 12:26:11.697139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.077353 (+380214us) service_pool.cc:224] Handling call
0709 12:26:12.151507 (+ 74154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.441222 (+289715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.441267 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.450419 (+  9152us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.756801 (+306382us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:12.756839 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.756913 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.984433 (+227520us) raft_consensus.cc:1685] finished
0709 12:26:12.984492 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.984949 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105392,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":65,"thread_start_us":461694,"threads_started":2,"wal-append.queue_time_us":344503}
I20260709 12:26:13.137617  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1900) took 2884 ms. Trace:
I20260709 12:26:13.125996 10375 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:13.146047  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.138386 10325 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:26:13.154011 10258 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.977s	user 0.013s	sys 0.004s
I20260709 12:26:13.155509 10345 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.159502 10282 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:13.164119  7731 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:13.143904  7736 rpcz_store.cc:276] 0709 12:26:10.249396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.697001 (+2447605us) service_pool.cc:224] Handling call
0709 12:26:13.116252 (+419251us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:13.116795 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.129282 (+ 12487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.133978 (+  4696us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462080}
I20260709 12:26:13.175429  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 488) took 1976 ms. Trace:
I20260709 12:26:13.177140 10372 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.179461 10349 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:13.186854  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97630.7us
I20260709 12:26:13.162645 10325 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:13.182796  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 479) took 3335 ms. Trace:
I20260709 12:26:13.176105  7731 rpcz_store.cc:276] 0709 12:26:11.195567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.106159 (+1910592us) service_pool.cc:224] Handling call
0709 12:26:13.106778 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.168968 (+ 62190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.172018 (+  3050us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:13.187450  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65853.3us
I20260709 12:26:13.205264  7737 rpcz_store.cc:276] 0709 12:26:09.847104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.421832 (+1574728us) service_pool.cc:224] Handling call
0709 12:26:11.456801 (+ 34969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.734661 (+277860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.181418 (+1446757us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b540037c654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:13.182190 (+   772us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b540037c650. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.182521 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592952064}
I20260709 12:26:13.253396  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 482) took 2856 ms. Trace:
I20260709 12:26:13.254062  7729 rpcz_store.cc:276] 0709 12:26:10.305022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.711038 (+2406016us) service_pool.cc:224] Handling call
0709 12:26:12.711882 (+   844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.734242 (+ 22360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.161206 (+426964us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b54000f94a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f25a02 00007fc001f21dd6 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:26:13.161617 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10827136}
I20260709 12:26:13.257104  7729 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:13.256958 10381 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 12:26:13.258894  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1879) took 4544 ms (client timeout 4688 ms). Trace:
I20260709 12:26:13.262599  7731 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:26:13.262789  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:13.260648  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 277524us
I20260709 12:26:13.268203  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 486) took 2842 ms. Trace:
I20260709 12:26:13.268781  7732 rpcz_store.cc:276] 0709 12:26:10.425916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.064203 (+2638287us) service_pool.cc:224] Handling call
0709 12:26:13.149223 (+ 85020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.246237 (+ 97014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.246293 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.249301 (+  3008us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:13.267904 (+ 18603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.267996 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.268117 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.271178  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 490) took 1531 ms. Trace:
I20260709 12:26:13.271795  7729 rpcz_store.cc:276] 0709 12:26:11.736198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.255862 (+1519664us) service_pool.cc:224] Handling call
0709 12:26:13.256596 (+   734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.263837 (+  7241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.267470 (+  3633us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:13.274305 10036 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.948s	user 0.006s	sys 0.017s
I20260709 12:26:13.275043 10036 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:13.264684 10249 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.457s	user 0.017s	sys 0.006s
I20260709 12:26:13.282357  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 491) took 1501 ms. Trace:
I20260709 12:26:13.282110 10341 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:13.259667  7807 rpcz_store.cc:269] 0709 12:26:08.714008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.754037 (+ 40029us) service_pool.cc:224] Handling call
0709 12:26:13.258119 (+4504082us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.258441 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357576576}
I20260709 12:26:13.260167  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 504) took 2493 ms. Trace:
I20260709 12:26:13.281246  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 506) took 1483 ms. Trace:
I20260709 12:26:13.290351  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 596) took 1435 ms. Trace:
I20260709 12:26:13.290091  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.292887 10024 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.231985  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:13.285491  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 489) took 1535 ms. Trace:
I20260709 12:26:13.301646  7731 rpcz_store.cc:276] 0709 12:26:11.735786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.226751 (+1490965us) service_pool.cc:224] Handling call
0709 12:26:13.262202 (+ 35451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.270622 (+  8420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.270898 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.298738  7732 rpcz_store.cc:276] 0709 12:26:11.780829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.280777 (+1499948us) service_pool.cc:224] Handling call
0709 12:26:13.281484 (+   707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.281894 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.281932 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.282010 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:13.282053 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.282113 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.282254 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.299777  8409 rpcz_store.cc:276] 0709 12:26:11.797340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.802606 (+  5266us) service_pool.cc:224] Handling call
0709 12:26:13.280591 (+1477985us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.280947 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98652416}
I20260709 12:26:13.308442 10341 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.259143 10381 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:13.311537 10381 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:13.313254 10381 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.315659 10334 raft_consensus.cc:2764] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 12:26:13.316009  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72765.5us
I20260709 12:26:13.317811  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:13.318836 10334 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:13.322080  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:13.323357 10370 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:13.324138  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:13.324108 10370 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:13.326200 10175 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.992s	user 0.022s	sys 0.003s
I20260709 12:26:13.325832 10370 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.326908 10175 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.327489 10282 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:13.329120 10133 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.329874 10133 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.330406  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:13.330859  7809 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:13.331516  8405 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.327581 10375 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.260259 10325 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.330657  7736 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:13.300282  8414 rpcz_store.cc:276] 0709 12:26:11.854562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.859095 (+  4533us) service_pool.cc:224] Handling call
0709 12:26:13.289850 (+1430755us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.290117 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029795584}
I20260709 12:26:13.326143  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:13.334142 10325 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:13.344457  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.336290 10282 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.299642  7661 rpcz_store.cc:276] 0709 12:26:10.713651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.495207 (+781556us) service_pool.cc:224] Handling call
0709 12:26:11.528434 (+ 33227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.528933 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.528980 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.529741 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.900666 (+370925us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:12.150291 (+249625us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.158510 (+1008219us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b18003cb4a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:26:13.158914 (+   404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.159040 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.159106 (+    66us) raft_consensus.cc:1685] finished
0709 12:26:13.169305 (+ 10199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.206789 (+ 37484us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":410504,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":400482,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":5251,"spinlock_wait_cycles":90616704,"thread_start_us":760310,"threads_started":3,"wal-append.queue_time_us":493407}
I20260709 12:26:13.345538  7807 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:13.346268 10345 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:13.347122  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:13.367837  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109424us
I20260709 12:26:13.368690 10036 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.373225  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 492) took 1513 ms. Trace:
I20260709 12:26:13.374042  7729 rpcz_store.cc:276] 0709 12:26:11.860281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.283790 (+1423509us) service_pool.cc:224] Handling call
0709 12:26:13.342859 (+ 59069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.372615 (+ 29756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.372651 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.372761 (+   110us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:13.372823 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.372911 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.373125 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.373255  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.354368 10381 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.389297  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:13.391305  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:13.396077 10282 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.398036  7732 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:13.399042  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1911) took 1246 ms. Trace:
I20260709 12:26:13.399263 10170 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:13.399621  7732 rpcz_store.cc:276] 0709 12:26:12.152155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.343281 (+1191126us) service_pool.cc:224] Handling call
0709 12:26:13.397383 (+ 54102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.398623 (+  1240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.398933 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.384070 10175 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.409996  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:13.410351 10168 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.012s
W20260709 12:26:13.411072 10292 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.015s
I20260709 12:26:13.410353  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:13.411840 10292 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.415896  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.416354  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 587) took 4196 ms. Trace:
I20260709 12:26:13.417302  7655 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:13.418053 10292 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.411038 10168 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.420790 10024 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:13.421310 10024 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:13.421599 10168 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.422487 10168 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.423374 10035 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.468s	user 0.012s	sys 0.012s
I20260709 12:26:13.424067  7731 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:13.425747 10308 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:13.426375 10308 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:13.416824  8411 rpcz_store.cc:276] 0709 12:26:09.220092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.250526 (+ 30434us) service_pool.cc:224] Handling call
0709 12:26:09.250916 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.015188 (+2764272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.415646 (+1400458us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54007a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:13.416002 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6355411840}
I20260709 12:26:13.414947  7809 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:13.429219  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.418864  7655 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:13.431208  7656 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.433115  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:13.433568 10309 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:13.434250 10309 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.434302  7803 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:26:13.434950  7803 pending_rounds.cc:85] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 1
I20260709 12:26:13.441990  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53562 (request call id 10) took 2715 ms. Trace:
I20260709 12:26:13.441992 10273 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.424000 10035 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.449620  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:13.444459  7732 rpcz_store.cc:276] 0709 12:26:10.726958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.403763 (+2676805us) service_pool.cc:224] Handling call
0709 12:26:13.441860 (+ 38097us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.456455  7655 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:13.457437  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.426627 10336 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.458384  7729 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:13.435180  7803 pending_rounds.cc:107] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:26:13.465153  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1914) took 1312 ms. Trace:
I20260709 12:26:13.466889  7729 rpcz_store.cc:276] 0709 12:26:12.153174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.386447 (+1233273us) service_pool.cc:224] Handling call
0709 12:26:13.431023 (+ 44576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.464770 (+ 33747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.465067 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.456557 10367 raft_consensus.cc:2764] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:26:13.468751 10321 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:13.469375 10321 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:13.463667  7803 raft_consensus.cc:2889] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.422614 10024 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:13.474243  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.423122 10168 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:13.480032 10292 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.479945 10321 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.480878 10292 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.484637 10292 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.485563 10321 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.489708 10308 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.492463 10308 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:13.493750 10378 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.565s	user 0.002s	sys 0.000s
I20260709 12:26:13.496021 10024 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:26:13.497804 10374 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.570s	user 0.000s	sys 0.001s
I20260709 12:26:13.490535  7730 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:13.498760  7730 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:13.477053 10273 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.506342 10273 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.509450 10273 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.510298 10253 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.479198 10282 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.527685  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:13.530097  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:13.533747  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1874) took 3442 ms. Trace:
I20260709 12:26:13.534294  7728 rpcz_store.cc:276] 0709 12:26:10.091175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.420731 (+1329556us) service_pool.cc:224] Handling call
0709 12:26:11.421085 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.707095 (+286010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.533086 (+1825991us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400212854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:13.533500 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242678656}
W20260709 12:26:13.535938 10377 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.512s	user 0.003s	sys 0.000s
I20260709 12:26:13.505127 10370 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.497864  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:13.541957  7655 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:26:13.523025 10330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.000s	sys 0.009s
W20260709 12:26:13.544755 10330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.001s	sys 0.009s
W20260709 12:26:13.549863  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 185383us
I20260709 12:26:13.591374  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.584462  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:13.592361 10168 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.583329  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:13.593838  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:13.595570 10148 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.596431 10148 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.577749  7812 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:13.549362  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:13.594003  7728 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:13.591470  7809 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:13.583916  7732 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:13.592867  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2096) took 1594 ms. Trace:
W20260709 12:26:13.608716  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.600513  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:13.606583  7654 rpcz_store.cc:276] 0709 12:26:11.998040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.017868 (+ 19828us) service_pool.cc:224] Handling call
0709 12:26:13.537499 (+1519631us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:13.592775 (+ 55276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10130816}
I20260709 12:26:13.600418 10035 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.616560  7803 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:13.609620 10367 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:26:13.619788 10372 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:13.578739  7730 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.583415  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:13.599014 10148 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.627027  7656 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:13.635725  7655 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:13.636862  7728 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:13.644153  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43832.1us
I20260709 12:26:13.610180 10370 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.645459  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:13.646088  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 592) took 1792 ms. Trace:
I20260709 12:26:13.646641  8408 rpcz_store.cc:276] 0709 12:26:11.853424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.854379 (+   955us) service_pool.cc:224] Handling call
0709 12:26:13.286975 (+1432596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.287587 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:13.363056 (+ 75469us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.504788 (+141732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.504850 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:13.504900 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.514760 (+  9860us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.645691 (+130931us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400320ad0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.645846 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":101004,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":4475,"raft.queue_time_us":6246,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":150,"spinlock_wait_cycles":3287493888,"thread_start_us":162901,"threads_started":2,"wal-append.queue_time_us":238173}
I20260709 12:26:13.651557  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.658838  7730 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:13.662189  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:13.664113 10404 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:26:13.667788  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:13.673417  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.678328  7660 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:13.647024 10370 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.678905 10341 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:13.684347  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.685508  7655 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:13.687672  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:13.691396 10361 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:13.692031 10282 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:13.692358 10361 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:13.692804 10282 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.694020 10361 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.691488 10035 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:13.667891 10336 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.700218  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:13.699134 10035 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.703714 10035 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:13.707718 10282 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:13.710706  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1884) took 3614 ms. Trace:
I20260709 12:26:13.680064 10330 raft_consensus.cc:2764] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:26:13.711717  7730 rpcz_store.cc:276] 0709 12:26:10.095871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.713466 (+1617595us) service_pool.cc:224] Handling call
0709 12:26:11.750615 (+ 37149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.490240 (+1739625us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540001d220. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f241c0 00007fc001f25231 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c
0709 12:26:13.707057 (+216817us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:13.707088 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.707135 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:13.707171 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.710469 (+  3298us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.710621 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":146,"spinlock_wait_cycles":261797632}
W20260709 12:26:13.714279  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.722234  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:13.724295  7728 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:13.719390  7654 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
W20260709 12:26:13.718688 10306 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:26:13.682701 10175 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:26:13.696506 10346 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:26:13.748533  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.750000  7657 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:13.731789 10365 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:26:13.750993  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:13.732818 10242 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:13.735364  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:13.752688  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.753666  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:13.754601  7730 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:13.750550  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2095) took 1753 ms (client timeout 2025 ms). Trace:
W20260709 12:26:13.755529  7624 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.990s (SENT)
W20260709 12:26:13.755995  7657 rpcz_store.cc:269] 0709 12:26:11.997666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.019113 (+ 21447us) service_pool.cc:224] Handling call
0709 12:26:13.748185 (+1729072us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:13.750465 (+  2280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7491456}
W20260709 12:26:13.739760 10399 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 12:26:13.740787 10380 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.853s	user 0.003s	sys 0.000s
I20260709 12:26:13.721196 10168 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:13.745194  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.745918  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:13.767997  8415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 741) took 3348 ms (client timeout 3033 ms). Trace:
I20260709 12:26:13.746870 10336 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:13.762903  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:13.761106 10325 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:26:13.747714  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.775072  7735 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:13.761489  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:13.767867  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:13.770937  8415 rpcz_store.cc:269] 0709 12:26:10.419468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.159183 (+739715us) service_pool.cc:224] Handling call
0709 12:26:13.767499 (+2608316us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540046d7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:13.767773 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247405824}
I20260709 12:26:13.780294  7730 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:13.781514  7729 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:13.782481  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93021.3us
I20260709 12:26:13.784766  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:13.786183  7730 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:13.787670 10336 consensus_queue.cc:1243] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:13.784539 10168 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.788882 10270 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.789508 10270 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.790549  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 494) took 1642 ms. Trace:
I20260709 12:26:13.774142 10375 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.794378  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.791028  7731 rpcz_store.cc:276] 0709 12:26:12.147967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.336694 (+1188727us) service_pool.cc:224] Handling call
0709 12:26:13.372628 (+ 35934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.424613 (+ 51985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.790225 (+365612us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b54001450d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:13.790472 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117652352}
W20260709 12:26:13.787750  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2097) took 1788 ms (client timeout 2311 ms). Trace:
I20260709 12:26:13.775363 10308 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.797511  7659 rpcz_store.cc:269] 0709 12:26:12.000876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.019887 (+ 19011us) service_pool.cc:224] Handling call
0709 12:26:13.747292 (+1727405us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:13.787677 (+ 40385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6023040}
I20260709 12:26:13.801678  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.806038  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:13.807263  8408 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:13.782204 10249 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.811997 10249 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.815408  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:13.815765  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69997.4us
I20260709 12:26:13.815979 10270 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.818267 10200 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:13.819873  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.820235 10200 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:13.820770  7659 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:13.818574  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:13.822592 10381 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:13.781700 10309 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:13.823473 10381 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:13.824251 10381 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.826109 10309 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.826737 10381 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:13.828393  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:13.823590  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:13.835361  7807 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:13.839167  7656 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:13.841297  7656 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:26:13.841743  7607 auto_rebalancer.cc:629] Failed to schedule move for tablet a3d64be480d9430189f5e290bab7c3b8: Illegal state: Replica fc963afbbf4c4b34b161448ef79bd2ed is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:13.842984  7728 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:13.844651  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.845660  7658 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:13.846182  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2093) took 1851 ms (client timeout 1958 ms). Trace:
W20260709 12:26:13.846639  7658 rpcz_store.cc:269] 0709 12:26:11.995265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.996069 (+   804us) service_pool.cc:224] Handling call
0709 12:26:13.844316 (+1848247us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:13.846103 (+  1787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10113920}
I20260709 12:26:13.848696 10266 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:13.826951 10309 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:13.845664  7656 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:26:13.851969  7659 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.849260 10266 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:13.874775  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:13.839911  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:13.884835  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56980.2us
I20260709 12:26:13.890468  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:13.815037 10249 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:13.782506 10321 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:26:13.887524  7729 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:13.894951  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:13.899525  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.901990  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:13.908257 10198 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:13.877126 10309 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.912432 10295 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:13.915593  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:13.916170  7775 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:13.799095 10375 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.916960 10349 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:13.874228 10133 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:13.917500 10402 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:13.960520  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:13.917690 10360 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:13.918242  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.917011  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77235us
W20260709 12:26:13.922089  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.927208  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 69321us
I20260709 12:26:13.947855  7728 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:13.883982 10308 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.970244  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:13.922906 10249 raft_consensus.cc:2764] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:26:13.971599  7729 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:13.925976 10309 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:13.972847  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:13.974764  7627 leader_election.cc:400] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:13.975548  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:13.976171  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:13.991021  7659 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:13.919289 10266 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:13.985149 10024 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:13.991021  7730 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:13.991138  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.004220  8411 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:13.928421 10198 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:13.963910 10336 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:14.007380  7728 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:13.996699 10338 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:13.997201 10360 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:13.993007 10375 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:14.015055  7729 pending_rounds.cc:85] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 2
I20260709 12:26:14.024971  7729 pending_rounds.cc:107] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:26:14.025616  7729 raft_consensus.cc:2889] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:14.045644  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.024410  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:14.102669  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86777.3us
W20260709 12:26:14.056905  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 43577.2us
W20260709 12:26:14.117822  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.996340 10035 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.015208  8405 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:14.128582  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 154062us
I20260709 12:26:14.020253 10175 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:14.127121  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:14.133760  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:14.135501  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 248683us
I20260709 12:26:14.017968 10024 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:14.018337 10360 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:14.003937 10349 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:26:14.071408 10266 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:14.024089 10336 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:14.142869  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 381713us
W20260709 12:26:14.150861  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47297.2us
W20260709 12:26:14.158728  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.159965  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:14.127804 10273 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.143183 10175 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:14.138672 10242 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:14.165719 10198 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.175637 10242 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.176256 10175 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.178774 10242 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:14.155851 10365 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:14.137055 10170 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:26:14.179419 10172 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:14.196797 10172 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.201974 10172 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:14.203774  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67620.6us
I20260709 12:26:14.206354 10369 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:14.207454 10369 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:14.208972 10369 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.219456 10410 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.226846 10321 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:14.227712 10282 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.228026 10416 ts_tablet_manager.cc:933] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:26:14.228987 10321 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:14.228533 10282 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:14.229316  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85149.9us
I20260709 12:26:14.229470 10321 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:14.230448  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:14.230831 10321 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.231593 10282 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:14.231487 10024 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.232506 10321 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:14.232764 10282 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:14.235446 10273 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.238354 10273 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:14.227995 10172 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.244084 10336 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:14.228525 10408 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.205s	user 0.001s	sys 0.001s
I20260709 12:26:14.181715 10200 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.264909  7728 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:14.266399  7728 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:26:14.268057  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:14.279151 10361 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:14.279812 10308 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:14.279754  7728 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:14.282284  7728 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:14.283982  7728 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:26:14.284754 10300 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.285571  7728 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:14.286511 10300 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:14.254441 10365 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:14.321590 10358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.415s	user 0.024s	sys 0.018s
W20260709 12:26:14.322340 10358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.416s	user 0.024s	sys 0.018s
W20260709 12:26:14.339668  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 140615us
I20260709 12:26:14.343904 10172 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:14.319129 10360 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.346521 10360 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:14.343923 10413 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:14.354003 10359 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:14.354599 10359 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:14.357225  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.381632  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.389240  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.394027  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:14.355794 10359 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.401862 10172 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:14.407483 10035 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:14.435410  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 205084us
I20260709 12:26:14.404153 10416 tablet_copy_client.cc:323] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.193:43519
I20260709 12:26:14.416815 10172 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:14.516623  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:14.517611  7735 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:14.533619  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.546629  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.519093 10172 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:14.571347  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2359) took 1166 ms. Trace:
W20260709 12:26:14.571699 10420 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:26:14.571977  7807 rpcz_store.cc:276] 0709 12:26:13.404982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.405156 (+   174us) service_pool.cc:224] Handling call
0709 12:26:14.570982 (+1165826us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54003c55d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:14.571194 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863655680}
I20260709 12:26:14.431771 10365 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:14.595836 10421 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.158s	user 0.001s	sys 0.001s
W20260709 12:26:14.607412 10169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.619s	user 0.016s	sys 0.036s
W20260709 12:26:14.614981  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 243923us
I20260709 12:26:14.632005  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 591) took 2747 ms. Trace:
I20260709 12:26:14.632406  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:14.632570  8406 rpcz_store.cc:276] 0709 12:26:11.853098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.877906 (+ 24808us) service_pool.cc:224] Handling call
0709 12:26:12.608545 (+730639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.699665 (+ 91120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.595541 (+1895876us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b540013fbd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:14.597341 (+  1800us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569142656}
I20260709 12:26:14.639660 10358 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:14.641763 10358 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:14.642652 10358 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.645979 10358 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:14.648578 10295 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.006s
I20260709 12:26:14.650857 10270 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:14.651573 10270 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:14.653184 10270 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.655057 10270 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:14.610667  7735 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:14.658631  8413 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:14.605995 10359 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:14.675949 10169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.643s	user 0.016s	sys 0.037s
I20260709 12:26:14.684147  7737 pending_rounds.cc:85] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 2
I20260709 12:26:14.688293  7737 pending_rounds.cc:107] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:26:14.688794  7737 raft_consensus.cc:2889] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.674328 10292 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.697201  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.698402  7806 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:14.698875  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.699819  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.701076 10286 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:14.701625 10286 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:14.704535  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.705116  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.705749  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.701449  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:14.711390  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1692) took 1597 ms (client timeout 1685 ms). Trace:
I20260709 12:26:14.714251  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:14.711896  7660 rpcz_store.cc:269] 0709 12:26:13.113907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.350843 (+236936us) service_pool.cc:224] Handling call
0709 12:26:14.711150 (+1360307us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b5400397bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:14.711314 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101430656}
I20260709 12:26:14.717065  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:14.691059 10035 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:14.690958 10024 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.692803 10198 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.724774  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:14.713372 10266 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.004s
I20260709 12:26:14.728689  7804 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:14.729311  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 504) took 2932 ms (client timeout 1654 ms). Trace:
W20260709 12:26:14.729871  8413 rpcz_store.cc:269] 0709 12:26:11.796601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.796739 (+   138us) service_pool.cc:224] Handling call
0709 12:26:14.729026 (+2932287us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:14.729212 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6427352832}
I20260709 12:26:14.730480  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2349) took 3008 ms. Trace:
I20260709 12:26:14.727133 10024 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.740392  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:14.730944  7804 rpcz_store.cc:276] 0709 12:26:11.721775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.127825 (+406050us) service_pool.cc:224] Handling call
0709 12:26:12.128157 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.729318 (+2601161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.730095 (+   777us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b5400373cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:14.730365 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5931677952}
I20260709 12:26:14.743508  8411 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:14.702803 10286 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.731494  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.712327 10292 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:14.754500  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:14.739357 10425 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:14.743484 10024 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:14.749853 10242 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:14.754900  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.746297  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:14.745832 10172 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:14.765765  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.769582  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.732342 10198 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.774760  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:14.775069  7732 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 2.
I20260709 12:26:14.741305 10035 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:14.774292  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.771314  7735 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:14.747401 10360 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:14.776332  8413 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:14.778208  8408 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:26:14.780376 10198 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:14.769245 10289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.651s	user 0.024s	sys 0.020s
I20260709 12:26:14.787952  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:14.765089 10242 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:14.795415  7804 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:14.795809  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:14.797319 10289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.768s	user 0.025s	sys 0.020s
I20260709 12:26:14.794683  8406 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:14.804193  7734 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:14.800978 10242 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:14.810143  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.815852  7654 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:14.809650 10172 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:14.819207  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:14.822204  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:14.801460  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.788825 10425 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:14.832473  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:14.804491  8405 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:14.835491  7655 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:14.840458  8409 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:14.844403  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:14.810834 10286 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:14.818162  8406 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 12:26:14.859645  7732 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:14.860818  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.825193 10198 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.866029  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:14.863616  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.859174  8406 pending_rounds.cc:85] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 4
I20260709 12:26:14.816852 10360 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.869902  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.879586  7660 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:14.873401  7804 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:14.857972 10336 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:14.872191  8406 pending_rounds.cc:107] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:26:14.893440  8409 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:14.895140 10168 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:14.901594  8411 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:14.901742 10035 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:14.903126  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.903177  7807 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:14.923887  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:14.925118  7732 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:14.927099  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:14.911952 10309 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.005s
W20260709 12:26:14.912544 10300 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.581s	user 0.015s	sys 0.000s
I20260709 12:26:14.934859  7803 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:14.933746  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:14.931348  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:14.921330 10434 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:26:14.940994  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.942642  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:14.915560  8406 raft_consensus.cc:2889] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:14.950565  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.950973  7809 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:14.952557  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53854.2us
I20260709 12:26:14.950860  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:14.929268 10369 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:26:14.959798 10370 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:14.960515 10370 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:14.962849  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75787.1us
I20260709 12:26:14.950734 10365 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:26:14.931145 10168 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:14.969491  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:14.967541  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:14.970376  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46638.5us
I20260709 12:26:14.947924 10289 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:14.904024 10198 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:14.948637 10425 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:14.968262 10365 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.005453  7807 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:14.941030 10035 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:26:15.011752  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:14.998804 10429 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 12:26:14.948745 10309 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:15.000855 10428 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
I20260709 12:26:15.023108  7803 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:15.003679 10425 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.951414 10175 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:26:15.026782  7804 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:15.026212  7811 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 4. Ops: []
I20260709 12:26:15.001438 10295 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.009902 10426 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
W20260709 12:26:15.031967  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.029984 10425 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.040786 10088 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:15.044816  7777 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.516s (SENT)
W20260709 12:26:15.040525  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.042055  7807 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 12:26:15.047459  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82066.7us
I20260709 12:26:15.050716  7806 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 2.
I20260709 12:26:15.051782  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1942) took 1168 ms. Trace:
I20260709 12:26:15.052227  7729 rpcz_store.cc:276] 0709 12:26:13.884668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.620419 (+735751us) service_pool.cc:224] Handling call
0709 12:26:14.620948 (+   529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.621487 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.621518 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.622211 (+   693us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.852032 (+229821us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:14.852085 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.852191 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.051553 (+199362us) raft_consensus.cc:1685] finished
0709 12:26:15.051572 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.051716 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":229063,"threads_started":1,"wal-append.queue_time_us":344997}
I20260709 12:26:15.054906  7661 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:26:15.040963 10368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.003s	user 0.009s	sys 0.034s
I20260709 12:26:15.058301  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:15.022114 10242 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.071732  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1899) took 4778 ms. Trace:
W20260709 12:26:15.062726 10368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.160s	user 0.011s	sys 0.034s
W20260709 12:26:15.076292 10435 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:26:15.076496 10360 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.078245 10198 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:14.172647  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:15.086758  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1941) took 1250 ms. Trace:
W20260709 12:26:15.061743 10431 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
W20260709 12:26:15.102110  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57333us
W20260709 12:26:15.104336  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79194.4us
I20260709 12:26:15.104485  7805 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:15.106539  7558 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 302) took 12862 ms (client timeout 15000 ms). Trace:
I20260709 12:26:15.107681 10168 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:15.108438 10168 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:15.108704  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:15.043771 10295 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.108997 10168 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.112143  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:15.116866  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 174388us
I20260709 12:26:13.256136  7543 catalog_manager.cc:5028] TS 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): tablet 59f766c3d13d4360bcdeb1d5f0f318a8 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:26:15.097175  7731 rpcz_store.cc:276] 0709 12:26:13.836118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.343239 (+507121us) service_pool.cc:224] Handling call
0709 12:26:14.343767 (+   528us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:14.344273 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:14.344642 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.438703 (+ 94061us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:14.681655 (+242952us) log.cc:844] Serialized 52 byte log entry
0709 12:26:14.681918 (+   263us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:14.939718 (+257800us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.940100 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.940213 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.080609 (+140396us) raft_consensus.cc:1685] finished
0709 12:26:15.080629 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.086633 (+  6004us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7368,"prepare.queue_time_us":44951,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":133,"raft.queue_time_us":10002,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":36435,"thread_start_us":342764,"threads_started":2,"wal-append.queue_time_us":370987}
I20260709 12:26:15.083540  7733 rpcz_store.cc:276] 0709 12:26:10.248963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.051910 (+1802947us) service_pool.cc:224] Handling call
0709 12:26:12.597674 (+545764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.597940 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:12.598329 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.961233 (+362904us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.961432 (+   199us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:12.961631 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.961857 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.020833 (+2058976us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400071cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:15.026754 (+  5921us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26331,"prepare.queue_time_us":725,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":43088,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":23755,"spinlock_wait_cycles":1254298368,"thread_start_us":361622,"threads_started":1,"wal-append.queue_time_us":233451,"wal-append.run_cpu_time_us":1468,"wal-append.run_wall_time_us":1003013}
I20260709 12:26:15.129009  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.129784  8411 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:15.132591  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:15.138399 10439 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:15.116024 10336 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.140615 10338 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:15.141225 10338 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:15.143339  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1686) took 2046 ms. Trace:
W20260709 12:26:15.143481  7543 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 1.8875e+06us
I20260709 12:26:15.143822  7662 rpcz_store.cc:276] 0709 12:26:13.096543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.096926 (+   383us) service_pool.cc:224] Handling call
0709 12:26:13.097670 (+   744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.098718 (+  1048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.098759 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.671470 (+1572711us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:26:14.672131 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.904999 (+232868us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:14.905044 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.905144 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.142835 (+237691us) raft_consensus.cc:1685] finished
0709 12:26:15.142854 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.143100 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":226,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":329,"spinlock_wait_cycles":19153280,"thread_start_us":232237,"threads_started":1,"wal-append.queue_time_us":343849}
I20260709 12:26:15.115907 10441 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:15.143760  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.152812  8409 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:15.115748 10088 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:26:15.161248  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:15.170720  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.161720 10379 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 2.253s	user 0.007s	sys 0.004s
I20260709 12:26:15.142517 10338 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.172766  7807 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:15.152375 10441 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:15.178968  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 68335.7us
I20260709 12:26:15.179140  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:15.180310  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1943) took 1290 ms. Trace:
I20260709 12:26:15.181703  7737 rpcz_store.cc:276] 0709 12:26:13.889750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.667188 (+777438us) service_pool.cc:224] Handling call
0709 12:26:14.683861 (+ 16673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.714090 (+ 30229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.714123 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.721921 (+  7798us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.816273 (+ 94352us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:14.816311 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.816380 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.180073 (+363693us) raft_consensus.cc:1685] finished
0709 12:26:15.180103 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.180244 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":81014,"threads_started":1,"wal-append.queue_time_us":136995}
I20260709 12:26:15.179338 10035 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:26:15.107107  7558 rpcz_store.cc:269] 0709 12:26:02.244485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.247763 (+  3278us) service_pool.cc:224] Handling call
0709 12:26:15.106199 (+12858436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:05.853485 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:05.872408 (+ 18923us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:05.872471 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:26:05.872503 (+    32us) tablet.cc:662] Decoding operations
0709 12:26:05.874281 (+  1778us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:05.874383 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:05.874434 (+    51us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:26:06.138396 (+263962us) tablet.cc:701] Row locks acquired
0709 12:26:06.138417 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:26:06.145435 (+  7018us) write_op.cc:270] Start()
0709 12:26:06.167594 (+ 22159us) write_op.cc:276] Timestamp: P: 1783599966141385 usec, L: 0
0709 12:26:06.167631 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:26:06.323268 (+155637us) log.cc:844] Serialized 4384 byte log entry
0709 12:26:07.442399 (+1119131us) op_driver.cc:464] REPLICATION: finished
0709 12:26:08.176696 (+734297us) write_op.cc:301] APPLY: starting
0709 12:26:08.438319 (+261623us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:08.502635 (+ 64316us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:08.532393 (+ 29758us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:08.584580 (+ 52187us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:08.688388 (+103808us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:26:08.725680 (+ 37292us) write_op.cc:312] APPLY: finished
0709 12:26:08.788603 (+ 62923us) log.cc:844] Serialized 83 byte log entry
0709 12:26:09.030248 (+241645us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:09.117072 (+ 86824us) write_op.cc:454] Released schema lock
0709 12:26:09.193152 (+ 76080us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:09.281629 (+ 88477us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:09.286945 (+  5316us) write_op.cc:454] Released schema lock
0709 12:26:11.513617 (+2226672us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
Metrics: {"mutex_wait_us":10828,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":658160,"apply.run_cpu_time_us":35780,"apply.run_wall_time_us":1083149,"num_ops":11,"prepare.queue_time_us":353737,"prepare.run_cpu_time_us":30310,"prepare.run_wall_time_us":674579,"raft.queue_time_us":450310,"raft.run_cpu_time_us":49287,"raft.run_wall_time_us":832752,"replication_time_us":1234329,"spinlock_wait_cycles":15953792,"thread_start_us":1646488,"threads_started":5,"wal-append.queue_time_us":1119882,"wal-append.run_cpu_time_us":21684,"wal-append.run_wall_time_us":4821274}]]}
W20260709 12:26:15.204010  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.204990  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:15.206251  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1939) took 1400 ms. Trace:
I20260709 12:26:15.206830  7728 rpcz_store.cc:276] 0709 12:26:13.828653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.335561 (+506908us) service_pool.cc:224] Handling call
0709 12:26:14.364638 (+ 29077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.373611 (+  8973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.373662 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.397575 (+ 23913us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.629287 (+231712us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:14.629351 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.629462 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.205996 (+576534us) raft_consensus.cc:1685] finished
0709 12:26:15.206023 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.206176 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":11547,"thread_start_us":214418,"threads_started":1,"wal-append.queue_time_us":352363}
W20260709 12:26:15.210960  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.228458  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:15.236205  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.234261  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:15.243063  7737 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:15.243429  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:15.243858  7733 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:15.195837 10459 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:15.234030 10088 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.220839 10266 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.234156 10309 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:15.263072  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:15.230011 10448 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:15.275305  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.275652  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.275995  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.276242  8411 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:15.276678  8413 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:15.268788 10368 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.277196  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:15.282087  8414 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:15.279126  8408 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:15.283367  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:15.254566 10447 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:15.275338  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:15.270454 10035 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:15.254722 10369 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:15.288201  7661 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:15.288820  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.290558  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.287528  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.294989  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.295141  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:15.285986 10359 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.008s
I20260709 12:26:15.299988  7659 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:15.260430 10459 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.308403  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.308162  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.316212  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:15.316957  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:15.317322  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.317884  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.283780 10295 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.301175  7733 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:15.267446 10088 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:15.347882  7700 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.654s (SENT)
W20260709 12:26:15.318312  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.320073  7808 raft_consensus.cc:2410] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:15.273036 10266 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.290511 10035 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:15.346877  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:15.279882 10242 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:15.294064 10368 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:15.364264 10418 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.959s	user 0.005s	sys 0.000s
I20260709 12:26:15.356758  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:15.358214  7811 raft_consensus.cc:1217] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Deduplicated request from leader. Original: 4.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 12:26:15.273341 10454 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:26:15.274103 10169 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:26:15.301280 10451 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.274832 10441 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.367523  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:15.410388  7656 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:15.533196  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.367060  7728 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
W20260709 12:26:15.585515  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 285993us
I20260709 12:26:15.424608  7737 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:15.426735  7662 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:15.486784 10306 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:15.587555  7811 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:15.423070  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.606863  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:15.317937 10462 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:15.595124  7659 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:15.608507  7659 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:15.611433 10446 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:26:15.594363  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 183728us
I20260709 12:26:15.612531 10463 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:15.614176  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 366122us
I20260709 12:26:15.610337  7659 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.589335 10088 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:15.573402 10449 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 12:26:15.351063 10360 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:15.618842  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:15.620507  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.621544  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.360064 10035 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:15.365871 10368 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.375957 10295 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.627947 10459 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:15.628765 10455 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 12:26:15.629676 10292 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:15.631564  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:15.637017  8414 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:15.603611 10266 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.647620  8408 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.650898  7659 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:15.490491 10465 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 12:26:15.655072  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.626187 10441 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.654852  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.648720 10295 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.625260 10453 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.625401 10360 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:15.672238 10461 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:26:15.678381  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1912) took 3526 ms. Trace:
W20260709 12:26:15.682307  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.737288  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.652303  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:15.697382  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 349499us
I20260709 12:26:15.664613 10292 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:15.643985 10472 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:15.671406 10473 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:15.624224 10306 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.713195  7810 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:26:15.676288 10266 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:15.656569  7654 raft_consensus.cc:2376] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6. Re-sending same reply.
I20260709 12:26:15.717135  7657 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:15.605932 10451 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:15.645035 10088 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:15.653162 10169 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.723109  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.747809 10292 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.746721 10172 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:15.754451  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:15.813177 10289 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.757340  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:15.759027  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:15.733776  7736 rpcz_store.cc:276] 0709 12:26:12.152507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.338704 (+1186197us) service_pool.cc:224] Handling call
0709 12:26:13.339268 (+   564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.958271 (+1619003us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:14.958315 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.958371 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:14.958417 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.677900 (+719483us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54002486d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:15.677980 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.678275 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4886,"raft.queue_time_us":11216,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":24969,"spinlock_wait_cycles":3713240448}
I20260709 12:26:15.622474 10462 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.667400 10411 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:15.784722 10459 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:15.830617  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:15.833354  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.698513  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48495.1us
I20260709 12:26:15.736313 10453 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.748178  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:26:15.952582  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.915951  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.872627  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.812716 10473 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.824676 10266 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.951831 10289 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.957033 10459 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:15.875665 10471 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:15.910622  7653 raft_consensus.cc:2376] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
I20260709 12:26:15.904227 10370 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.876367 10169 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.040442  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.960070  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.960496 10462 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:15.887722  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:15.860620 10292 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:16.057217  7658 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:16.046334  7661 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:16.047387  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.062255  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:15.742743  7557 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 5 to 6, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 6 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:15.949648 10365 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.938s	user 0.018s	sys 0.009s
I20260709 12:26:16.052042  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:16.043517  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72733.5us
W20260709 12:26:16.082912  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 469555us
I20260709 12:26:15.971706 10266 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:16.084647  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:16.084754  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 330636us
I20260709 12:26:15.971171 10175 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.950762 10451 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.015210 10443 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.138242  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:16.115170  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53036.5us
I20260709 12:26:15.857717 10469 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.257s
W20260709 12:26:16.120277  7777 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.685s (SENT)
I20260709 12:26:15.871881 10088 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.057719 10370 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.152470  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1922) took 1013 ms. Trace:
W20260709 12:26:16.126742  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 505333us
W20260709 12:26:16.132582  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48802.9us
I20260709 12:26:16.135520  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.115844 10365 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:16.136395  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.178678  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:16.016023 10471 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:16.155501  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.180711  7662 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:16.159808  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.012s	sys 0.042s
W20260709 12:26:16.181154  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.012s	sys 0.042s
I20260709 12:26:16.165616  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.168049  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 484) took 5744 ms. Trace:
I20260709 12:26:16.154937  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:16.170748  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1917) took 1455 ms. Trace:
I20260709 12:26:16.152961  7811 rpcz_store.cc:276] 0709 12:26:15.138953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.142257 (+  3304us) service_pool.cc:224] Handling call
0709 12:26:15.142659 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.616204 (+473545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.152075 (+535871us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54001b1f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:16.152403 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495333632}
W20260709 12:26:16.188279  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.137321 10454 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:16.182169  7734 rpcz_store.cc:276] 0709 12:26:10.423592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.768199 (+2344607us) service_pool.cc:224] Handling call
0709 12:26:12.768685 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.804876 (+2036191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.146513 (+1341637us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400348854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:16.167956 (+ 21443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4680065408}
I20260709 12:26:16.183022  7808 rpcz_store.cc:276] 0709 12:26:14.714881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.719506 (+  4625us) service_pool.cc:224] Handling call
0709 12:26:16.161932 (+1442426us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54003ea8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:16.170645 (+  8713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330473216}
W20260709 12:26:16.196480  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 86728.6us
W20260709 12:26:16.141752 10464 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.002s	sys 0.044s
I20260709 12:26:16.081240 10170 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.139981 10446 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.135659  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.080817 10462 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:16.188611  8378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.026s	sys 0.039s
I20260709 12:26:16.120697 10175 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.163354 10370 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.151747 10443 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.202473  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:16.706875  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 776) took 1067 ms. Trace:
I20260709 12:26:16.707381  8411 rpcz_store.cc:276] 0709 12:26:15.639786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.640003 (+   217us) service_pool.cc:224] Handling call
0709 12:26:15.653111 (+ 13108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.653366 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.653392 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.715559 (+ 62167us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.175476 (+459917us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:16.175528 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.175622 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.706599 (+530977us) raft_consensus.cc:1685] finished
0709 12:26:16.706627 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.706805 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":362589,"threads_started":1,"wal-append.queue_time_us":731126}
W20260709 12:26:16.709945  8378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.033s	sys 0.078s
W20260709 12:26:16.718990  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.691179  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.013s	sys 0.002s
W20260709 12:26:16.721138  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.014s	sys 0.002s
W20260709 12:26:16.721593  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 581426us
I20260709 12:26:16.722539  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:16.379984  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66208.5us
I20260709 12:26:16.306604  7737 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:16.217640  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.722734  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.723896  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 521983us
W20260709 12:26:16.724469  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.605865  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 449221us
I20260709 12:26:16.174721 10360 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.731122  7659 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:16.731439  7654 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:26:16.321049  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122992us
I20260709 12:26:16.203830  7557 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:26:16.216981  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.005s	sys 0.015s
I20260709 12:26:16.188289 10451 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:16.299805  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:16.742237  7736 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:16.681072  8415 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.005s
W20260709 12:26:16.742905  8415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.004s	sys 0.006s
I20260709 12:26:16.727941  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 775) took 1094 ms. Trace:
I20260709 12:26:16.728943  7658 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:26:16.445240 10168 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.299s	user 0.007s	sys 0.014s
I20260709 12:26:16.307338  7661 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:16.330924 10425 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.263s	user 0.006s	sys 0.022s
W20260709 12:26:16.397497 10336 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.257s	user 0.020s	sys 0.009s
W20260709 12:26:16.430265 10286 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.565s	user 0.018s	sys 0.014s
I20260709 12:26:16.584033 10476 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.746271  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1731) took 1137 ms. Trace:
W20260709 12:26:16.239208 10463 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.015s	sys 0.006s
W20260709 12:26:16.435702 10447 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.136s	user 0.007s	sys 0.014s
W20260709 12:26:16.727058  8408 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.077s	user 0.021s	sys 0.022s
I20260709 12:26:16.745515  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:16.204362 10464 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.002s	sys 0.045s
W20260709 12:26:16.733537  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.012s	sys 0.036s
I20260709 12:26:16.754976  7661 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:16.646624 10459 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.335083 10295 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.479665 10460 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.218907  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.436524 10480 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.865483  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1709) took 1576 ms. Trace:
I20260709 12:26:16.200213 10306 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.866007  7660 rpcz_store.cc:276] 0709 12:26:15.288906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.289032 (+   126us) service_pool.cc:224] Handling call
0709 12:26:15.289361 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:15.289661 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:15.486223 (+196562us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.865045 (+1378822us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:16.865112 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:16.865176 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.865244 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.865402 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86537,"prepare.queue_time_us":320823,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"thread_start_us":1336172,"threads_started":2,"wal-append.queue_time_us":1059556}
I20260709 12:26:16.760556  7658 rpcz_store.cc:276] 0709 12:26:15.608761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.177133 (+568372us) service_pool.cc:224] Handling call
0709 12:26:16.728617 (+551484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.745863 (+ 17246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.746175 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21120}
W20260709 12:26:16.733067  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.937s (SENT) [suppressed 41 similar messages]
I20260709 12:26:16.767084 10336 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:16.452193 10453 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:16.771847  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.013s	sys 0.008s
W20260709 12:26:16.874648  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.013s	sys 0.008s
W20260709 12:26:16.782541 10338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.008s	sys 0.013s
I20260709 12:26:16.741449 10462 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.733023 10481 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.706328 10170 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.741668 10451 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:16.782266  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.893263  7659 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:16.767053 10463 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.024s	sys 0.044s
I20260709 12:26:16.897681  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.876727 10474 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.786759  8409 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:26:16.782665  7700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.010s	sys 0.005s
W20260709 12:26:16.916705  7700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.011s	sys 0.006s
W20260709 12:26:16.917215  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 803603us
I20260709 12:26:16.781646  7661 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:16.869993  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 136954us
I20260709 12:26:16.745016  8413 rpcz_store.cc:276] 0709 12:26:15.634054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.634238 (+   184us) service_pool.cc:224] Handling call
0709 12:26:15.636227 (+  1989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.654203 (+ 17976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.654235 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.692947 (+ 38712us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.131868 (+438921us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:16.131915 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.131992 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.329474 (+197482us) raft_consensus.cc:1685] finished
0709 12:26:16.329495 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.727872 (+398377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21374,"raft.queue_time_us":162206,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":12875,"thread_start_us":374144,"threads_started":1,"wal-append.queue_time_us":606374}
I20260709 12:26:16.774986  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:16.771066 10475 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:16.739882 10446 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.970189 10336 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:16.970655 10446 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.401652 10471 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.970927 10336 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.972350 10471 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:16.862923 10459 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.973446 10475 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.974107 10459 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.771703  8408 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:16.784392 10286 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:16.786386 10447 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:16.976609 10286 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.768548 10425 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:16.896799  8407 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.018s	sys 0.013s
W20260709 12:26:16.998534  8407 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.019s	sys 0.014s
I20260709 12:26:16.890064  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:16.774456 10476 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.874517 10460 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:16.874517 10480 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:16.887468 10487 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:16.911777  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:16.890563 10295 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:16.769593 10491 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:26:16.957041  7672 tablet_copy_service.cc:140] P be9c043c6d424ba1ab8def55dc9bb3e2: Received BeginTabletCopySession request for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40408)
W20260709 12:26:16.960335 10488 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:26:16.962111  8413 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:26:17.060284  8413 pending_rounds.cc:85] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 4
I20260709 12:26:17.060612  8413 pending_rounds.cc:107] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:26:17.061139  8413 raft_consensus.cc:2889] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.963629  7661 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:16.968300 10490 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:26:16.975967 10433 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.178s	user 0.031s	sys 0.037s
I20260709 12:26:17.022936 10463 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:16.978745  7805 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:16.759019 10168 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:16.882146 10338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.009s	sys 0.013s
I20260709 12:26:16.911820  7557 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.042059 10295 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:16.767577 10198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.004s	sys 0.024s
I20260709 12:26:16.978286 10443 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:16.997833 10336 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:16.966796 10170 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:17.024124  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 778) took 1306 ms. Trace:
I20260709 12:26:16.997680  8405 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:17.035658  7777 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.513s (SENT)
I20260709 12:26:17.038403  7658 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
W20260709 12:26:16.998653 10292 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.890s	user 0.013s	sys 0.001s
I20260709 12:26:16.945556 10474 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:17.043372  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 297975us
I20260709 12:26:17.055392  7729 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:16.996592 10502 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:16.978892 10289 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.063169 10454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.008s	sys 0.005s
W20260709 12:26:17.299414 10454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.008s	sys 0.005s
I20260709 12:26:17.299835 10454 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:17.066130  7810 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:26:17.064723  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 891325us
I20260709 12:26:16.977061 10447 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:17.033947 10446 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.076867  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:16.982198  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 773) took 1365 ms (client timeout 1539 ms). Trace:
W20260709 12:26:17.084394  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.302590  8408 rpcz_store.cc:269] 0709 12:26:15.616983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.623460 (+  6477us) service_pool.cc:224] Handling call
0709 12:26:16.979209 (+1355749us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:17.088941 10368 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:17.092295 10460 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.112226  8407 rpcz_store.cc:276] 0709 12:26:15.717881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.741214 (+ 23333us) service_pool.cc:224] Handling call
0709 12:26:15.741708 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.827067 (+ 85359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.827113 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.858946 (+ 31833us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.023635 (+1164689us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:17.023683 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.023766 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.023812 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:17.023828 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.024042 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151779,"raft.queue_time_us":89825,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":32856,"thread_start_us":1088479,"threads_started":1,"wal-append.queue_time_us":872954}
I20260709 12:26:17.094460  7654 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:17.311275  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.096625 10425 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:17.097250 10266 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.965s	user 0.013s	sys 0.000s
I20260709 12:26:17.092314 10480 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.097965 10462 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.312930  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1008135880001646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:26:17.099364 10500 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:17.059279 10487 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.118120  7557 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 4 to 6, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 6 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.152933  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.353452 10170 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:17.391260  7806 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:17.076478  7672 tablet_copy_service.cc:161] P be9c043c6d424ba1ab8def55dc9bb3e2: Beginning new tablet copy session on tablet 59f766c3d13d4360bcdeb1d5f0f318a8 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40408: session id = 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8
I20260709 12:26:17.391410 10170 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.391564 10454 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.038478  7659 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:17.205375 10292 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.153136 10478 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.207495  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 202723us
I20260709 12:26:17.211597  7661 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:26:17.423513  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69438.2us
I20260709 12:26:17.423699  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1728) took 1845 ms. Trace:
I20260709 12:26:17.386116  7803 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:17.424057  7661 rpcz_store.cc:276] 0709 12:26:15.578723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.166444 (+587721us) service_pool.cc:224] Handling call
0709 12:26:16.198868 (+ 32424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.423417 (+1224549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.423638 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:17.424646  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1735) took 1814 ms. Trace:
I20260709 12:26:17.079416 10370 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.104008 10198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.004s	sys 0.025s
I20260709 12:26:17.297365 10502 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.105018 10295 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:16.892477 10481 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.092949  7660 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 4. Ops: []
I20260709 12:26:17.310942  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:17.351879  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.310199  7700 connection.cc:256] RPC call timeout handler was delayed by -1.057431238999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
I20260709 12:26:17.098951 10410 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:17.395496  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2117) took 1403 ms. Trace:
I20260709 12:26:17.447959  7659 rpcz_store.cc:276] 0709 12:26:15.992271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.998891 (+1006620us) service_pool.cc:224] Handling call
0709 12:26:17.038289 (+ 39398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.395119 (+356830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.395412 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:17.348434 10447 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:17.449759  7777 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.818s (SENT)
W20260709 12:26:17.425830  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.015s	sys 0.015s
I20260709 12:26:17.206580 10485 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.194s
I20260709 12:26:17.425314  7656 rpcz_store.cc:276] 0709 12:26:15.610282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.187102 (+576820us) service_pool.cc:224] Handling call
0709 12:26:16.187487 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.202924 (+ 15437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.202962 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.312144 (+109182us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.770386 (+458242us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:16.770414 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.770472 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.424420 (+653948us) raft_consensus.cc:1685] finished
0709 12:26:17.424444 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.424579 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":211853,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":5638,"thread_start_us":348170,"threads_started":1,"wal-append.queue_time_us":490363}
I20260709 12:26:17.418632  7557 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.424351  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:17.387435 10035 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.743s	user 0.018s	sys 0.004s
I20260709 12:26:17.318265 10368 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:17.439924 10292 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.152834 10463 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:17.444425  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.386041 10172 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:26:17.460254  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:17.386746 10266 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.450120  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:17.444598  7700 leader_election.cc:336] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.999s (SENT)
W20260709 12:26:17.458365  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.447450 10472 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.615s	user 0.014s	sys 0.001s
W20260709 12:26:17.450484  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.016s	sys 0.015s
I20260709 12:26:17.435370 10500 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:17.494911  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 772430us
I20260709 12:26:17.445319 10491 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:26:17.499444  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111659us
I20260709 12:26:17.447695 10410 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:17.422492 10478 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.396179 10504 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:26:17.422907 10446 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:17.499943  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88397.8us
W20260709 12:26:17.517014  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67261.9us
I20260709 12:26:17.448479 10460 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:26:17.504998  7557 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:17.519485  7777 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.023s (SENT)
I20260709 12:26:17.501101 10472 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:17.498528 10465 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:17.447646 10505 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 12:26:17.525352  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.714s (SENT)
I20260709 12:26:17.486101 10510 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:17.499359 10321 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:17.528837  7808 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:17.486827 10508 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:26:17.532169  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 87601.5us
I20260709 12:26:17.597471  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.458316 10035 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.532269  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:17.508102 10495 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:17.486708 10462 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:26:17.587805  7809 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:26:17.588100  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91967.4us
W20260709 12:26:17.588446  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.012s	sys 0.059s
W20260709 12:26:17.613449  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.013s	sys 0.059s
W20260709 12:26:17.613827  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 568152us
I20260709 12:26:17.597798  7659 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:17.521347 10168 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.600916  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.454826 10451 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.589169  7557 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.525267 10292 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.609246  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.531023 10465 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.528607 10453 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:17.573230 10506 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.619680  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.524577 10198 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:26:17.595813 10457 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.508437 10491 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.610134 10495 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:17.639642  7811 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:26:17.602057 10476 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.635593  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.644115  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.657487  7659 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:17.629153  7557 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index -1 to 4, term changed from 4 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added, no longer has a pending config: VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194), VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195), NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:26:17.647332 10502 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.647778  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 777) took 1796 ms. Trace:
I20260709 12:26:17.648424  7807 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:26:17.648838 10514 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:26:17.649365 10452 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:17.654578  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.640872 10465 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:17.673440  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:17.636534 10441 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.947s	user 0.017s	sys 0.000s
I20260709 12:26:17.605410 10321 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.707769  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.691979  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.687480  7809 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:17.688478  7810 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:17.643124 10360 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:17.630486 10443 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:26:17.642567 10370 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:17.740886  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95880.2us
I20260709 12:26:17.714082  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.714653  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:17.713479  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.691973  7557 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.709456  7811 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:17.640372 10292 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.662151 10289 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:17.719961 10494 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.751606  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.721649 10321 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.612062 10368 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.577710 10510 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.652004 10507 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:26:17.644888 10454 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:26:17.598191 10447 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.725065 10521 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 12:26:17.725991 10035 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.727607 10478 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.728041 10513 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:26:17.725085  8405 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
W20260709 12:26:17.729395 10512 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:26:17.729928 10481 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.649997 10480 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:17.718995  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:17.688788  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75626.2us
I20260709 12:26:17.748350  7811 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:26:17.674747 10168 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.720212  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.708292 10451 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:17.687418 10452 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:17.801396 10480 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.802181 10452 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.803645 10452 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:17.804747 10168 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.807106 10168 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.752846  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 233983us
I20260709 12:26:17.811084 10509 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:17.811128  7672 tablet_copy_source_session.cc:215] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:26:17.783056 10368 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:17.811837 10509 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.757841  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:17.724339 10487 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.733685 10425 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.734193 10504 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.759979  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.654114 10457 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.770573 10495 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.707757 10516 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
W20260709 12:26:17.773432  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58860.1us
I20260709 12:26:17.756682 10289 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.756377 10494 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.750422 10474 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.728271  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.762614 10502 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:17.663053 10500 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.618398 10410 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.785523 10491 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.787518  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 607) took 1047 ms. Trace:
I20260709 12:26:17.783612 10266 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.789026 10292 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.789244 10198 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:17.650454 10506 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.797006  7659 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:17.723064 10463 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.751253 10370 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.751708  7557 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.793733 10446 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.830796  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.826998 10035 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:26:17.833474  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:17.832466  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.857024  7736 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:17.820416 10510 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.842157  7737 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:26:17.844324  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91960.1us
I20260709 12:26:17.842775 10495 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:17.854101 10370 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.855715 10506 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:17.856729  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.834553  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.857597  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.849704 10175 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.859754  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 772) took 2243 ms. Trace:
I20260709 12:26:17.825551 10425 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.844405  8405 rpcz_store.cc:276] 0709 12:26:16.740713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.747958 (+  7245us) service_pool.cc:224] Handling call
0709 12:26:17.787441 (+1039483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:17.845067 10454 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:17.868512  7557 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 3, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 3 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.855108 10358 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:17.829931 10504 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.882493 10358 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:17.882898 10509 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:17.878693  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.884081  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.883674 10358 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:17.884583 10504 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:17.884897  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.885253 10358 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:17.673933  8412 rpcz_store.cc:276] 0709 12:26:15.648342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.720912 (+ 72570us) service_pool.cc:224] Handling call
0709 12:26:15.733244 (+ 12332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.826533 (+ 93289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.830256 (+  3723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.830899 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.319347 (+488448us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:16.324666 (+  5319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.324784 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.426336 (+1101552us) raft_consensus.cc:1685] finished
0709 12:26:17.431038 (+  4702us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.445035 (+ 13997us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143694,"raft.queue_time_us":92724,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":20735,"thread_start_us":385608,"threads_started":1,"wal-append.queue_time_us":817720}
I20260709 12:26:17.885660 10475 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.886322 10088 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:17.885370  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.881064  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:17.889011  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.889779  7661 raft_consensus.cc:2376] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
I20260709 12:26:17.838709 10501 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:26:17.882898 10289 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.846258 10528 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:26:17.891415 10169 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.893633 10480 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.895812  7557 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:17.897501  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 517) took 1873 ms. Trace:
I20260709 12:26:17.906369  7660 rpcz_store.cc:276] 0709 12:26:16.023842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.062776 (+1038934us) service_pool.cc:224] Handling call
0709 12:26:17.081894 (+ 19118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.443105 (+361211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.897104 (+453999us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b54000df0a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f25a02 00007fc001f21dd6 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:26:17.897412 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176132480}
I20260709 12:26:17.898452 10170 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.891434  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.909336  7661 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:17.911697  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.877671  8415 rpcz_store.cc:276] 0709 12:26:15.616553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.616728 (+   175us) service_pool.cc:224] Handling call
0709 12:26:15.617178 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.955685 (+338507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.955720 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.044652 (+ 88932us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.743678 (+699026us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:16.743721 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.856158 (+1112437us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54005e0e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:17.856207 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.856253 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:17.856265 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.859677 (+  3412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7931,"raft.queue_time_us":124521,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":492782976,"thread_start_us":598879,"threads_started":1,"wal-append.queue_time_us":678158}
I20260709 12:26:17.913973  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.902441  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1733) took 2292 ms. Trace:
I20260709 12:26:17.914688  7662 rpcz_store.cc:276] 0709 12:26:15.609540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.185561 (+576021us) service_pool.cc:224] Handling call
0709 12:26:16.185914 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.186137 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:16.790563 (+604426us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.901281 (+1110718us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b18001b03a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:26:17.901636 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.901883 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.901941 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:17.901992 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.902075 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.902357 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106930,"prepare.queue_time_us":1607331,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":133,"raft.queue_time_us":1801,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12218,"spinlock_wait_cycles":158073216,"thread_start_us":826578,"threads_started":2,"wal-append.queue_time_us":716679}
I20260709 12:26:17.916725 10457 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.917179  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:17.905797  7557 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.903582 10510 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:26:17.933702 10459 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:17.933969  7557 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.922873 10475 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.935864 10475 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.937625  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:17.939455  7672 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:40408 (request call id 521) took 1758 ms. Trace:
I20260709 12:26:17.940039  7672 rpcz_store.cc:276] 0709 12:26:16.588066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.758995 (+170929us) service_pool.cc:224] Handling call
0709 12:26:17.810541 (+1051546us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b54000f4e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f2c3f7 00007fc006ec2036 00007fc006ecba3e 00007fbffe78f27b 00007fc006ecb9ba 00007fc006ec6acc 00007fc006ec1c7b 00007fc006eb68d1 00007fc005f6a4b4 00007fc005f6a442
0709 12:26:17.930856 (+120315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84590976}
I20260709 12:26:17.940866 10508 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:17.941431 10508 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:17.936136  7557 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 2, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.943737  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:17.944360  7557 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:17.934760  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 511) took 2768 ms. Trace:
I20260709 12:26:17.945401  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:17.956236  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 434) took 2330 ms. Trace:
I20260709 12:26:17.945968  7557 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.945926  8409 rpcz_store.cc:276] 0709 12:26:15.166489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.166729 (+   240us) service_pool.cc:224] Handling call
0709 12:26:15.275187 (+108458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.916520 (+1641333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.934503 (+1017983us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:17.934694 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446026496}
I20260709 12:26:17.956880  7805 rpcz_store.cc:276] 0709 12:26:15.626006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.626180 (+   174us) service_pool.cc:224] Handling call
0709 12:26:15.647735 (+ 21555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.075399 (+1427664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.946620 (+871221us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:17.956155 (+  9535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3060560256}
W20260709 12:26:17.964859  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 315162us
W20260709 12:26:17.965382  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97256us
I20260709 12:26:17.938931 10475 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:17.911000 10295 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:17.975692  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:17.976694  8405 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:17.977933  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:17.978619  8405 raft_consensus.cc:2376] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4. Re-sending same reply.
I20260709 12:26:17.979569 10198 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:17.979787  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98786us
I20260709 12:26:17.983855  7737 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:26:17.986845  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.986954  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.987932  7811 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:17.987972  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:17.987982  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:17.990047  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:17.945408  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:18.009029  7807 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:17.990890  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:18.009752  8409 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:17.991843  8415 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:17.996635  7700 leader_election.cc:336] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.597s (SENT)
W20260709 12:26:18.011234  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.011716  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.960520 10459 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:18.012341  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.002806  7805 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:26:18.005913 10471 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.033s	user 0.015s	sys 0.008s
I20260709 12:26:18.006510  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.007468  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:17.960927 10457 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.007972  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.008198  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.010056  8414 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:26:18.009151 10425 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:18.016259  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.017232  7808 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:18.000957 10340 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:26:18.004479 10295 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:18.023736  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:18.024772  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:18.026700  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45994.7us
I20260709 12:26:18.019397 10459 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.033002  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.034037  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.045822  7657 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:26:18.046315  7657 pending_rounds.cc:85] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 1
I20260709 12:26:18.046556  7657 pending_rounds.cc:107] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:26:18.046841  7657 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:18.047600  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61355.2us
I20260709 12:26:18.052403  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:18.054636  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.055336  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.018878 10172 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:18.056527 10172 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:18.056955 10172 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:18.060521  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 524) took 1326 ms. Trace:
I20260709 12:26:18.061022  7658 rpcz_store.cc:276] 0709 12:26:16.734495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.880017 (+145522us) service_pool.cc:224] Handling call
0709 12:26:18.060296 (+1180279us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:18.060444 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321325312}
W20260709 12:26:18.062430  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.991206 10504 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:26:18.078903  8409 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:18.000293  7659 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:18.011535  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.013991  7731 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:18.012061 10516 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:26:18.021304  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.086858  7737 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.021682 10480 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.021997  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.022666  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.084617  7736 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:17.942658 10508 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:18.148085  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50591.5us
I20260709 12:26:18.156232  7728 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:18.098688  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.157280 10516 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:18.157845 10516 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:18.150342  7730 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:18.150800  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.161478  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.163762  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.164729  8409 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:26:18.164953  8413 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:26:18.166565  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 138768us
W20260709 12:26:18.167939  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 171367us
I20260709 12:26:18.012246 10509 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 12:26:18.162179  8415 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:18.033968 10459 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:18.096827 10368 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:18.176537  7700 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.720s (SENT)
I20260709 12:26:18.162524  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.177767  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.179282  8407 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:18.150022 10425 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:18.182745  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.184376  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.174261 10340 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:18.069948 10451 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
W20260709 12:26:18.193924  7700 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.209s (SENT)
I20260709 12:26:18.194643  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:18.196242  7700 leader_election.cc:336] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.326s (SENT)
I20260709 12:26:18.196818  7700 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.189869  8409 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:18.191018  8413 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:18.154161  8405 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:18.201433  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 388072us
I20260709 12:26:18.100736 10485 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 12:26:18.178258 10368 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.191769 10340 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:18.207271  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.186827 10480 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.214303  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.215945  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.216857  8413 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:26:18.222585  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.028131 10463 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.186s
I20260709 12:26:18.212819 10368 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.103327 10452 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 12:26:18.027666 10457 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.175165 10516 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.230970  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.238436  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.211603 10365 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:26:18.219036 10425 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:18.207180 10525 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:18.229482  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.216871 10480 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.259150  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 606) took 1534 ms. Trace:
W20260709 12:26:18.262907  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.298159  8411 rpcz_store.cc:276] 0709 12:26:16.724743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.724886 (+   143us) service_pool.cc:224] Handling call
0709 12:26:16.740809 (+ 15923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.082780 (+341971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.082821 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.234567 (+151746us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.486498 (+251931us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:18.252878 (+766380us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b1800279640. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:18.258318 (+  5440us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b1800279640. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:18.258488 (+   170us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.258795 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.258880 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.258921 (+    41us) raft_consensus.cc:1685] finished
0709 12:26:18.258935 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.259077 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100913,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":32305,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":11056,"spinlock_wait_cycles":28459520,"thread_start_us":438392,"threads_started":3,"wal-append.queue_time_us":227118}
W20260709 12:26:18.233309  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.265364  8409 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:18.300771  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.266297  7732 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:26:18.236586 10465 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.005s
I20260709 12:26:18.238904 10463 consensus_queue.cc:1243] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:18.245312  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:18.247541 10340 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.249369 10516 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:18.250415 10532 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:26:18.290390 10492 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:18.301764  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.325810  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:18.270670  7737 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:18.280521 10500 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:18.280692  7728 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:18.285818  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.331836  7735 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:18.289521  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 501) took 2979 ms (client timeout 2562 ms). Trace:
I20260709 12:26:18.277511 10494 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:18.332365  7729 rpcz_store.cc:269] 0709 12:26:15.310041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.310248 (+   207us) service_pool.cc:224] Handling call
0709 12:26:18.289216 (+2978968us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:18.289396 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3390989184}
W20260709 12:26:18.295256  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.335213  7733 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:26:18.263531  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 609) took 1475 ms. Trace:
I20260709 12:26:18.335772  8414 rpcz_store.cc:276] 0709 12:26:16.787996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.788204 (+   208us) service_pool.cc:224] Handling call
0709 12:26:16.879529 (+ 91325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.015845 (+1136316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.263167 (+247322us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:18.263450 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2599153536}
I20260709 12:26:18.289870 10368 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:18.283171  7734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2214) took 1979 ms (client timeout 1714 ms). Trace:
I20260709 12:26:18.334419  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:18.337402  7734 rpcz_store.cc:269] 0709 12:26:16.304116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.282870 (+1978754us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:18.283047 (+   177us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:18.312448  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1732) took 2703 ms. Trace:
W20260709 12:26:18.316885  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71663.7us
W20260709 12:26:18.251348 10537 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:18.293740  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:18.340176  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:18.340703  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47090.1us
I20260709 12:26:18.341598  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:18.321760  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.277657  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1730) took 2698 ms. Trace:
I20260709 12:26:18.262146 10457 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:18.265195 10480 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:18.260394  7653 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:26:18.270483 10410 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.291442 10508 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.262681  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:18.372021  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46314.2us
I20260709 12:26:18.332791  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.373116  7737 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:18.373453  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:18.269186  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:18.294399  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50685.1us
I20260709 12:26:18.338550  7657 rpcz_store.cc:276] 0709 12:26:15.609188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.179575 (+570387us) service_pool.cc:224] Handling call
0709 12:26:17.349162 (+1169587us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:17.349414 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.047328 (+697914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.312156 (+264828us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b540028d054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:18.312373 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632252160}
I20260709 12:26:18.327427 10289 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.317704 10465 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.327322 10533 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:18.369244  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.325723 10492 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:18.330716 10500 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:18.372236  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109690us
I20260709 12:26:18.380378  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.342922  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:18.382949  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.398981  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:18.399200  7737 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:18.399693  7624 leader_election.cc:343] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Tablet error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Illegal state: cannot vote while shut down
I20260709 12:26:18.383440  7728 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:18.381071  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:18.399848  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57053.9us
I20260709 12:26:18.402617  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:18.346009  7655 rpcz_store.cc:276] 0709 12:26:15.580777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.169746 (+588969us) service_pool.cc:224] Handling call
0709 12:26:16.193692 (+ 23946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.034073 (+840381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.034115 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.037415 (+  3300us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.196378 (+158963us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:17.196700 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.277173 (+1080473us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54004371d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:18.277227 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.277279 (+    52us) raft_consensus.cc:1685] finished
0709 12:26:18.277293 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.277537 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83186,"raft.queue_time_us":88016,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":12660,"spinlock_wait_cycles":1788108544,"thread_start_us":157115,"threads_started":1,"wal-append.queue_time_us":417030}
I20260709 12:26:18.367568  7653 pending_rounds.cc:85] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 4
I20260709 12:26:18.383805 10480 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:18.382365  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.412694  7734 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.382282 10198 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.396571  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.394546  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.372236 10340 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.422335  7736 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:18.371956 10482 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:18.403821  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:18.404649  7728 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:18.386220  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.385785  8406 raft_consensus.cc:2376] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
I20260709 12:26:18.429205  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.407466  7729 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:18.408164 10465 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:18.409691 10539 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 12:26:18.400236  7624 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.411613  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:18.386991 10525 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:18.410614  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:18.387888 10286 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:18.417438  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:18.371646 10292 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:18.473738  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 70022.7us
I20260709 12:26:18.425540  7733 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.429347  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.408811  8415 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:18.424036 10441 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:26:18.411651  7653 pending_rounds.cc:107] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:26:18.405556 10464 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:26:18.440303  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.477824  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.478837  7729 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:18.479311  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.460865  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:18.411998 10526 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:18.481406  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.482369  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.482859  7737 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:18.415212 10451 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:18.416324 10480 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:18.467547  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.468115  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.469142  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.470417  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.495554  7728 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.464253  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.475013  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:18.483408  7735 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:18.486927  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 87260.8us
W20260709 12:26:18.491823  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80301.1us
W20260709 12:26:18.492787  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82263.1us
I20260709 12:26:18.497387  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:18.498126  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.490546  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.419601 10491 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:18.414611 10533 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:18.498452  7735 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:18.529188  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.530403  7735 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:18.514396  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.500712  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.533049  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.534683  7728 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.536630  7728 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:26:18.538247  7728 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.499575  7732 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:18.541908  7732 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:26:18.542435  7732 pending_rounds.cc:85] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:26:18.542778  7732 pending_rounds.cc:107] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:26:18.424175 10482 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.543507 10482 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.499966  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.523376  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.549963  7737 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.526049  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.551627  7733 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.440620 10295 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:26:18.557623 10525 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:18.470970 10492 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.559154 10492 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.558933 10525 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.484982 10292 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:18.480686 10462 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.484496 10286 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.487839  7653 raft_consensus.cc:2889] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.531840  7729 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
W20260709 12:26:18.563400  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65590.1us
W20260709 12:26:18.563560  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 88670.4us
W20260709 12:26:18.564003  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65356.9us
I20260709 12:26:18.594944  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:18.522738 10169 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:26:18.516222 10528 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:18.566217  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.491597 10510 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:18.496569 10475 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:18.543089  7732 raft_consensus.cc:2889] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.487507 10464 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:18.573427  7728 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.499926 10480 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.583798  8407 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.585716 10504 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:18.619401 10501 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:18.620642 10501 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:18.621119 10501 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:18.622104 10501 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:18.585978  7700 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.840s (SENT)
I20260709 12:26:18.623632  7700 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:18.619439 10504 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:18.516914 10451 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:18.488545 10526 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.627635 10526 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.627487 10451 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.629307 10447 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:18.531316 10491 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:18.599541 10446 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.631459 10491 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.631649 10501 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.603056 10510 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:18.632629 10510 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.600726  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:18.571390 10292 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.576436 10530 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:18.600338 10528 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.639701 10528 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.614516 10533 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.609234 10464 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.614415 10198 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:18.625756  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:18.596828  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:18.601899 10473 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:18.667464  8406 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:18.668056 10495 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.007s
I20260709 12:26:18.592465 10462 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.670691 10462 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:18.641745 10530 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:18.671607 10465 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.672243 10465 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.593926 10175 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
W20260709 12:26:18.680472  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54819.4us
I20260709 12:26:18.681313  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.599078 10169 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.654430  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.603896 10368 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:18.681584  7734 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:18.682466  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.683846  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.647445  7733 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:18.641402  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:18.658739 10172 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.601s	user 0.003s	sys 0.011s
I20260709 12:26:18.693010  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.692903  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:18.714080  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.691069  7737 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.715055  7734 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:18.675122 10465 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:18.687834  7735 raft_consensus.cc:2376] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4. Re-sending same reply.
I20260709 12:26:18.731782  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.732160  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.733420  7735 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.733814  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.734586  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.717803  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:18.721278  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.706707 10169 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:18.707602 10175 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:18.723141  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.725551 10528 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.755832  7734 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
I20260709 12:26:18.726148  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1742) took 1192 ms. Trace:
I20260709 12:26:18.678426 10500 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.735157  7733 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:18.659646 10198 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:18.671653 10530 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:18.730443  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:18.686185 10457 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:18.656212 10453 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.740712  7729 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:18.715557 10459 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:18.694024  7728 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:18.770557  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.770769  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.731743 10286 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:18.718688 10289 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:18.789196  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:18.790825 10509 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.791534 10509 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:18.794808  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.760715  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.797400  7733 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:18.762055 10198 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:18.745112  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:18.771801  7734 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 3.
I20260709 12:26:18.779577  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.780668 10459 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:18.741035 10441 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:26:18.782564  8413 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:18.739266 10510 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.783270  8405 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:18.803536  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:18.802248  7735 raft_consensus.cc:2376] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
W20260709 12:26:18.813373  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 313489us
I20260709 12:26:18.727800 10172 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.777197 10492 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:18.819011  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:18.822055  7807 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:18.824100  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.823371  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:18.826714  7808 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:26:18.828847 10368 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:18.829926 10463 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:18.831413 10446 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:18.831861 10446 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:18.832518  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.833030  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.833084 10446 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.834801 10446 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:18.833914  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.839365 10452 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:18.842013 10368 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:18.844043 10452 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.846309  7808 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:18.849561  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.850723  7811 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:26:18.851433  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.851768 10368 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.852398 10495 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:18.853434 10495 consensus_queue.cc:1243] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:18.857815 10495 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:18.867652  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.847988 10452 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:18.876791  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 183929us
W20260709 12:26:18.813926  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217238us
W20260709 12:26:18.885573  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.804863 10175 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:18.801364 10509 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:18.888690  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.896704  7812 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:18.897038  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.894721 10492 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:18.903582  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.795459 10453 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.907740 10492 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:18.815018 10286 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:18.909596  7807 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:18.908308 10528 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:18.912760  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.912716  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:18.917785  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.919373  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.922133  7654 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:18.924407  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.759328  7653 rpcz_store.cc:276] 0709 12:26:17.534072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.929592 (+395520us) service_pool.cc:224] Handling call
0709 12:26:17.930050 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.613548 (+683498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.725787 (+112239us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540033e5d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:18.726054 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758539136}
I20260709 12:26:18.930369  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.895745 10175 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:18.927374  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.933782  7662 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:18.817386 10358 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:26:18.895385 10473 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.925487  7657 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:18.936026  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.930389  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.939968  7659 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:18.940136  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.931252  7654 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:18.911391  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 188577us
I20260709 12:26:18.947629  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.948711  7733 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:18.951083  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.955796  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.940302  7805 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:18.965214  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:18.966545  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.966954  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.967322  7729 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:18.968212  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.968357  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.968876  7733 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:18.966382  7656 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:18.978559  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.981781  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:18.939108 10175 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.956904 10358 consensus_queue.cc:1243] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:18.989667  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.896407 10568 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:26:18.959986  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:19.009804  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.011586  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:18.964661 10485 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.026012  7735 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:19.013979  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:18.996502  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.017686  7731 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:19.024171  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.034355  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.035867 10452 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:19.047884  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.010946 10557 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.057060  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.063329  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.064080  7811 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:19.065155  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:18.960196 10482 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:18.985737 10443 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:26:19.034485 10509 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:19.106308  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.049919 10368 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:19.108155  8405 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:18.986768 10507 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:26:19.052842  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.110973  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.064795 10495 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:19.132622  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:19.116466 10540 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:26:19.135247  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2141626730003736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 12:26:19.150321  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1052626730001975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 12:26:19.151304  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.135792  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:19.129433 10553 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.325s	user 0.001s	sys 0.001s
W20260709 12:26:19.156193  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.127554 10509 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.162973 10509 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:19.042414 10485 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.188692  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.138072 10558 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:19.126438 10463 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:26:19.202072  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.203831  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.204188  8411 pending_rounds.cc:85] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 4
I20260709 12:26:19.207958  8411 pending_rounds.cc:107] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 4.5
W20260709 12:26:19.166415 10526 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.004s
I20260709 12:26:19.148960 10580 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:19.190104 10480 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.001s
I20260709 12:26:19.058117 10286 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.210547  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.208349  8411 raft_consensus.cc:2889] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.185585 10172 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.219069  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:19.220157  7728 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:19.158306  7653 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:19.190404 10510 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:19.226465  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75337us
I20260709 12:26:19.229972  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.071105 10557 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.225332  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.212489 10568 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.263360  7657 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:19.270769  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.271858 10485 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.272584  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.273432  7653 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:19.273718  8412 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.003s
I20260709 12:26:19.274189  8412 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:19.274711  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:19.245169 10560 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
W20260709 12:26:19.245164 10565 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.214s	user 0.000s	sys 0.004s
I20260709 12:26:19.237071 10452 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:19.262588 10578 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:26:19.278240  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48409.3us
W20260709 12:26:19.310628  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.283496  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:19.333140  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.343704  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.277258  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.007s	sys 0.003s
I20260709 12:26:19.217274 10526 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.223793 10480 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:19.278795 10491 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.008s
W20260709 12:26:19.400420  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.367564  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.007s	sys 0.004s
W20260709 12:26:19.285777 10570 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:19.399718  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:19.292241 10410 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.005s
I20260709 12:26:19.404857  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:19.422230  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:19.253407 10568 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:19.459904  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158980us
I20260709 12:26:19.464071  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:19.474936  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.482085  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 346419us
W20260709 12:26:19.509298  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45317.6us
I20260709 12:26:19.442307 10365 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.511181  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.453933 10580 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:19.402336 10585 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:19.461743 10443 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:19.519737  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:19.526068  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44088.6us
I20260709 12:26:19.526962  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.527146  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:19.479411 10592 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:26:19.425143 10491 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.537137  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.538883  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.520979 10580 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:19.544337 10580 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:19.545735  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2238) took 1138 ms. Trace:
I20260709 12:26:19.546484 10480 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:19.516135 10365 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:19.522080 10443 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.522173 10172 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:19.529489 10567 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
I20260709 12:26:19.517261 10482 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:19.551872  8408 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:19.530304 10495 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:19.567749 10495 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.568248 10495 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:19.568845  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 611) took 1512 ms. Trace:
I20260709 12:26:19.569288  8407 rpcz_store.cc:276] 0709 12:26:18.056111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.259956 (+203845us) service_pool.cc:224] Handling call
0709 12:26:18.583415 (+323459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.609620 (+ 26205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.568610 (+958990us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400780350. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:19.568777 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742641536}
I20260709 12:26:19.573170  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 526) took 1634 ms. Trace:
I20260709 12:26:19.573680  8413 rpcz_store.cc:276] 0709 12:26:17.938570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.223723 (+285153us) service_pool.cc:224] Handling call
0709 12:26:18.782180 (+558457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.810859 (+ 28679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.572930 (+762071us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400183b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:19.573095 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262398336}
I20260709 12:26:19.552642  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.546216  7737 rpcz_store.cc:276] 0709 12:26:18.407325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.551078 (+143753us) service_pool.cc:224] Handling call
0709 12:26:18.690800 (+139722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.715032 (+ 24232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.545501 (+830469us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54002481d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:19.545670 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271796992}
I20260709 12:26:19.555002  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:19.534722 10577 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.392s	user 0.000s	sys 0.002s
I20260709 12:26:19.561667  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.605320  7655 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:26:19.540778 10475 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.540242 10543 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:19.564849 10500 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:19.565649  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:19.566599  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:19.567205 10172 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:19.575212  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2216) took 1295 ms. Trace:
W20260709 12:26:19.537161 10573 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.345s	user 0.001s	sys 0.001s
I20260709 12:26:19.617455  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:19.617008  7736 rpcz_store.cc:276] 0709 12:26:18.279314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.279456 (+   142us) service_pool.cc:224] Handling call
0709 12:26:18.422063 (+142607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.422847 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.574959 (+1152112us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54003b56d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:19.575121 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326890240}
W20260709 12:26:19.582541  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.010s	sys 0.019s
W20260709 12:26:19.618774  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.011s	sys 0.019s
W20260709 12:26:19.619488  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.19026e+06us
I20260709 12:26:19.621037  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:19.585925  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46331.9us
I20260709 12:26:19.586871  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:19.602468  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47531.2us
W20260709 12:26:19.564051 10588 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:26:19.577078 10446 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.716s	user 0.012s	sys 0.000s
I20260709 12:26:19.533896 10581 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:19.551469 10575 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:26:19.542076 10485 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:19.553292 10576 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:26:19.547145 10358 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:19.587704 10292 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.952s	user 0.014s	sys 0.002s
I20260709 12:26:19.622465  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.624104  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:19.554502 10365 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.526178 10558 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 12:26:19.614708 10198 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.802s	user 0.015s	sys 0.000s
I20260709 12:26:19.554960  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.627545  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.591156 10443 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:19.588294 10480 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.631304  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.668816  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:19.606600 10462 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:19.635777  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:19.605716 10457 raft_consensus.cc:2764] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:19.610172 10543 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:19.540638 10572 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:26:19.628751 10482 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.624982 10172 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:19.631359 10451 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:19.629911 10557 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:19.669170  7655 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:19.604866 10585 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:19.624271 10475 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:19.673020  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45598.8us
I20260709 12:26:19.700222  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:19.701969  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:19.674707  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.662688  8405 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:19.706991 10595 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:19.707702 10595 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.677949  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2397) took 1067 ms. Trace:
I20260709 12:26:19.708178 10595 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:19.708302  7804 rpcz_store.cc:276] 0709 12:26:18.610259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.610410 (+   151us) service_pool.cc:224] Handling call
0709 12:26:18.619987 (+  9577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:18.636541 (+ 16554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.636576 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:18.679450 (+ 42874us) log.cc:844] Serialized 52 byte log entry
0709 12:26:18.889132 (+209682us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:18.889192 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.890587 (+  1395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.677620 (+787033us) raft_consensus.cc:1685] finished
0709 12:26:19.677658 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.677871 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15967,"raft.queue_time_us":22394,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":19664,"thread_start_us":209237,"threads_started":1,"wal-append.queue_time_us":319255}
I20260709 12:26:19.711400 10556 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:19.712105 10556 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.643906 10446 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:26:19.668668 10198 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.716068  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.717844  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:19.718094 10198 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:19.664115 10292 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:19.719569 10598 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:26:19.721202 10508 raft_consensus.cc:2764] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:26:19.721589 10292 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.690927 10462 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:19.723887  7806 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:19.724921  8414 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:19.678810 10169 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.917s	user 0.004s	sys 0.010s
I20260709 12:26:19.727830 10169 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.729048  7655 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:19.729259 10475 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.728497 10446 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:19.729822 10446 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:19.730214 10169 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:19.730698 10446 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.731527 10599 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:19.731112  7811 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:19.710884  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:19.736097  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61523.9us
I20260709 12:26:19.736974  7657 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:19.737354  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:19.742750  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.681216 10543 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:19.749423  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.750398  7655 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:19.751034  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.680651 10592 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:19.744282  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 792) took 1131 ms. Trace:
I20260709 12:26:19.732283 10446 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:19.752789  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.752027  8409 rpcz_store.cc:276] 0709 12:26:18.612674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.928509 (+315835us) service_pool.cc:224] Handling call
0709 12:26:18.928906 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.929135 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:19.571956 (+642821us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b1800321fa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:19.572149 (+   193us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.743863 (+171714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:19.743936 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:19.743987 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.744063 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.744213 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4943,"prepare.queue_time_us":34883,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":52,"raft.queue_time_us":9945,"spinlock_wait_cycles":6366976,"thread_start_us":226566,"threads_started":2}
I20260709 12:26:19.703685 10593 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:19.800719  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:19.800391  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47679.6us
I20260709 12:26:19.679688 10480 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.808990  7804 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:19.814386  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:19.738665 10368 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.002s
I20260709 12:26:19.747437 10508 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.723800 10568 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.792339 10572 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:19.756258 10592 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:19.812177 10501 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.180s	user 0.015s	sys 0.000s
I20260709 12:26:19.791458 10585 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:19.698724 10482 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:19.836580  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 214217us
I20260709 12:26:19.837590  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.748816 10292 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 12:26:19.832371 10530 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.002s
I20260709 12:26:19.823046 10508 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.849124  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:19.816936 10491 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:19.835724 10175 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.812s	user 0.007s	sys 0.007s
I20260709 12:26:19.819809 10457 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:19.850725  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5652491660002852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:26:19.822739 10368 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:19.824249 10485 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:19.854579  7804 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:19.723986 10462 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.835549 10592 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:19.838868  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:19.887210  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5571491660002721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:26:19.892729 10175 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.894200 10480 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.901224 10482 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:19.904317 10365 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:19.909341 10550 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.317s	user 0.001s	sys 0.002s
I20260709 12:26:19.910084 10416 ts_tablet_manager.cc:1916] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 12:26:19.840605 10585 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.912568 10495 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.907040  7661 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:19.916571 10580 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.836047 10572 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:19.918569 10172 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.923425 10594 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:19.855166 10508 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.853471 10530 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.924876  7653 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:19.951107 10175 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:19.923425 10584 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.480s	user 0.000s	sys 0.001s
W20260709 12:26:19.925333  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 528) took 1986 ms (client timeout 1696 ms). Trace:
W20260709 12:26:19.952420  8412 rpcz_store.cc:269] 0709 12:26:17.939158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.945126 (+  5968us) service_pool.cc:224] Handling call
0709 12:26:19.925063 (+1979937us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:19.925244 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654203136}
I20260709 12:26:19.927652 10368 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:19.928510 10509 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.002s
I20260709 12:26:19.944425 10572 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:19.929865  8379 connection.cc:256] RPC call timeout handler was delayed by -1.4050491660004809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:26:19.930423  7700 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.696s (SENT)
W20260709 12:26:19.982141  8379 connection.cc:256] RPC call timeout handler was delayed by -1.0594491660003769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
W20260709 12:26:19.982795  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67564us
W20260709 12:26:19.984375  7700 leader_election.cc:336] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.874s (SENT)
I20260709 12:26:19.984951  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:19.930140  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91363.7us
W20260709 12:26:19.930672  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116404us
I20260709 12:26:19.987689  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.989447  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:19.930802 10464 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.223s	user 0.015s	sys 0.000s
I20260709 12:26:19.943281  7808 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:19.934921 10500 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.955487  7655 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:19.955858  7731 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:19.964881  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 512) took 1696 ms. Trace:
I20260709 12:26:19.992880  7730 rpcz_store.cc:276] 0709 12:26:18.268092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.268286 (+   194us) service_pool.cc:224] Handling call
0709 12:26:18.268717 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.269282 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.269319 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.269998 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.365267 (+ 95269us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:18.365319 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.365536 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.908397 (+1542861us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:19.964404 (+ 56007us) raft_consensus.cc:1685] finished
0709 12:26:19.964438 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.964781 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4720,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":143,"spinlock_wait_cycles":11708672,"thread_start_us":94800,"threads_started":1,"wal-append.queue_time_us":215095}
I20260709 12:26:19.928406  7656 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:19.994557  7803 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:19.995021  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:19.939213 10595 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:19.998708  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.004933  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.011674  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:20.006847  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:20.017796  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2399) took 1406 ms. Trace:
I20260709 12:26:20.018579  7812 rpcz_store.cc:276] 0709 12:26:18.611014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.611153 (+   139us) service_pool.cc:224] Handling call
0709 12:26:18.623105 (+ 11952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.897306 (+274201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.017311 (+1120005us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54003c46d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:20.017684 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616860032}
I20260709 12:26:19.991989 10500 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:20.036337 10613 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.973304 10509 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.037866 10571 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:19.996670 10368 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:19.996703 10464 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:20.083516 10547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.009s	sys 0.009s
I20260709 12:26:20.088930  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:20.096560 10606 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:19.997416 10480 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:20.018430  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:20.154574  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 136239us
I20260709 12:26:20.157771  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:20.080227 10608 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:20.078920 10617 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:26:20.025789 10292 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:26:20.091456 10615 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:20.095557 10607 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:20.115410  7737 pending_rounds.cc:85] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:26:20.169412  7737 pending_rounds.cc:107] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:26:20.169872  7737 raft_consensus.cc:2889] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.170002 10603 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:20.170864 10603 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:20.171507 10603 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:20.115842  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:20.173753  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58010.6us
I20260709 12:26:20.117326  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:20.123162  8411 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:20.130429  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:19.993820 10485 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.176664 10554 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:20.176664  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:20.126151 10500 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:20.145851  7660 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:20.114974 10465 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:20.125083 10581 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:26:20.165112  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.167555  8415 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:20.145550 10547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.478s	user 0.011s	sys 0.013s
I20260709 12:26:20.180609  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:20.186570 10530 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.186324  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:20.159680 10480 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:20.193665 10443 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.502s	user 0.017s	sys 0.000s
I20260709 12:26:20.149401  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:20.205648 10172 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.159905 10446 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:26:20.206620 10618 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.178s	user 0.000s	sys 0.005s
I20260709 12:26:20.135715 10543 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.204483 10607 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:20.218122  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:20.202603 10358 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.200227 10581 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:20.221063  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 226156us
I20260709 12:26:20.212857 10480 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:20.236846 10595 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.240185  8412 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:20.241035  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.249470  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.207325 10465 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.257759 10595 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:20.258044 10465 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.215978 10443 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.259670 10509 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.248075 10495 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.238453 10607 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:20.162390 10613 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.241460 10365 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:20.253847 10580 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.262125 10368 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.259783 10464 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.287293  8412 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:20.246744 10358 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.263485 10585 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.283631  8413 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:20.278440 10617 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:20.294219 10453 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:20.294868 10621 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:20.249276 10592 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.273252  7660 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:20.288167 10556 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.291711 10443 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:20.309830  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:20.344365 10548 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.682s	user 0.019s	sys 0.018s
I20260709 12:26:20.347165  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2388) took 2364 ms. Trace:
I20260709 12:26:20.330335 10572 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.347781  7809 rpcz_store.cc:276] 0709 12:26:17.983200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.983440 (+   240us) service_pool.cc:224] Handling call
0709 12:26:18.015361 (+ 31921us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:18.181881 (+166520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.182048 (+   167us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:18.208870 (+ 26822us) log.cc:844] Serialized 52 byte log entry
0709 12:26:18.335324 (+126454us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:18.379899 (+ 44575us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.409057 (+ 29158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.409158 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.343473 (+1934315us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:20.346713 (+  3240us) raft_consensus.cc:1685] finished
0709 12:26:20.346839 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.347065 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251526,"prepare.queue_time_us":47395,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":192687,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":19001,"spinlock_wait_cycles":102759680,"thread_start_us":102312,"threads_started":1,"wal-append.queue_time_us":442863}
I20260709 12:26:20.331447 10368 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.332269 10556 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:20.380901  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 521) took 2443 ms (client timeout 1597 ms). Trace:
W20260709 12:26:20.381577  8408 rpcz_store.cc:269] 0709 12:26:17.937194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.937313 (+   119us) service_pool.cc:224] Handling call
0709 12:26:20.380416 (+2443103us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.380785 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3483019648}
I20260709 12:26:20.389209  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1978) took 1185 ms. Trace:
W20260709 12:26:20.390799  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 392168us
I20260709 12:26:20.395717  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:20.291766 10581 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.389907 10443 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.389885  7733 rpcz_store.cc:276] 0709 12:26:19.204202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.726456 (+522254us) service_pool.cc:224] Handling call
0709 12:26:19.726829 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.727152 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:19.933069 (+205917us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.388616 (+455547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.388702 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:20.388787 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.388892 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.389094 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":354281,"prepare.queue_time_us":219261,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":11801,"raft.queue_time_us":5631,"thread_start_us":454913,"threads_started":1}
I20260709 12:26:20.412846  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:20.413748  7733 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:20.391477 10556 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:20.396028 10447 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.403846  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1979) took 1199 ms. Trace:
I20260709 12:26:20.522564  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.528446  7659 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:20.377821 10368 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.525205  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.524035  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 789) took 2320 ms. Trace:
I20260709 12:26:20.519498  7729 rpcz_store.cc:276] 0709 12:26:19.204527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.817861 (+613334us) service_pool.cc:224] Handling call
0709 12:26:19.836367 (+ 18506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.836598 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:20.103722 (+267124us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.403407 (+299685us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.403472 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:20.403523 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.403591 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.403785 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148893,"prepare.queue_time_us":189320,"prepare.run_cpu_time_us":10541,"prepare.run_wall_time_us":56050,"raft.queue_time_us":6079,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":390397,"threads_started":2}
I20260709 12:26:20.403069 10543 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:20.532181  8414 rpcz_store.cc:276] 0709 12:26:18.203255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.570446 (+367191us) service_pool.cc:224] Handling call
0709 12:26:18.570926 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.725414 (+1154488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.523710 (+798296us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:20.523957 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653447296}
I20260709 12:26:20.531883  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1764) took 1314 ms. Trace:
I20260709 12:26:20.546090  7659 rpcz_store.cc:276] 0709 12:26:19.217883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.522217 (+1304334us) service_pool.cc:224] Handling call
0709 12:26:20.531823 (+  9606us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:20.562870  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57430.7us
I20260709 12:26:20.528251 10572 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.560316  7654 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:20.571537  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:20.591601  7729 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:20.592029  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:20.593004  8407 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:20.565299 10587 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.014s	sys 0.049s
I20260709 12:26:20.531147 10530 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:20.577142  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:20.594278 10587 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.015s	sys 0.049s
I20260709 12:26:20.594641  7659 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:20.595327  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2134) took 1382 ms. Trace:
I20260709 12:26:20.595719  7659 rpcz_store.cc:276] 0709 12:26:19.213083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.557453 (+1344370us) service_pool.cc:224] Handling call
0709 12:26:20.595237 (+ 37784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.599738  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 790) took 2378 ms. Trace:
I20260709 12:26:20.600337  8415 rpcz_store.cc:276] 0709 12:26:18.221126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.580166 (+359040us) service_pool.cc:224] Handling call
0709 12:26:18.586747 (+  6581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.202296 (+1615549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.599377 (+397081us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54005e0e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:20.599643 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3624853376}
I20260709 12:26:20.606884 10587 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:20.607918 10587 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:20.608938  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1980) took 1404 ms. Trace:
W20260709 12:26:20.609112  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2129) took 1638 ms (client timeout 1815 ms). Trace:
W20260709 12:26:20.609802  7657 rpcz_store.cc:269] 0709 12:26:18.970658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.970926 (+   268us) service_pool.cc:224] Handling call
0709 12:26:20.608829 (+1637903us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540021a7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.609007 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542398848}
I20260709 12:26:20.610803 10587 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.613469 10571 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:20.614205 10571 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:20.614892  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.615525  7734 rpcz_store.cc:276] 0709 12:26:19.204848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.833573 (+628725us) service_pool.cc:224] Handling call
0709 12:26:19.850502 (+ 16929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.851204 (+   702us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:20.274891 (+423687us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.608467 (+333576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.608542 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.608597 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.608680 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.608866 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153556,"prepare.queue_time_us":293359,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":3813,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":593007,"threads_started":2}
I20260709 12:26:20.616567  7729 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:20.517535 10443 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.618038 10587 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:20.617990  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:20.638666  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:20.640040  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:20.617367 10572 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.620214 10443 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.531504 10447 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.622658  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2267) took 1102 ms. Trace:
I20260709 12:26:20.624070  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:20.625769  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:20.556475 10581 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:20.617419  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2131) took 1562 ms (client timeout 1756 ms). Trace:
W20260709 12:26:20.692479  7656 rpcz_store.cc:269] 0709 12:26:19.054704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.201737 (+147033us) service_pool.cc:224] Handling call
0709 12:26:20.617139 (+1415402us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540033e0d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.617328 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628582784}
I20260709 12:26:20.628492  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2400) took 1820 ms. Trace:
I20260709 12:26:20.628726  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:20.695791  7657 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:20.628600  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:20.630635  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:20.634776  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2130) took 1623 ms (client timeout 2061 ms). Trace:
I20260709 12:26:20.636309  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1944) took 1108 ms. Trace:
I20260709 12:26:20.640869  7733 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:26:20.701198  7730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.019s	sys 0.041s
W20260709 12:26:20.701766  7730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.019s	sys 0.041s
I20260709 12:26:20.701153  7808 rpcz_store.cc:276] 0709 12:26:19.527966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.742094 (+214128us) service_pool.cc:224] Handling call
0709 12:26:19.747178 (+  5084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.991115 (+243937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.635885 (+644770us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:20.636221 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450041600}
I20260709 12:26:20.667718  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:20.671566  8405 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.019s	sys 0.033s
I20260709 12:26:20.679657  7731 rpcz_store.cc:276] 0709 12:26:19.520507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.617212 (+ 96705us) service_pool.cc:224] Handling call
0709 12:26:20.622201 (+1004989us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000721d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.622557 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776074496}
I20260709 12:26:20.697443  7803 rpcz_store.cc:276] 0709 12:26:18.807778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.808005 (+   227us) service_pool.cc:224] Handling call
0709 12:26:18.810892 (+  2887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.997841 (+1186949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.628040 (+630199us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003f3754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:20.628380 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2700267776}
I20260709 12:26:20.745823  7731 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:26:20.680150  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56247.4us
I20260709 12:26:20.748919  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:20.698292  8413 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:20.702688  7733 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:20.588397 10592 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.696388  8415 raft_consensus.cc:2376] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4. Re-sending same reply.
I20260709 12:26:20.673431 10613 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.711023  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:20.738168  7729 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:20.665086  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.751356  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2266) took 1231 ms. Trace:
W20260709 12:26:20.684937 10525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.023s	sys 0.052s
I20260709 12:26:20.760461  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1986) took 1458 ms. Trace:
I20260709 12:26:20.681591 10358 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:20.574259 10459 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:26:20.746829  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1987) took 1216 ms. Trace:
I20260709 12:26:20.768287  7731 rpcz_store.cc:276] 0709 12:26:19.530885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.744520 (+1213635us) service_pool.cc:224] Handling call
0709 12:26:20.745097 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.746424 (+  1327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.746747 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.760414  7657 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:26:20.760998  7733 rpcz_store.cc:276] 0709 12:26:19.520183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.701816 (+1181633us) service_pool.cc:224] Handling call
0709 12:26:20.702240 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.750932 (+ 48692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.751275 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.770777  7731 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:26:20.743326  8405 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.023s	sys 0.036s
I20260709 12:26:20.676141 10286 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:20.762809  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.763127  7729 rpcz_store.cc:276] 0709 12:26:19.397015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.667425 (+1270410us) service_pool.cc:224] Handling call
0709 12:26:20.737650 (+ 70225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.759880 (+ 22230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.760353 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.790601  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.696414 10614 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:20.764186  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:20.791419  7728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.048s	sys 0.091s
I20260709 12:26:20.773138  7733 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:20.797940  7733 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:20.761521  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 96575us
I20260709 12:26:20.799942 10464 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:20.800716  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:20.801875 10368 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:20.802675  7729 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:20.804248  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:20.805742 10620 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:20.805868  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.807021  7659 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:26:20.786039  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 388728us
W20260709 12:26:20.786796  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 391218us
I20260709 12:26:20.679626 10572 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.795285  7656 raft_consensus.cc:2410] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:20.771962  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1988) took 1240 ms. Trace:
W20260709 12:26:20.767982 10525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.025s	sys 0.054s
W20260709 12:26:20.752286 10500 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.004s
I20260709 12:26:20.754879 10292 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:20.796034  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:20.809747  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:20.762631 10592 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:20.809775  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:20.688704 10556 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:20.700762  7653 rpcz_store.cc:269] 0709 12:26:19.011415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.517382 (+505967us) service_pool.cc:224] Handling call
0709 12:26:20.634429 (+1117047us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400219dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.634682 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406325120}
I20260709 12:26:20.808480 10286 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.791736  7657 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:26:20.814484  7728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.048s	sys 0.091s
I20260709 12:26:20.819978  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1776) took 1183 ms. Trace:
I20260709 12:26:20.922987  7659 rpcz_store.cc:276] 0709 12:26:19.637895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.805517 (+1167622us) service_pool.cc:224] Handling call
0709 12:26:20.819685 (+ 14168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:20.827818  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.841159  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:20.927736  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:20.843178  7733 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:20.928334  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2137) took 1502 ms. Trace:
I20260709 12:26:20.928678  7656 rpcz_store.cc:276] 0709 12:26:19.426246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.818869 (+1392623us) service_pool.cc:224] Handling call
0709 12:26:20.928265 (+109396us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:20.860683  7736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.038s	sys 0.072s
W20260709 12:26:20.929704  7736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.038s	sys 0.072s
I20260709 12:26:20.930389  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1976) took 1726 ms. Trace:
I20260709 12:26:20.930738  7736 rpcz_store.cc:276] 0709 12:26:19.203575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.695527 (+491952us) service_pool.cc:224] Handling call
0709 12:26:19.731874 (+ 36347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.732081 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:19.941676 (+209595us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.929999 (+988323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.930059 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.930105 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.930193 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.930334 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336658,"prepare.queue_time_us":222979,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":3704,"raft.queue_time_us":212554,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7311,"thread_start_us":900727,"threads_started":2,"wal-append.queue_time_us":269201}
I20260709 12:26:20.898440  8407 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:20.905833  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 535) took 1050 ms. Trace:
I20260709 12:26:20.837818  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2403) took 2014 ms. Trace:
W20260709 12:26:20.838312  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.003s	sys 0.028s
W20260709 12:26:20.935896  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.003s	sys 0.028s
I20260709 12:26:20.935557  7807 rpcz_store.cc:276] 0709 12:26:18.823554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.823734 (+   180us) service_pool.cc:224] Handling call
0709 12:26:20.837465 (+2013731us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b5400331b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:20.837710 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194000128}
W20260709 12:26:20.936678  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 687285us
W20260709 12:26:20.813336  7735 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.009s	sys 0.011s
W20260709 12:26:20.937381  7735 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.010s	sys 0.011s
I20260709 12:26:20.937623  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:20.814925  7731 rpcz_store.cc:276] 0709 12:26:19.531135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.769444 (+1238309us) service_pool.cc:224] Handling call
0709 12:26:20.769911 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.771570 (+  1659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.771883 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.935411  7657 rpcz_store.cc:276] 0709 12:26:19.855143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.778708 (+923565us) service_pool.cc:224] Handling call
0709 12:26:20.905669 (+126961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.899961  7734 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:20.942230  8411 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:26:21.003826  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.004356  7736 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:20.876729 10603 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.704s	user 0.016s	sys 0.001s
W20260709 12:26:20.916445 10485 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.010s	sys 0.010s
W20260709 12:26:21.003722  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66185.7us
I20260709 12:26:20.835469 10425 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.938616 10615 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:20.816488 10628 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:20.826691 10500 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:20.871591 10368 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:20.876962 10606 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:21.054804 10485 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.011s
I20260709 12:26:20.825878 10632 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.263s
I20260709 12:26:21.029178 10525 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:20.877859 10491 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.195s
W20260709 12:26:21.084832  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81915.3us
I20260709 12:26:21.078100 10425 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:20.943593 10580 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:20.922287 10501 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 12:26:21.073987 10640 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:21.085978 10631 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:26:21.056229 10603 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.113161  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:21.134074  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 315566us
W20260709 12:26:21.104164 10629 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:26:21.024605 10634 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:21.101364 10628 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:21.101920 10447 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:21.127802 10549 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 2.488s	user 0.011s	sys 0.006s
I20260709 12:26:21.142391  7662 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:21.157009  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1761) took 2022 ms. Trace:
I20260709 12:26:21.149355  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:21.152765  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.089994 10368 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.157476  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:21.158339  8414 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:21.157534  7662 rpcz_store.cc:276] 0709 12:26:19.135096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.141719 (+2006623us) service_pool.cc:224] Handling call
0709 12:26:21.142140 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.156573 (+ 14433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.156922 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:21.164281  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51436.2us
I20260709 12:26:21.092598 10606 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.104739 10525 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.188824  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.205817  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:21.163777  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.486s	user 0.019s	sys 0.030s
I20260709 12:26:21.218395  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:21.149355 10358 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.220732  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:21.130110 10640 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.155607 10614 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:21.194983 10572 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.201694 10485 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.187570 10645 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:21.215374  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.545s	user 0.020s	sys 0.030s
I20260709 12:26:21.214613 10632 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:21.222847 10508 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.255s	user 0.014s	sys 0.003s
I20260709 12:26:21.212252 10634 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:21.225850  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:21.238919  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 429258us
I20260709 12:26:21.222072  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:21.244707 10482 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:21.255501 10482 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:21.256117 10482 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:21.244880  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:21.258579  7803 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:21.245791  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:21.264138 10640 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.266080 10525 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.223706 10464 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:21.267006 10628 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.223889 10636 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:21.226280 10642 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:21.238449 10643 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:26:21.250100  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 516) took 2683 ms. Trace:
W20260709 12:26:21.251760 10607 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.974s	user 0.000s	sys 0.015s
I20260709 12:26:21.252422 10614 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:21.236750 10480 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.986s	user 0.011s	sys 0.003s
I20260709 12:26:21.271976  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:21.275719  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.277842 10592 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.296692  7803 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:21.297529  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1983) took 2020 ms. Trace:
I20260709 12:26:21.282027  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:21.234318 10632 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.286885 10603 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.267984 10425 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.271907 10633 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.233819 10485 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:21.288090  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.278299  7732 rpcz_store.cc:276] 0709 12:26:18.566223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.614785 (+ 48562us) service_pool.cc:224] Handling call
0709 12:26:18.637483 (+ 22698us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:18.637978 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.638014 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:18.655951 (+ 17937us) log.cc:844] Serialized 52 byte log entry
0709 12:26:18.938055 (+282104us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.959840 (+ 21785us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.033560 (+ 73720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.033686 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.656584 (+622898us) raft_consensus.cc:1685] finished
0709 12:26:19.656611 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.249893 (+1593282us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b54003b56d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:21.250027 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13669,"prepare.queue_time_us":17490,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":207124,"raft.run_cpu_time_us":435,"raft.run_wall_time_us":48480,"spinlock_wait_cycles":49357184,"thread_start_us":279192,"threads_started":2,"wal-append.queue_time_us":404428,"wal-append.run_cpu_time_us":4912,"wal-append.run_wall_time_us":1542525}
I20260709 12:26:21.330817  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.294945  7807 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:21.310498 10526 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:21.310732  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.313246 10632 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:21.315820 10606 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.319367 10592 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.271806 10645 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.291567 10571 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.008s
I20260709 12:26:21.278538 10572 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.298028  7737 rpcz_store.cc:276] 0709 12:26:19.276997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.008288 (+731291us) service_pool.cc:224] Handling call
0709 12:26:20.059624 (+ 51336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.170751 (+111127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.170807 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.171489 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.402430 (+230941us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.520353 (+117923us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.562550 (+ 42197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.562662 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.297327 (+734665us) raft_consensus.cc:1685] finished
0709 12:26:21.297347 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.297465 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41829,"prepare.queue_time_us":124777,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":48487,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":6086,"thread_start_us":230365,"threads_started":1,"wal-append.queue_time_us":651377}
W20260709 12:26:21.359432  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 356002us
I20260709 12:26:21.238265 10508 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.408766  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.288102 10368 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.420680  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.301865 10642 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.226783 10358 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.293279 10607 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.424161  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.406719 10425 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.406406 10650 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.425887  7732 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:21.428872  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.295507 10647 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:21.322284 10480 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.322567 10634 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.429709  7732 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:21.426503  7777 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.861s (SENT)
I20260709 12:26:21.300793 10500 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.432592  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:21.409451 10635 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.035s
W20260709 12:26:21.441798  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.039s (SENT)
W20260709 12:26:21.451556  7777 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.155s (SENT)
W20260709 12:26:21.452435  7777 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.185s (SENT)
I20260709 12:26:21.452804  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:21.320996 10286 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.443994  7737 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:21.420953 10571 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.460459 10571 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.460973 10480 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.437849 10464 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.462432 10464 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.423548 10572 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.421413 10645 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.466065 10572 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.467617 10543 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:21.468668 10568 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:21.469741 10568 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:21.470122 10568 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:21.466799 10645 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.461318 10571 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:21.470968 10416 ts_tablet_manager.cc:1929] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 5.5
I20260709 12:26:21.472932 10607 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.439698 10650 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.477664 10416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f766c3d13d4360bcdeb1d5f0f318a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:21.434921 10425 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.481597 10425 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.483428 10485 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.484488 10653 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.485143 10653 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.497335  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.498114  7731 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:21.498476 10640 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:21.500182 10640 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:21.500921  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1942) took 2099 ms. Trace:
I20260709 12:26:21.501331  7811 rpcz_store.cc:276] 0709 12:26:19.401097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.404442 (+  3345us) service_pool.cc:224] Handling call
0709 12:26:21.500694 (+2096252us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54003ea3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:21.500854 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674159872}
I20260709 12:26:21.501335 10640 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:21.502831  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.503736  7737 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:21.504726  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.505467  7737 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:21.506336  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.506934  7737 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:21.502234 10640 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:21.509668 10640 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.516404  7659 pending_rounds.cc:85] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 5
I20260709 12:26:21.420122 10633 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.514039 10640 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:21.526139  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:21.561568  7658 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:21.523706  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:21.562851  7656 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:21.528421  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:21.564478  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1772) took 1956 ms (client timeout 2020 ms). Trace:
W20260709 12:26:21.565095  7660 rpcz_store.cc:269] 0709 12:26:19.607868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.621285 (+1013417us) service_pool.cc:224] Handling call
0709 12:26:21.528175 (+906890us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:21.564390 (+ 36215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18180480}
I20260709 12:26:21.524893  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.568261  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:21.569360  7657 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:21.582726  7732 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:26:21.454232 10635 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.009s	sys 0.036s
I20260709 12:26:21.587929  7736 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:21.598968  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.589886 10633 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:21.590495 10464 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.491523 10482 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.516964  7659 pending_rounds.cc:107] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.6
I20260709 12:26:21.429412 10642 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.616101  7659 raft_consensus.cc:2917] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Aborting config change with OpId 5.6: Aborted: Op aborted by new leader
I20260709 12:26:21.619333  7659 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.454053 10634 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.594959 10650 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:21.619310  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.624702  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.598165 10653 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:21.631264  7732 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:21.644129  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:21.646595  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.636592  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:21.632813  7656 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:21.651970  7657 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:21.655355  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1767) took 2362 ms. Trace:
I20260709 12:26:21.657503  7657 rpcz_store.cc:276] 0709 12:26:19.292948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.650603 (+2357655us) service_pool.cc:224] Handling call
0709 12:26:21.651059 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.653821 (+  2762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.655267 (+  1446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.663456  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.665881  7732 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:26:21.617244 10633 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.471073 10568 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.669844  7734 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:21.662462 10462 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.699739  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.700763  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:21.701215  7656 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:21.729056  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:21.704149 10568 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:21.732636  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:21.715301 10582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.086s	user 0.019s	sys 0.015s
I20260709 12:26:21.715960 10636 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:21.745280  7656 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:21.737537  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 284216us
I20260709 12:26:21.746541  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1769) took 2270 ms. Trace:
I20260709 12:26:21.675072 10633 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.710222 10358 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:21.718936 10292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.023s	sys 0.027s
W20260709 12:26:21.743980  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 343886us
I20260709 12:26:21.747115  7656 rpcz_store.cc:276] 0709 12:26:19.475939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.720813 (+2244874us) service_pool.cc:224] Handling call
0709 12:26:21.744990 (+ 24177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.746142 (+  1152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.746449 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:21.741271 10582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.116s	user 0.019s	sys 0.016s
I20260709 12:26:21.721419 10592 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.754357  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.738612 10645 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.703724 10462 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:21.757730  8379 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.512s (SENT) [suppressed 32 similar messages]
I20260709 12:26:21.735541 10516 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:21.756337 10616 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.795s	user 0.001s	sys 0.014s
I20260709 12:26:21.738583 10658 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:21.754781 10292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.024s	sys 0.027s
I20260709 12:26:21.743680 10607 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:21.785063  7731 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:21.793881  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:21.751653 10459 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:21.768249 10592 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:21.751170 10452 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:26:21.802430  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.771483 10358 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.814990  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:21.817595  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.819614  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:21.821148  8415 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:21.784838 10292 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:21.769928 10645 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.822726 10459 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:21.804491 10642 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.823405  7658 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:21.828281  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:21.804347 10500 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:21.837518  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:21.849321  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:21.850963  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.826344 10485 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:21.789007 10658 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.822458 10592 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:21.855928  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:21.856755  8407 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:21.833299 10459 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:21.832753 10607 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:21.853334 10654 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.177s	user 0.000s	sys 0.002s
I20260709 12:26:21.816417 10358 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.875041  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:21.870811  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83793.8us
I20260709 12:26:21.850423 10663 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:21.856333 10603 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:21.881700  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:21.859432 10485 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.864228 10642 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.888866  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1977) took 2685 ms. Trace:
I20260709 12:26:21.893262 10457 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:21.894004 10457 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:21.894508 10457 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:21.896111 10668 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:26:21.900138 10659 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:26:21.901223 10658 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.903059  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1765) took 2663 ms. Trace:
I20260709 12:26:21.903754  7662 rpcz_store.cc:276] 0709 12:26:19.239942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.187941 (+1947999us) service_pool.cc:224] Handling call
0709 12:26:21.188561 (+   620us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:21.201876 (+ 13315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.201928 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.207772 (+  5844us) log.cc:844] Serialized 52 byte log entry
0709 12:26:21.620359 (+412587us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:21.620425 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.620535 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.902807 (+282272us) raft_consensus.cc:1685] finished
0709 12:26:21.902827 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.902990 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12773,"raft.queue_time_us":218822,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":9934,"thread_start_us":403443,"threads_started":1,"wal-append.queue_time_us":539380}
I20260709 12:26:21.905236 10457 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:21.905793  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.833349 10636 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:21.858004 10368 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 12:26:21.874431 10581 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.173s	user 0.018s	sys 0.001s
I20260709 12:26:21.945608  8408 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:21.949362  7658 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:21.967137  7662 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:21.938369  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1766) took 2677 ms. Trace:
W20260709 12:26:21.945399  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 800) took 1812 ms (client timeout 1767 ms). Trace:
I20260709 12:26:21.873004 10607 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.949975  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2402) took 3128 ms. Trace:
I20260709 12:26:21.973381  7737 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:26:21.973973  7737 pending_rounds.cc:85] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
W20260709 12:26:21.971606  8413 rpcz_store.cc:269] 0709 12:26:20.133011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.176299 (+ 43288us) service_pool.cc:224] Handling call
0709 12:26:21.945100 (+1768801us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:21.945301 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240979200}
I20260709 12:26:21.974712  7737 pending_rounds.cc:107] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:26:21.975095  7737 raft_consensus.cc:2889] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.970431  7659 rpcz_store.cc:276] 0709 12:26:19.260641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.515568 (+2254927us) service_pool.cc:224] Handling call
0709 12:26:21.516145 (+   577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.643387 (+127242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.643440 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.644136 (+   696us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.830582 (+186446us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:21.830634 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.830716 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.932721 (+102005us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:21.937985 (+  5264us) raft_consensus.cc:1685] finished
0709 12:26:21.938010 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.938290 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43311360,"thread_start_us":130648,"threads_started":1,"wal-append.queue_time_us":191272}
I20260709 12:26:21.873234 10634 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.932801  7700 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:21.889690  7735 rpcz_store.cc:276] 0709 12:26:19.203899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.719249 (+515350us) service_pool.cc:224] Handling call
0709 12:26:19.719726 (+   477us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:19.753673 (+ 33947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.753725 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:19.795453 (+ 41728us) log.cc:844] Serialized 78 byte log entry
0709 12:26:19.943702 (+148249us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.150781 (+207079us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.938099 (+787318us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.938432 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.938645 (+   213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.514291 (+575646us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b1800086800. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:21.735712 (+221421us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b240010c508. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee7409d 00007fc001aa610e 00007fc001abd88c 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221
0709 12:26:21.888373 (+152661us) raft_consensus.cc:1685] finished
0709 12:26:21.888421 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.888745 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208370,"prepare.queue_time_us":210493,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":127,"raft.queue_time_us":125732,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":15904,"spinlock_wait_cycles":159816448,"thread_start_us":892833,"threads_started":2,"wal-append.queue_time_us":137650}
I20260709 12:26:21.985351  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:21.915782 10639 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.995s	user 0.004s	sys 0.000s
W20260709 12:26:22.016963  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44306.8us
I20260709 12:26:22.020328  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.891935 10485 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:21.993997  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:22.021091  7658 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:21.941731 10645 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.927906 10633 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.924587 10462 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:21.997931 10607 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.043092  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1768) took 2610 ms. Trace:
W20260709 12:26:21.950412 10649 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.665s	user 0.003s	sys 0.000s
W20260709 12:26:21.907311 10660 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:26:21.933173 10603 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:22.023972  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:21.907765  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1759) took 2869 ms. Trace:
I20260709 12:26:21.979709 10453 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:26:21.971846 10653 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.025594  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:21.909430 10457 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:22.023960  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 91207.2us
I20260709 12:26:21.990658  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:21.972990  7806 rpcz_store.cc:276] 0709 12:26:18.821997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.822156 (+   159us) service_pool.cc:224] Handling call
0709 12:26:18.901673 (+ 79517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.724531 (+822858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.949644 (+2225113us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54003723d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:21.949862 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890240256}
W20260709 12:26:22.048960  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.017997 10663 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.043712  7657 rpcz_store.cc:276] 0709 12:26:19.432624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.661726 (+2229102us) service_pool.cc:224] Handling call
0709 12:26:21.676011 (+ 14285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.690444 (+ 14433us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:21.852605 (+162161us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.042591 (+189986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.042663 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:22.042737 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.042828 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.043000 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172131,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"thread_start_us":292290,"threads_started":2}
I20260709 12:26:22.020799 10658 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:21.910427 10642 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.091676  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.081650  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:22.085469  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 192296us
I20260709 12:26:22.093927  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:22.027323 10634 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:22.098184  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.099442  7657 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 4: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:22.101009  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:22.091027 10633 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:21.912668 10292 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.103039  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:22.111631 10504 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.122474  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.123651  7731 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:22.125350 10603 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.126022 10633 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:22.127818  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:21.908078 10459 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.129586 10459 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.103577  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.133548  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.134819  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:22.109510 10640 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
I20260709 12:26:22.111631 10642 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.102455 10457 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:22.097405 10658 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.066705  7653 rpcz_store.cc:276] 0709 12:26:19.038544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.025707 (+1987163us) service_pool.cc:224] Handling call
0709 12:26:21.101221 (+ 75514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:21.150242 (+ 49021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.154789 (+  4547us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.276824 (+122035us) log.cc:844] Serialized 52 byte log entry
0709 12:26:21.752393 (+475569us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:21.759654 (+  7261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.767089 (+  7435us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.907413 (+140324us) raft_consensus.cc:1685] finished
0709 12:26:21.907446 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.907662 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22270,"raft.queue_time_us":7995,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":4768,"thread_start_us":457902,"threads_started":1,"wal-append.queue_time_us":545019}
I20260709 12:26:22.105449 10645 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:22.116183 10650 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:22.179634 10645 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:22.180752 10645 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:22.184015 10675 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:22.090736 10663 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.188424  7662 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:22.108912 10581 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:22.189924 10581 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:22.190963 10581 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:22.191871  7661 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1756) took 3214 ms (client timeout 1861 ms). Trace:
W20260709 12:26:22.192296  7661 rpcz_store.cc:269] 0709 12:26:18.977527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.977706 (+   179us) service_pool.cc:224] Handling call
0709 12:26:22.191617 (+3213911us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b5400435dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:22.191754 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033770240}
I20260709 12:26:22.136895  7657 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.944670 10592 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.032259 10528 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 12:26:22.100204 10556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.010s	sys 0.026s
I20260709 12:26:22.145373 10633 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.200736  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.174916  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:22.132535 10622 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 12:26:22.228749  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.104861 10634 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:22.071740 10416 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.592s	user 0.014s	sys 0.000s
I20260709 12:26:22.085525 10653 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.238114  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.239682  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.239876 10653 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:22.240962  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.228343  7659 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:22.144610 10464 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.150727 10457 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.232352  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1985) took 2942 ms. Trace:
I20260709 12:26:22.213817 10640 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.152801 10632 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.002s
I20260709 12:26:22.242728  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 1981) took 3007 ms. Trace:
I20260709 12:26:22.244230  7730 rpcz_store.cc:276] 0709 12:26:19.290206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.019897 (+729691us) service_pool.cc:224] Handling call
0709 12:26:20.116384 (+ 96487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.752552 (+636168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.752593 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.772576 (+ 19983us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.131506 (+358930us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:21.131573 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.131707 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.949752 (+818045us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:22.151115 (+201363us) raft_consensus.cc:1685] finished
0709 12:26:22.151140 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.232268 (+ 81128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10564,"raft.queue_time_us":141952,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":182,"spinlock_wait_cycles":19328768,"thread_start_us":889337,"threads_started":2,"wal-append.queue_time_us":489874}
I20260709 12:26:22.253909  7728 rpcz_store.cc:276] 0709 12:26:19.235855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.927003 (+691148us) service_pool.cc:224] Handling call
0709 12:26:19.927431 (+   428us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:19.927823 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.927852 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:19.928511 (+   659us) log.cc:844] Serialized 52 byte log entry
0709 12:26:20.107421 (+178910us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.149485 (+ 42064us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.917337 (+767852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.917444 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.909188 (+991744us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b180029f440. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:22.131876 (+222688us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:22.242425 (+110549us) raft_consensus.cc:1685] finished
0709 12:26:22.242457 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.242638 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41646,"prepare.queue_time_us":42266,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3556,"raft.queue_time_us":128721,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":14478,"spinlock_wait_cycles":99297408,"thread_start_us":945870,"threads_started":2,"wal-append.queue_time_us":267488}
W20260709 12:26:22.251902 10556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.011s	sys 0.027s
W20260709 12:26:22.271901 10568 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
I20260709 12:26:22.272194  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.254761 10457 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.276640 10658 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:22.277496  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2139) took 2737 ms (client timeout 1838 ms). Trace:
W20260709 12:26:22.308377  7655 rpcz_store.cc:269] 0709 12:26:19.539915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.841887 (+301972us) service_pool.cc:224] Handling call
0709 12:26:22.277174 (+2435287us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400398fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:22.277381 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233942912}
I20260709 12:26:22.242667 10634 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.278611  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 541) took 1673 ms. Trace:
I20260709 12:26:22.264607 10632 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.282044 10482 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.283586 10508 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.264212 10640 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.287652 10650 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:22.233024 10633 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.253543 10464 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.290622  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:22.270980 10638 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.351s	user 0.001s	sys 0.001s
I20260709 12:26:22.304734  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.310101  8411 rpcz_store.cc:276] 0709 12:26:20.605193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.627269 (+ 22076us) service_pool.cc:224] Handling call
0709 12:26:20.934273 (+307004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.999869 (+ 65596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.278343 (+1278474us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400712ad0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:22.278539 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704930688}
I20260709 12:26:22.302784 10580 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.312492  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.336716  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.337769  7655 raft_consensus.cc:2368] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:22.340685  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.341558  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:22.312698 10670 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.403s	user 0.004s	sys 0.000s
I20260709 12:26:22.342926  7655 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:22.340687  7661 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:22.343744  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 273496us
I20260709 12:26:22.345099  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.348928  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.341732  7662 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:22.346825  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.352823  7661 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:22.350169  7655 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:22.355991  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:22.327775 10650 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.305186 10485 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.317734 10663 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.339881 10580 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:22.384791  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 251133us
I20260709 12:26:22.357486 10642 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.354866 10568 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.388557  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.366115 10595 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:22.401059  7661 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:22.401551  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:22.402300  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.363296 10658 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.393231  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.394241  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:22.386929  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:22.381345 10416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40594 (request call id 602) took 8469 ms. Trace:
I20260709 12:26:22.403965  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:22.402251  8414 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:22.410018  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.413343  7662 raft_consensus.cc:2410] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:22.416688  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:22.391304  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:22.411893  8411 raft_consensus.cc:1800] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEARNER]: voting while copying based on last-logged opid 5.5
I20260709 12:26:22.424014  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:22.447311  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:22.458667  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:22.461695  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 103613us
I20260709 12:26:22.405304 10568 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.398998 10642 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.438500  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:22.470496  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:22.443022 10464 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.457039 10663 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:22.477409  7653 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:22.411052 10658 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.479429 10453 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.482355 10480 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:22.480888 10595 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:22.519368  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48848.5us
I20260709 12:26:22.410876 10416 rpcz_store.cc:276] 0709 12:26:13.903492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.912995 (+  9503us) service_pool.cc:224] Handling call
0709 12:26:14.228756 (+315761us) ts_tablet_manager.cc:934] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
0709 12:26:19.746735 (+5517979us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:26:20.345926 (+599191us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:21.470761 (+1124835us) tablet_metadata.cc:665] Metadata flushed
0709 12:26:21.473004 (+  2243us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:26:22.253984 (+780980us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:26:22.310885 (+ 56901us) tablet_replica.cc:206] Creating consensus instance
0709 12:26:22.361079 (+ 50194us) ts_tablet_manager.cc:989] Replica registered.
0709 12:26:22.363566 (+  2487us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":13087,"tablet-copy.queue_time_us":314049,"thread_start_us":359448,"threads_started":2}
I20260709 12:26:22.521863 10416 tablet_copy_client.cc:806] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:26:22.522615 10416 tablet_copy_client.cc:670] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
W20260709 12:26:22.525216  8440 heartbeater.cc:646] Failed to heartbeat to 127.6.76.254:33941 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.76.254:33941 timed out after 15.000s (SENT)
I20260709 12:26:22.528322 10613 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:22.528887 10613 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:22.530014 10613 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:22.532680  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84558.3us
I20260709 12:26:22.526815  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.536899  7735 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:22.502925 10632 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.517479 10453 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.516849 10480 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.516216 10516 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:22.538177 10640 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.546419 10613 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:22.567546 10690 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:26:22.567713 10485 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:22.569540 10691 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 12:26:22.571822 10694 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:22.551750 10480 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:22.578347 10683 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 12:26:22.579892  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.545950 10633 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.584005  7735 raft_consensus.cc:2410] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:22.591044  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:22.531972 10595 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.565145 10698 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.600541 10410 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:22.620280 10410 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.620216 10698 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.604060 10358 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.609162 10642 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.623513  7653 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:22.624414  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1774) took 3007 ms. Trace:
I20260709 12:26:22.624797  7653 rpcz_store.cc:276] 0709 12:26:19.617220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.563059 (+2945839us) service_pool.cc:224] Handling call
0709 12:26:22.612560 (+ 49501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.624093 (+ 11533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.624343 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.626289  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:22.628079 10607 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
I20260709 12:26:22.629004 10607 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:22.610711  7736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 527) took 1377 ms (client timeout 1519 ms). Trace:
I20260709 12:26:22.590668 10464 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:26:22.582333 10640 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.641685  7736 rpcz_store.cc:269] 0709 12:26:21.233516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.238110 (+  4594us) service_pool.cc:224] Handling call
0709 12:26:22.610488 (+1372378us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:22.610604 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674081536}
I20260709 12:26:22.642149 10640 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.621241 10410 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.643430  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.645045  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.645818  7736 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.608806 10457 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.650446  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:22.636369  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.605376 10582 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:22.640503  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:22.666218 10358 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.669625 10706 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:22.627012  7653 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:22.654155  7735 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:22.652532 10652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.006s	sys 0.012s
I20260709 12:26:22.679623  7662 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:22.670842  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:22.684501 10698 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:22.685189 10453 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:22.688145 10634 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.689146 10480 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.690294 10595 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.691470 10410 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.693813 10634 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.695571 10410 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.695286 10650 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.699187 10650 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:22.699795 10410 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:22.701426 10480 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.702993 10697 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:22.703554 10568 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.656946 10582 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:22.662850 10464 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:22.703902  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:22.705967  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157945us
I20260709 12:26:22.712193  7735 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:22.709156  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.715163  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:22.683753 10652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.007s	sys 0.012s
I20260709 12:26:22.732941  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.729022  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.730376  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.730955  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.735158  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.735666  7736 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:22.735682  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.734098  7806 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:22.737344  7808 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.738207  7807 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:22.738516  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.725865 10464 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.739857  7811 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:22.742401  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75114.6us
I20260709 12:26:22.744134  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:22.679729 10358 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.747885  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.748224 10457 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.748515  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.749146  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.749830  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.748735  7811 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:22.732069 10485 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:22.751292  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.749558 10580 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.754343  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.745231 10464 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.750116  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:22.753480  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.739396 10480 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.764386  7803 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:22.765954  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:22.771939  7777 leader_election.cc:400] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:22.755357 10516 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.723999 10410 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.778208 10580 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.779070  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.779340 10582 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.728974 10697 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.745492  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:22.780194 10607 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:22.864261  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.886188  7805 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:26:22.839493  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1770) took 3314 ms. Trace:
I20260709 12:26:22.881976  7807 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:26:22.882102  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.880925  7777 leader_election.cc:403] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:26:22.806056  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.877290  7808 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:26:22.842370  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:22.844642  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.884819  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.843725  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.889729  7736 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:22.887245  7656 rpcz_store.cc:276] 0709 12:26:19.524826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.753349 (+2228523us) service_pool.cc:224] Handling call
0709 12:26:21.768753 (+ 15404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.796278 (+ 27525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.796328 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.797141 (+   813us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.969576 (+172435us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:21.969621 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.838964 (+869343us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b540028e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:22.839034 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.839090 (+    56us) raft_consensus.cc:1685] finished
0709 12:26:22.839105 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.839410 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10027,"raft.queue_time_us":5677,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":6525,"spinlock_wait_cycles":49378560,"thread_start_us":137244,"threads_started":1,"wal-append.queue_time_us":210505}
I20260709 12:26:22.891709  7803 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:22.887982  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138015us
I20260709 12:26:22.895543  7735 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:22.897635  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 125864us
I20260709 12:26:22.901711  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.900427  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:22.905028  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:22.905596  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.906529  7806 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:22.785014 10485 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:22.784214 10656 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:22.908826  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.783890 10640 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.897222 10568 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.912618  7803 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:22.901355 10582 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:22.897841 10695 raft_consensus.cc:2764] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:22.885711 10516 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.905977 10482 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:22.929684  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.921244 10410 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.916898 10568 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:22.924700  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.911762 10656 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:22.926527 10697 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:22.917739  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.915179 10528 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:22.930872  7736 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:22.939719  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:22.940451  7736 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:22.940299 10485 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.941278 10656 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.942478 10656 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.933030 10482 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.886961 10480 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.916941 10698 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:22.933534  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.929898 10457 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.944718  7808 raft_consensus.cc:2376] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5. Re-sending same reply.
I20260709 12:26:22.913540 10706 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:22.944911 10457 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:22.949958 10516 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.950315  7803 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:22.951654  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.952425  7806 raft_consensus.cc:2376] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6. Re-sending same reply.
W20260709 12:26:22.953941  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.954725 10480 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.945066 10482 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:22.956039 10568 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.966024  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.967715  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:22.970634  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:22.945497 10634 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:22.947415 10358 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:22.947582 10528 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:22.983732 10528 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.984391  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.985273  7812 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:26:22.986183  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.946189 10698 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.975541  7736 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:22.975579  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 133281us
I20260709 12:26:22.980296  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.989048  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.989632  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 544) took 1233 ms. Trace:
I20260709 12:26:22.980410  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.990032  7658 rpcz_store.cc:276] 0709 12:26:21.756335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.848802 (+ 92467us) service_pool.cc:224] Handling call
0709 12:26:22.989436 (+1140634us) spinlock_profiling.cc:243] Waited 99.2 ms on lock 0x7b540028d554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:22.989550 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228147968}
I20260709 12:26:22.981098  7735 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:22.982019 10634 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.993405  7806 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:22.993556  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.998610  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.998936  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.999262  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:22.999472  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:22.999759  7809 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.000414  7805 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:26:23.000661  7808 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:22.995951  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:23.003800  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.005002  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.005976  7736 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:23.006758  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.007251  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:23.008452  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.009418 10712 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:23.009413 10595 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:23.010157 10712 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.010556 10712 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.010468 10595 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.013646  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.014554 10170 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:23.015322 10170 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.015959 10634 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.016922  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.016634 10170 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:22.988060 10698 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.024132  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.024152  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.028025  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.029253  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.030094  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.031033  7736 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:23.031613  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.035763  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.036659  7805 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:23.037766 10651 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.753s	user 0.003s	sys 0.002s
I20260709 12:26:23.041079  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.037766  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.044935  7809 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:23.046379  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.002451 10457 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.047346  7736 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:23.052060  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.053383  7806 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:23.055995  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.056181  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.057649  7806 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 3: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:23.059253  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:23.060120  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.060343  7736 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:23.062165  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:23.063727  8412 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:23.064379  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 631) took 1919 ms. Trace:
I20260709 12:26:23.064775  8412 rpcz_store.cc:276] 0709 12:26:21.145024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.149246 (+  4222us) service_pool.cc:224] Handling call
0709 12:26:23.055808 (+1906562us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:23.064296 (+  8488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16349952}
I20260709 12:26:23.067799  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.068682  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.069463 10368 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:23.070674  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:23.071655  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 530) took 3226 ms (client timeout 1580 ms). Trace:
I20260709 12:26:23.072328  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 622) took 4810 ms. Trace:
I20260709 12:26:23.071532  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:23.072139  7654 rpcz_store.cc:269] 0709 12:26:19.845692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.524881 (+679189us) service_pool.cc:224] Handling call
0709 12:26:23.071423 (+2546542us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b540033d6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.071568 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64845184}
I20260709 12:26:23.074906  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.075626  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:23.075964  7729 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:23.069454  7731 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:23.076838  7736 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:23.078826  7731 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:23.079593  7654 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.080471  7731 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.078029 10410 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:23.083339  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1781) took 3401 ms. Trace:
I20260709 12:26:23.081773 10410 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.054132 10706 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.085558  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.084110  7736 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.084136  7654 rpcz_store.cc:276] 0709 12:26:19.681820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.074854 (+3393034us) service_pool.cc:224] Handling call
0709 12:26:23.075159 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.082895 (+  7736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.083248 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.088258  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.093838  7806 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:23.092206  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 524) took 2952 ms. Trace:
I20260709 12:26:23.094105  7654 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:23.094559  7733 rpcz_store.cc:276] 0709 12:26:20.140055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.817001 (+676946us) service_pool.cc:224] Handling call
0709 12:26:20.817482 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.928326 (+110844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.091560 (+2163234us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:23.092119 (+   559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57902848}
I20260709 12:26:23.099663  7736 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.099694  7733 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.101341  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 527) took 3257 ms. Trace:
I20260709 12:26:23.101605  7729 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.103392  7654 rpcz_store.cc:276] 0709 12:26:19.844486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.093470 (+3248984us) service_pool.cc:224] Handling call
0709 12:26:23.093883 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.098500 (+  4617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.101265 (+  2765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.104524  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.105373  7654 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:23.106228  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 534) took 3256 ms. Trace:
I20260709 12:26:23.087677 10410 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.108522  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.106683  7654 rpcz_store.cc:276] 0709 12:26:19.850238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.104790 (+3254552us) service_pool.cc:224] Handling call
0709 12:26:23.105167 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.105911 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.106162 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.112617  7654 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:23.114255  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.118860  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2140) took 3209 ms. Trace:
I20260709 12:26:23.119508  7654 rpcz_store.cc:276] 0709 12:26:19.910044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.110746 (+3200702us) service_pool.cc:224] Handling call
0709 12:26:23.112341 (+  1595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.117650 (+  5309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.118791 (+  1141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.083503 10464 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.121351  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.122618  7654 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:26:23.124786  7654 pending_rounds.cc:85] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 5
I20260709 12:26:23.125180  7654 pending_rounds.cc:107] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:26:23.125514  7654 raft_consensus.cc:2889] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.125602  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.126463  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 537) took 2925 ms. Trace:
I20260709 12:26:23.126946  7654 rpcz_store.cc:276] 0709 12:26:20.200919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.121980 (+2921061us) service_pool.cc:224] Handling call
0709 12:26:23.122272 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.126098 (+  3826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.126397 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.091635 10706 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.141002  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.143277  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.137120  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.108086 10508 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:26:23.141002  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.138489  7654 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:23.152827  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.158336  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 538) took 2945 ms. Trace:
I20260709 12:26:23.159562  7654 rpcz_store.cc:276] 0709 12:26:20.213333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.133960 (+2920627us) service_pool.cc:224] Handling call
0709 12:26:23.138238 (+  4278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.157832 (+ 19594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.158136 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.167827  7654 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.172750  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:23.175947  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.177062  7809 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:26:23.180559  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2146) took 1658 ms (client timeout 2072 ms). Trace:
I20260709 12:26:23.180694  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:23.181146  7660 rpcz_store.cc:269] 0709 12:26:21.521751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.620124 (+ 98373us) service_pool.cc:224] Handling call
0709 12:26:23.172563 (+1552439us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:23.180496 (+  7933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29846400}
I20260709 12:26:23.181597  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 545) took 1860 ms. Trace:
I20260709 12:26:23.182312  7806 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:23.183264 10713 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:26:23.181175 10504 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:23.188347  7804 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:23.189602  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.191493  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.192323  7812 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.193120  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.194276  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:23.182436 10714 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 12:26:23.196483  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1780) took 3522 ms. Trace:
I20260709 12:26:23.196959  7658 rpcz_store.cc:276] 0709 12:26:19.673805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.995908 (+3322103us) service_pool.cc:224] Handling call
0709 12:26:22.996305 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.996917 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.996966 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.997608 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.034626 (+ 37018us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:23.034675 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.034765 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.196206 (+161441us) raft_consensus.cc:1685] finished
0709 12:26:23.196247 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.196411 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5105,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4464,"thread_start_us":20985,"threads_started":1,"wal-append.queue_time_us":28233}
I20260709 12:26:23.182024  8408 rpcz_store.cc:276] 0709 12:26:21.321721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.321888 (+   167us) service_pool.cc:224] Handling call
0709 12:26:21.945321 (+623433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.968188 (+ 22867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.181421 (+1213233us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400183b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.181537 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244085120}
I20260709 12:26:23.201885 10170 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.132548 10613 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:23.202697  7658 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:26:23.203819  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.205724  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.207504  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.146274 10368 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:23.209303  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2275) took 1905 ms. Trace:
I20260709 12:26:23.072975  8406 rpcz_store.cc:276] 0709 12:26:18.261825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.668709 (+406884us) service_pool.cc:224] Handling call
0709 12:26:18.669171 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.669612 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.669648 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.670234 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.295382 (+1625148us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:26:21.241412 (+946030us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:21.244429 (+  3017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.244558 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.037025 (+1792467us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:23.072039 (+ 35014us) raft_consensus.cc:1685] finished
0709 12:26:23.072069 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.072236 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1839,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":48359936,"thread_start_us":2560968,"threads_started":1,"wal-append.queue_time_us":2614378}
I20260709 12:26:23.213526 10464 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.217577  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.219555  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.220532  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:23.221665 10706 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:23.172390 10640 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:23.190866  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.269809  7804 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:23.230676 10482 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.255669  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 546) took 1307 ms. Trace:
I20260709 12:26:23.270848  7658 rpcz_store.cc:276] 0709 12:26:21.948555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.198593 (+1250038us) service_pool.cc:224] Handling call
0709 12:26:23.199212 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.255265 (+ 56053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.255476 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.271539  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2000) took 1119 ms. Trace:
I20260709 12:26:23.257896  8408 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:23.257303  7806 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:26:23.256038  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.259308  8413 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:23.259616  7730 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.229120  7811 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:23.260419  7737 rpcz_store.cc:276] 0709 12:26:21.304347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.712308 (+407961us) service_pool.cc:224] Handling call
0709 12:26:21.712851 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.975656 (+262805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.209008 (+1233352us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54002d7b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:23.209228 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598169984}
I20260709 12:26:23.283047  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.286386  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:23.287212 10443 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:26:23.294968 10443 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:23.288618  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.290717  7809 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:23.293301 10648 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:23.294467 10712 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.269975  8405 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.300135  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:23.303608  7807 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:26:23.290884 10528 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:26:23.296789  8412 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.304045 10365 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:23.261991 10368 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:23.275345 10640 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.307829  7805 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:23.309381  7810 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.297469 10648 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:23.318629  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 119672us
I20260709 12:26:23.325556  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.325914  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:23.326622  8415 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.326483  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.326943  8407 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.327709  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:23.329389  8414 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.294467 10410 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.331449  8415 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:23.333864 10482 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.332710 10698 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.333127  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.337484  8411 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:23.341449 10648 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.349540 10606 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.350471 10606 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.342558 10712 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:23.345880 10648 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.355916 10658 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.355903 10595 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.361661 10640 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:23.367439 10698 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.180509  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 540) took 2898 ms. Trace:
I20260709 12:26:23.407249  7654 rpcz_store.cc:276] 0709 12:26:20.282179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.167283 (+2885104us) service_pool.cc:224] Handling call
0709 12:26:23.167622 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.180230 (+ 12608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.180448 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.411407  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.414669  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.416532  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.419041  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.420353  7661 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:26:23.418830  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2147) took 1896 ms. Trace:
I20260709 12:26:23.421162  7660 rpcz_store.cc:276] 0709 12:26:21.522065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.182164 (+1660099us) service_pool.cc:224] Handling call
0709 12:26:23.182491 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.182779 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.182818 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.188035 (+  5217us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.417988 (+229953us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.418136 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.418276 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.418354 (+    78us) raft_consensus.cc:1685] finished
0709 12:26:23.418365 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.418589 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322304,"thread_start_us":229205,"threads_started":1,"wal-append.queue_time_us":155854}
I20260709 12:26:23.422613  7654 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:23.423501  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:23.424333  7660 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.428144  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2006) took 1043 ms. Trace:
I20260709 12:26:23.420717  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:23.287755  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.436280  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:23.437140  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.437763  7737 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:23.438589  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.439895  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 135962us
I20260709 12:26:23.429602  7736 rpcz_store.cc:276] 0709 12:26:22.384316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.113659 (+729343us) service_pool.cc:224] Handling call
0709 12:26:23.114015 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.116559 (+  2544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.116628 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.134731 (+ 18103us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.426925 (+292194us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:23.427063 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.427265 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.427331 (+    66us) raft_consensus.cc:1685] finished
0709 12:26:23.427349 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.427755 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1738,"raft.queue_time_us":1949,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":9173,"thread_start_us":260700,"threads_started":1,"wal-append.queue_time_us":187257}
I20260709 12:26:23.442741  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.448581  7812 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:23.453488  8406 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.452904 10443 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:23.455057 10443 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:23.456848 10443 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:23.457002  8412 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:23.457836 10443 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.459357 10365 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:23.460005 10365 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.461715 10365 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.464663 10365 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.465874 10368 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.460001 10443 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.473426 10453 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:23.424399  7654 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:23.329008  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.475284  7658 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.446949  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:23.483096  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1782) took 1481 ms. Trace:
I20260709 12:26:23.483793  7654 rpcz_store.cc:276] 0709 12:26:21.997173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.423484 (+1426311us) service_pool.cc:224] Handling call
0709 12:26:23.424060 (+   576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.478452 (+ 54392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.478704 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.482234  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1785) took 1483 ms. Trace:
I20260709 12:26:23.488021  7658 rpcz_store.cc:276] 0709 12:26:21.998005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.434042 (+1436037us) service_pool.cc:224] Handling call
0709 12:26:23.434884 (+   842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.481262 (+ 46378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.481621 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.476485 10453 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.495208  7736 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.501528  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2010) took 1093 ms. Trace:
W20260709 12:26:23.508450  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.505183  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.509984  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.516376  7736 rpcz_store.cc:276] 0709 12:26:22.406393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.443877 (+1037484us) service_pool.cc:224] Handling call
0709 12:26:23.444509 (+   632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.499143 (+ 54634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.499607 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:23.514544  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2153) took 1724 ms (client timeout 1779 ms). Trace:
I20260709 12:26:23.515086  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2008) took 1120 ms. Trace:
I20260709 12:26:23.503129  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2009) took 1105 ms. Trace:
W20260709 12:26:23.515471  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 186420us
I20260709 12:26:23.512442  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1771) took 3964 ms. Trace:
W20260709 12:26:23.525342  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 78735us
I20260709 12:26:23.523588  7737 rpcz_store.cc:276] 0709 12:26:22.397436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.439023 (+1041587us) service_pool.cc:224] Handling call
0709 12:26:23.498497 (+ 59474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.502342 (+  3845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.502403 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.502699 (+   296us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:23.502756 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.502849 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.503030 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:23.520598  7659 rpcz_store.cc:269] 0709 12:26:21.789691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.977917 (+188226us) service_pool.cc:224] Handling call
0709 12:26:23.513973 (+1536056us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54004371d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.514259 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476519296}
I20260709 12:26:23.531939 10453 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.532965  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2287) took 1411 ms. Trace:
I20260709 12:26:23.533473  7734 rpcz_store.cc:276] 0709 12:26:22.121513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.127399 (+  5886us) service_pool.cc:224] Handling call
0709 12:26:23.532327 (+1404928us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.532696 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545645952}
I20260709 12:26:23.534368 10453 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:23.538316 10658 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.511760 10595 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.539511 10658 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.540136 10595 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.540720  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.522848  7729 rpcz_store.cc:276] 0709 12:26:22.394753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.144853 (+750100us) service_pool.cc:224] Handling call
0709 12:26:23.155070 (+ 10217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.155496 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.155530 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.175577 (+ 20047us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.480127 (+304550us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:23.480239 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.480374 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.514628 (+ 34254us) raft_consensus.cc:1685] finished
0709 12:26:23.514651 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.514834 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6406,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":42269,"thread_start_us":202816,"threads_started":1,"wal-append.queue_time_us":223294}
I20260709 12:26:23.544366 10595 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.545670  7728 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:23.544366  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2273) took 2240 ms. Trace:
I20260709 12:26:23.547812 10528 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:23.548717 10528 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.547986  7732 rpcz_store.cc:276] 0709 12:26:21.303676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.673578 (+369902us) service_pool.cc:224] Handling call
0709 12:26:21.674132 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.767501 (+ 93369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.767535 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.776660 (+  9125us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.869823 (+ 93163us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:21.869894 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.543581 (+1673687us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b54001c43d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:23.543673 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.543749 (+    76us) raft_consensus.cc:1685] finished
0709 12:26:23.543759 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.544150 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2224,"raft.queue_time_us":3435,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9463,"spinlock_wait_cycles":208339200,"thread_start_us":78346,"threads_started":1,"wal-append.queue_time_us":191914,"wal-append.run_cpu_time_us":3455,"wal-append.run_wall_time_us":1189691}
I20260709 12:26:23.550146 10410 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.551880  7732 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:23.552799  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2015) took 1105 ms. Trace:
I20260709 12:26:23.553252  7732 rpcz_store.cc:276] 0709 12:26:22.447005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.550900 (+1103895us) service_pool.cc:224] Handling call
0709 12:26:23.551525 (+   625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.552412 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.552706 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.554163  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2005) took 1186 ms. Trace:
I20260709 12:26:23.554683  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:23.556334  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:23.554644  7733 rpcz_store.cc:276] 0709 12:26:22.367335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.101893 (+734558us) service_pool.cc:224] Handling call
0709 12:26:23.102413 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.102696 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:23.261602 (+158906us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.511145 (+249543us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.542442 (+ 31297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.546683 (+  4241us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:23.553529 (+  6846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.553687 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.553954 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101841,"prepare.queue_time_us":41207,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":135,"raft.queue_time_us":91247,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":1844,"thread_start_us":299652,"threads_started":2,"wal-append.queue_time_us":76775}
I20260709 12:26:23.558163  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.559588  7733 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:23.561161  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.508980 10568 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:23.566452 10568 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.567628  7659 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:26:23.568163  7659 pending_rounds.cc:85] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 2
I20260709 12:26:23.568475  7659 pending_rounds.cc:107] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:26:23.568841  7659 raft_consensus.cc:2889] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.512202 10582 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:23.569484  7732 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:23.575529  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:23.578023  7736 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.515996 10613 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:23.579005  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2011) took 1170 ms. Trace:
W20260709 12:26:23.572602  7728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2295) took 1434 ms (client timeout 1839 ms). Trace:
I20260709 12:26:23.582023  7736 rpcz_store.cc:276] 0709 12:26:22.409135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.520359 (+1111224us) service_pool.cc:224] Handling call
0709 12:26:23.577642 (+ 57283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.578697 (+  1055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.578948 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376128}
I20260709 12:26:23.568612  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.584100  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.578955  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.586397 10462 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:23.586880  8406 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:23.587384 10504 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:23.589278  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.586158  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:23.582216  7728 rpcz_store.cc:269] 0709 12:26:22.130367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.540046 (+1409679us) service_pool.cc:224] Handling call
0709 12:26:23.564648 (+ 24602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.563767  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:23.597572  8412 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.524012  7657 rpcz_store.cc:276] 0709 12:26:19.548173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.412617 (+2864444us) service_pool.cc:224] Handling call
0709 12:26:23.073335 (+660718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.073720 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.073749 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.082376 (+  8627us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.260449 (+178073us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.260510 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.260690 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.415666 (+154976us) raft_consensus.cc:1685] finished
0709 12:26:23.415739 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.511722 (+ 95983us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b540033d6d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.512164 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":526,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6456,"spinlock_wait_cycles":1518640512,"thread_start_us":144912,"threads_started":1,"wal-append.queue_time_us":212742}
I20260709 12:26:23.586982 10462 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.605067 10410 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:23.605746 10410 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.606148 10410 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.611249  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.611776  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.611858 10292 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:23.612785  8406 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:23.612879 10292 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:23.613140  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2154) took 1818 ms. Trace:
I20260709 12:26:23.613906 10482 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:23.613703  7661 rpcz_store.cc:276] 0709 12:26:21.794430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.409326 (+614896us) service_pool.cc:224] Handling call
0709 12:26:23.612826 (+1203500us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540002fb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.613042 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207379200}
I20260709 12:26:23.618484 10464 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:26:23.633183  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.634227  8411 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:23.635602 10508 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:23.636721  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.637017 10508 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:23.637562 10508 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:23.637683  7811 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:23.638255  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.639495 10416 tablet_copy_client.cc:538] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:26:23.639988  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:23.642592  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.644027  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.615042 10292 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.550642 10528 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.656615  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:23.656908  7660 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:23.634907 10582 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:23.657204  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.657559  7657 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.675640  7730 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:23.679880  7736 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 2. Current term is 3. Ops: []
I20260709 12:26:23.665452  7659 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.704493  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1802) took 1092 ms. Trace:
I20260709 12:26:23.705384  7659 rpcz_store.cc:276] 0709 12:26:22.612293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.570876 (+958583us) service_pool.cc:224] Handling call
0709 12:26:23.619245 (+ 48369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.704223 (+ 84978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.704426 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.658257  7732 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:26:23.659502  7661 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:26:23.689854  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2150) took 1913 ms. Trace:
I20260709 12:26:23.663036  7733 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:23.665074  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.616269 10462 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.697383  7808 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:23.699512  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:23.713608  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 536) took 1116 ms. Trace:
I20260709 12:26:23.713713  7734 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.677930  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1773) took 4069 ms. Trace:
I20260709 12:26:23.713862  7654 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.715314  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 535) took 1115 ms. Trace:
I20260709 12:26:23.715874  7732 rpcz_store.cc:276] 0709 12:26:22.593302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.593426 (+1000124us) service_pool.cc:224] Handling call
0709 12:26:23.657769 (+ 64343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.708142 (+ 50373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.708377 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.603372 10568 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.714700  7736 rpcz_store.cc:276] 0709 12:26:22.593604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.600058 (+1006454us) service_pool.cc:224] Handling call
0709 12:26:23.679625 (+ 79567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.709728 (+ 30103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.710041 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.713587  7808 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:23.695493  7803 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:23.717840  7662 rpcz_store.cc:276] 0709 12:26:19.608326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.426184 (+2817858us) service_pool.cc:224] Handling call
0709 12:26:22.438306 (+ 12122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.681494 (+243188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.677123 (+995629us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400219dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:23.677654 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553882368}
I20260709 12:26:23.723441  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2013) took 1275 ms. Trace:
I20260709 12:26:23.724016  8412 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:23.723894  7734 rpcz_store.cc:276] 0709 12:26:22.440326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.534912 (+1094586us) service_pool.cc:224] Handling call
0709 12:26:23.601203 (+ 66291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.715405 (+114202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.715718 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:23.725652  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 1993) took 1589 ms (client timeout 1903 ms). Trace:
W20260709 12:26:23.671666 10716 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.358s	user 0.004s	sys 0.002s
I20260709 12:26:23.701318  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:23.726184  7730 rpcz_store.cc:269] 0709 12:26:22.130427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.207117 (+1076690us) service_pool.cc:224] Handling call
0709 12:26:23.709375 (+502258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.696475  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:23.735584  7806 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:23.735903 10580 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.737118 10580 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.688804  8405 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:23.739578 10636 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:23.740505 10636 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.745978  8413 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.746122  7808 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:23.664062 10613 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 12:26:23.747929  7808 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:23.748215  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.749529 10582 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.749902 10613 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:23.751509 10658 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:23.751652 10170 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:23.639034 10508 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.756798 10508 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:23.759622  8415 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.759738  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.750934  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.760548  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:23.728471  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2014) took 1286 ms. Trace:
I20260709 12:26:23.764472  7809 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:23.777647  7732 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.778676  8408 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:23.778872  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2017) took 1036 ms. Trace:
I20260709 12:26:23.788698  7732 rpcz_store.cc:276] 0709 12:26:22.742337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.776466 (+1034129us) service_pool.cc:224] Handling call
0709 12:26:23.777168 (+   702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.778456 (+  1288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.778763 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.779132  7736 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:23.781884  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2012) took 1360 ms. Trace:
I20260709 12:26:23.777365 10636 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.781164  7733 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:23.750861 10613 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:26:23.783648  7728 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.784538  7805 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:26:23.785610  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 152059us
I20260709 12:26:23.797510  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 539) took 1159 ms. Trace:
I20260709 12:26:23.791486  7811 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:23.797935  7728 rpcz_store.cc:276] 0709 12:26:22.638660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.601088 (+962428us) service_pool.cc:224] Handling call
0709 12:26:23.771340 (+170252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.797181 (+ 25841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.797445 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.808571  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2016) took 1061 ms. Trace:
I20260709 12:26:23.809306  7733 rpcz_store.cc:276] 0709 12:26:22.732374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.780230 (+1047856us) service_pool.cc:224] Handling call
0709 12:26:23.780881 (+   651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.793380 (+ 12499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.793617 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.765446  7729 rpcz_store.cc:276] 0709 12:26:22.441595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.544424 (+1102829us) service_pool.cc:224] Handling call
0709 12:26:23.545168 (+   744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.574804 (+ 29636us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:23.575319 (+   515us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.727995 (+152676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.728063 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.728124 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.728216 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.728387 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":930,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":78,"raft.queue_time_us":14948,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":16107,"spinlock_wait_cycles":15449984,"thread_start_us":132556,"threads_started":1,"wal-append.queue_time_us":168230}
W20260709 12:26:23.794185  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 94812.4us
I20260709 12:26:23.659502 10410 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:23.792366  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.811962  7808 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:26:23.790845  7737 rpcz_store.cc:276] 0709 12:26:22.421582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.537471 (+1115889us) service_pool.cc:224] Handling call
0709 12:26:23.552901 (+ 15430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.555413 (+  2512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.555443 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:23.556670 (+  1227us) log.cc:844] Serialized 52 byte log entry
0709 12:26:23.625718 (+ 69048us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:23.625800 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.625915 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.781560 (+155645us) raft_consensus.cc:1685] finished
0709 12:26:23.781587 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.781800 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10312,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":50306,"thread_start_us":46290,"threads_started":1,"wal-append.queue_time_us":169003}
I20260709 12:26:23.794031  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2018) took 1050 ms. Trace:
I20260709 12:26:23.711198  7655 rpcz_store.cc:276] 0709 12:26:21.771841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.192782 (+1420941us) service_pool.cc:224] Handling call
0709 12:26:23.193090 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.193307 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:23.416214 (+222907us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.592751 (+176537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.646337 (+ 53586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.655668 (+  9331us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:23.684082 (+ 28414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.684401 (+   319us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.684618 (+   217us) inbound_call.cc:177] Queueing success response
0709 12:26:23.690181 (+  5563us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
Metrics: {"mutex_wait_us":86252,"prepare.queue_time_us":151671,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":1954,"raft.queue_time_us":50840,"raft.run_cpu_time_us":600,"raft.run_wall_time_us":16211,"spinlock_wait_cycles":29677312,"thread_start_us":285758,"threads_started":2,"wal-append.queue_time_us":121141}
I20260709 12:26:23.818439  7736 rpcz_store.cc:276] 0709 12:26:22.743258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.778187 (+1034929us) service_pool.cc:224] Handling call
0709 12:26:23.778858 (+   671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.792692 (+ 13834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.793935 (+  1243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.815126  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:23.821455 10504 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:23.822049 10504 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.822544 10504 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.823423  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:23.826082  7661 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:23.832924 10410 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:23.834431  7661 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:23.836010 10452 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:23.838089  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:23.838380  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1790) took 1829 ms. Trace:
I20260709 12:26:23.841414  8406 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:26:23.820860  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82797.7us
I20260709 12:26:23.842502  7654 rpcz_store.cc:276] 0709 12:26:22.008730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.499036 (+1490306us) service_pool.cc:224] Handling call
0709 12:26:23.701801 (+202765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.719399 (+ 17598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.838130 (+118731us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540021bbd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:23.838296 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396576128}
I20260709 12:26:23.855655  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:23.846624  7662 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:23.869537  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:23.872134  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:23.836130 10580 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:23.879403 10658 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:23.887992  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.889652  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:23.893756  7661 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:23.896373  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:23.898545  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.271972  7731 rpcz_store.cc:276] 0709 12:26:22.152281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.081355 (+929074us) service_pool.cc:224] Handling call
0709 12:26:23.096755 (+ 15400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.096996 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:23.110829 (+ 13833us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.271113 (+160284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.271194 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.271266 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.271341 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.271479 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13832,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":948,"thread_start_us":158558,"threads_started":1}
I20260709 12:26:23.904031  7655 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:23.905735  7729 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:23.906176 10292 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:23.908748 10734 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:23.909600 10734 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:23.872072 10582 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:23.915545  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 250607us
W20260709 12:26:23.916960  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60362.2us
I20260709 12:26:23.931348  7657 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:23.924252 10292 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:26:23.943161  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2022) took 1183 ms. Trace:
I20260709 12:26:23.943776  7730 rpcz_store.cc:276] 0709 12:26:22.759934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.792075 (+1032141us) service_pool.cc:224] Handling call
0709 12:26:23.824836 (+ 32761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.825522 (+   686us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:23.826059 (+   537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.942767 (+116708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.942826 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:23.942883 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.942960 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.943100 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2047,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"thread_start_us":60558,"threads_started":1}
I20260709 12:26:23.942885  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:23.943711  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51748.7us
W20260709 12:26:23.964884  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51838.4us
W20260709 12:26:23.969194  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.970134  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.997457  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66633.3us
W20260709 12:26:23.993662  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97451.2us
I20260709 12:26:23.930076 10582 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.005563 10756 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:23.996129  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 54036.2us
I20260709 12:26:24.016120 10734 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.018635 10734 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:24.019529 10572 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:24.003564 10170 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:26:24.024614 10170 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:24.025148 10170 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:24.029392 10751 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:24.030082 10416 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:26:24.030282 10751 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:24.031495 10751 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:24.033324  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.033497 10582 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:24.035881  7733 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:24.036813  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:24.037850  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73661.3us
I20260709 12:26:24.038569 10595 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:24.008004 10568 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.040231 10595 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.040231 10568 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.044745  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.045579 10595 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.046999 10636 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:24.047474 10636 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.047817 10636 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.048403 10580 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:24.051267  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.049335  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:24.054915  7728 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:24.062974  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:24.069197  7654 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.069636  7657 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:24.064709 10582 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:26:24.075870  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:24.077368 10706 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:24.078573 10706 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:24.079340 10706 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:24.079852 10706 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:24.083035 10453 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:24.026961 10170 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.088754 10170 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.089576 10568 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:24.092237 10753 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.093667 10753 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.073866  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2021) took 1318 ms. Trace:
I20260709 12:26:24.095902  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 540) took 1438 ms. Trace:
I20260709 12:26:24.095767  7732 rpcz_store.cc:276] 0709 12:26:22.755569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.790489 (+1034920us) service_pool.cc:224] Handling call
0709 12:26:23.864310 (+ 73821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.903142 (+ 38832us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:23.903556 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.073317 (+169761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.073409 (+    92us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:24.073479 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.073578 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.073767 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67875,"prepare.queue_time_us":173,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":101642,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":152001280,"thread_start_us":83441,"threads_started":1,"wal-append.queue_time_us":100005}
I20260709 12:26:24.102280 10462 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:24.116851  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.117524  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1798) took 1623 ms. Trace:
I20260709 12:26:24.100589  7734 rpcz_store.cc:276] 0709 12:26:22.657820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.725182 (+1067362us) service_pool.cc:224] Handling call
0709 12:26:23.782237 (+ 57055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.782864 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.782900 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.796818 (+ 13918us) log.cc:844] Serialized 26 byte log entry
0709 12:26:24.001833 (+205015us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:24.001967 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.002285 (+   318us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.094503 (+ 92218us) raft_consensus.cc:1685] finished
0709 12:26:24.094525 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.095811 (+  1286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5533,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":24957,"thread_start_us":145215,"threads_started":1,"wal-append.queue_time_us":253819}
W20260709 12:26:24.124709  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.114948 10568 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:24.122074 10658 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:24.126343  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 79911.2us
W20260709 12:26:24.139779  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 100576us
I20260709 12:26:24.148739 10749 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:24.149993 10749 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.151407 10568 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:24.151674 10485 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.152467 10485 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.153116 10568 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.155483 10754 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:26:24.157212  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:24.157346 10754 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.128006 10756 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.158903 10706 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.159828 10755 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:24.159621 10756 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.160945 10706 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.161881 10755 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:24.161150 10749 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.162370 10755 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.164626 10755 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.166941 10755 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.167644 10485 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.168500 10485 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:24.170787 10485 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:24.152920 10462 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:24.172536 10749 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:24.120961  7658 rpcz_store.cc:276] 0709 12:26:22.487695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.500447 (+1012752us) service_pool.cc:224] Handling call
0709 12:26:23.537337 (+ 36890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.604717 (+ 67380us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:23.798115 (+193398us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.974367 (+176252us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.110525 (+136158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.110624 (+    99us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:24.110706 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.110809 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.111005 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14676,"prepare.queue_time_us":3702,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":294195,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":236,"thread_start_us":380824,"threads_started":4,"wal-append.queue_time_us":173684}
I20260709 12:26:24.146636 10751 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.185799 10751 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.189205 10504 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.190570 10464 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:26:24.209808  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.214800  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:24.224624  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:24.200541 10464 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:24.226779  7734 raft_consensus.cc:2376] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9. Re-sending same reply.
I20260709 12:26:24.229907 10752 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:24.133381 10753 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.220360 10580 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.240692  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:24.241047 10464 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.242522  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:26:24.243414 10504 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:24.247499  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.258756  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 205776us
I20260709 12:26:24.247925 10752 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:24.249609 10731 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.505s	user 0.004s	sys 0.000s
I20260709 12:26:24.252223 10658 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.282828  7654 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:26:24.290303  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 162630us
I20260709 12:26:24.303699 10464 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.305351  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:24.306356  8412 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.306203 10757 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.316819 10572 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:24.317624 10572 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:24.305279  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:24.323876  8411 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:24.239068 10754 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:26:24.332808 10606 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.338539  8412 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
W20260709 12:26:24.339624  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 108829us
W20260709 12:26:24.341986  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51976.4us
I20260709 12:26:24.349356  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:24.355926 10452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.009s
W20260709 12:26:24.385757  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.375913 10452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.006s	sys 0.010s
I20260709 12:26:24.388389  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.385656 10751 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:24.389063 10754 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:24.378288 10636 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.319087 10572 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.407605 10770 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.427431  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.428756  7806 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:26:24.430544  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:24.433935  7803 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:24.438118  8411 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:24.422660  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 545) took 1209 ms. Trace:
I20260709 12:26:24.411212  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:24.439286  7736 rpcz_store.cc:276] 0709 12:26:23.213822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.847658 (+633836us) service_pool.cc:224] Handling call
0709 12:26:23.912579 (+ 64921us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.964794 (+ 52215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.964845 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:24.008683 (+ 43838us) log.cc:844] Serialized 52 byte log entry
0709 12:26:24.237379 (+228696us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:24.237433 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.237527 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.422316 (+184789us) raft_consensus.cc:1685] finished
0709 12:26:24.422343 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.422541 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65445,"raft.queue_time_us":30377,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":5387,"thread_start_us":152808,"threads_started":1,"wal-append.queue_time_us":328656}
I20260709 12:26:24.394724 10452 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.444205  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:24.448194  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.421489  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.449947 10751 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.459164  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.460538  7803 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 4: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:24.461242 10606 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:24.461992 10606 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.423810 10572 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:24.463128 10606 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.464630  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1778) took 4779 ms. Trace:
W20260709 12:26:24.466599  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45228.9us
W20260709 12:26:24.420579 10730 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.675s	user 0.000s	sys 0.002s
I20260709 12:26:24.405051 10754 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.465243  7656 rpcz_store.cc:276] 0709 12:26:19.654426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.913021 (+3258595us) service_pool.cc:224] Handling call
0709 12:26:23.530134 (+617113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.539557 (+  9423us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:23.540196 (+   639us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.715096 (+174900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.715150 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:23.715189 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.725111 (+  9922us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.419729 (+694618us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:24.433274 (+ 13545us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54004371d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:24.433418 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54268,"prepare.queue_time_us":1059,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":40,"raft.queue_time_us":61202,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":42478,"spinlock_wait_cycles":1348941440,"thread_start_us":113010,"threads_started":1,"wal-append.queue_time_us":204269}
I20260709 12:26:24.411468 10764 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:26:24.426347 10595 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.480233 10595 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.413618 10636 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:24.466892 10170 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:24.467140 10755 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:24.506433  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95400.8us
I20260709 12:26:24.506052  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:24.517181  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45825.9us
I20260709 12:26:24.518251  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:24.519939  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.522493  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.505970 10170 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.519490  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2429) took 1332 ms. Trace:
I20260709 12:26:24.528239  7803 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:26:24.529065 10170 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.535792  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.538686  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:24.540228  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.526505  7804 rpcz_store.cc:276] 0709 12:26:23.184590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.296244 (+111654us) service_pool.cc:224] Handling call
0709 12:26:23.299160 (+  2916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.299442 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:23.630919 (+331477us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.928750 (+297831us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.944605 (+ 15855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.959923 (+ 15318us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.993052 (+ 33129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.008311 (+ 15259us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.091692 (+ 83381us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b18002863c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:24.093078 (+  1386us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b18002863c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1
0709 12:26:24.093604 (+   526us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b18002863c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:24.516460 (+422856us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b54003737d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:24.516615 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":724,"prepare.queue_time_us":354823,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":105,"raft.queue_time_us":14836,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":53086,"spinlock_wait_cycles":34659456,"thread_start_us":396739,"threads_started":2,"wal-append.queue_time_us":264080}
I20260709 12:26:24.550697  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:24.538259 10595 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:24.553440 10754 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.414824 10753 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.561967  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:24.569617 10453 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.577227 10365 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:24.577687  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.557037 10751 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.575001 10636 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:24.581678 10636 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:24.582229 10636 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:24.583323 10292 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:24.584236 10769 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
I20260709 12:26:24.585079 10528 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:24.586410 10528 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:24.586810 10528 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:24.586441  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 543) took 1687 ms. Trace:
I20260709 12:26:24.577898 10365 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:24.588732  7733 rpcz_store.cc:276] 0709 12:26:22.898787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.819822 (+921035us) service_pool.cc:224] Handling call
0709 12:26:23.881573 (+ 61751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.036467 (+154894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.586178 (+549711us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540037e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:24.586378 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353730304}
I20260709 12:26:24.604534  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.606660 10706 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:24.607472  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.609555 10757 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:24.609648  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.610415 10582 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.610464 10757 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:24.612020 10365 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.612864 10292 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:24.612963 10757 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.613593 10365 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:24.587754 10528 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.616149 10292 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.617942  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:24.618647  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.616931 10292 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:24.619961 10742 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.614s	user 0.000s	sys 0.005s
I20260709 12:26:24.556409 10595 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:24.626386  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.631433  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.632941  8412 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:24.633445 10580 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:24.634752 10368 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:24.635280 10368 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.635509 10741 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:24.636170 10741 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:24.636442 10368 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.637660 10368 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.637212 10741 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.639621 10758 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:24.641112 10582 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:24.617053 10528 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.643316 10580 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:24.643874 10580 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:24.644130 10572 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:24.644323 10580 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:24.644596 10572 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:24.645038 10572 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:24.647485 10743 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.641s	user 0.005s	sys 0.000s
I20260709 12:26:24.647660 10480 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.648000  8411 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:24.648322 10480 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.589418 10756 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.651077  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 126725us
I20260709 12:26:24.656337  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:24.627444 10781 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.623425 10706 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:24.652472 10595 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:24.678755  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:24.673446  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:24.679757  8411 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:24.628762 10292 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:24.547132 10452 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.691008  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.691368 10480 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:24.692160 10784 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:24.692994  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.697718  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.686358 10365 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.699836 10365 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.702526 10365 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:24.693354 10568 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.706503  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:24.725374  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45281.2us
I20260709 12:26:24.681599 10443 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:24.678610 10706 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:24.693090 10747 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.688s	user 0.006s	sys 0.000s
I20260709 12:26:24.742496  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:24.737401 10780 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:24.743511  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:24.689059 10764 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:24.753392  8411 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:24.692970 10784 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:24.756392 10784 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:24.751694  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81601.2us
W20260709 12:26:24.760295  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 62696.6us
W20260709 12:26:24.755060  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 151553us
I20260709 12:26:24.708971  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:24.765091  8412 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:24.729439 10752 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:24.750130 10749 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.000s
I20260709 12:26:24.769337  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.688230 10756 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.776700  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:24.777431 10753 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:24.751286 10443 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:24.788424  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45437.4us
W20260709 12:26:24.791823  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.764355 10568 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.797128  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:24.812938  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.820197  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.815171 10749 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.831408  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:24.844214  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.855654  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67996.2us
I20260709 12:26:24.786856 10764 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:24.829795 10734 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.867331  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.868781  7803 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:24.872757 10733 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.857s	user 0.009s	sys 0.000s
W20260709 12:26:24.874187  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97705.6us
I20260709 12:26:24.874585  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1779) took 5206 ms. Trace:
I20260709 12:26:24.875463  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:24.877211 10768 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.475s	user 0.000s	sys 0.003s
I20260709 12:26:24.875396  7653 rpcz_store.cc:276] 0709 12:26:19.668235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.927089 (+3258854us) service_pool.cc:224] Handling call
0709 12:26:22.929336 (+  2247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.935883 (+  6547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.936960 (+  1077us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.939049 (+  2089us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.610463 (+671414us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b18003f5c20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:23.821799 (+211336us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:23.821886 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.822015 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.822551 (+1000536us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:24.873974 (+ 51423us) raft_consensus.cc:1685] finished
0709 12:26:24.874016 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.874262 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":2141,"spinlock_wait_cycles":107372160,"thread_start_us":182375,"threads_started":1,"wal-append.queue_time_us":1061483}
I20260709 12:26:24.868804  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.819051 10792 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:26:24.893537  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.900123 10759 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.905297  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:24.902657  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.888849 10568 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:24.907733  8412 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:24.834558 10170 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.911103  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:24.925756  7808 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:24.866040 10783 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
W20260709 12:26:24.940621  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61379.3us
I20260709 12:26:24.942395  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:24.950305  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.936901 10734 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.963613 10580 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:24.964442 10774 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.374s	user 0.003s	sys 0.000s
I20260709 12:26:24.966449 10580 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:24.966655 10698 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.967371 10698 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.970196 10698 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:24.972575 10170 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:24.973573 10170 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:24.979691 10763 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:26:24.980161 10790 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:26:24.980304 10763 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:24.981608 10763 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.990239 10758 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:24.991978  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 204015us
W20260709 12:26:24.995662  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111814us
I20260709 12:26:24.996738  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:24.998129 10741 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:24.998736 10480 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:25.003036 10791 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:26:25.003996 10528 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:25.004575 10528 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.004699 10764 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.004776  8406 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:25.007284 10754 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:25.008244 10785 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 12:26:25.008482 10365 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:25.009017 10365 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.006539 10528 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.010555 10528 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:25.020555 10795 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:25.022612  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.010124 10365 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.024006 10738 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:25.025458 10365 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:24.948832 10606 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:25.026942 10741 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:24.908541 10443 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.955087 10636 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:24.994809  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:24.193226  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:25.050609  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 108469us
I20260709 12:26:25.027154  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:25.054219  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59502.9us
W20260709 12:26:25.081112  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 216040us
I20260709 12:26:25.051796  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:25.087867  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.090512  8408 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:25.125571  7557 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 179) took 17824 ms (client timeout 15000 ms). Trace:
I20260709 12:26:24.970988 10698 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:26:25.137037  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.050648 10416 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:26:25.035959 10756 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.053457 10794 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:25.047554 10170 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:25.125489 10595 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.143644 10698 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:25.052475  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.052595 10763 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:25.054896 10758 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:25.139500  7777 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.903s (SENT)
I20260709 12:26:25.159380  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:25.038165 10636 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:25.149894 10368 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.003s
W20260709 12:26:25.182713  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.050575 10606 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 12:26:25.027761 10738 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.148s
I20260709 12:26:25.174728 10698 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:25.188076  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 161066us
I20260709 12:26:25.143646 10741 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:25.200067  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.203322  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66390us
I20260709 12:26:25.186544  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.179291 10757 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:25.041684 10572 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.178416 10706 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.228559  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:25.221477  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 62267.3us
W20260709 12:26:25.227084  7777 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.970s (SENT)
I20260709 12:26:25.177589 10170 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:25.235728  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 342507us
I20260709 12:26:25.187593 10758 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.226805 10368 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:25.232513  8406 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:25.238849  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.242506  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:25.246975  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44193.7us
I20260709 12:26:25.238862 10416 tablet_bootstrap.cc:492] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
W20260709 12:26:25.252678  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.238862 10756 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:25.240609 10508 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:25.132815  7557 rpcz_store.cc:269] 0709 12:26:07.301404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.301687 (+   283us) service_pool.cc:224] Handling call
0709 12:26:25.125284 (+17823597us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:18.489145 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:18.662068 (+172923us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:18.662122 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:26:18.662134 (+    12us) tablet.cc:662] Decoding operations
0709 12:26:18.663811 (+  1677us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:18.663878 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:18.663917 (+    39us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:26:18.665419 (+  1502us) tablet.cc:701] Row locks acquired
0709 12:26:18.665473 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:26:18.665677 (+   204us) write_op.cc:270] Start()
0709 12:26:18.665851 (+   174us) write_op.cc:276] Timestamp: P: 1783599978665590 usec, L: 0
0709 12:26:18.665878 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:26:18.666848 (+   970us) log.cc:844] Serialized 6672 byte log entry
0709 12:26:19.919371 (+1252523us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:26:20.835732 (+916361us) op_driver.cc:464] REPLICATION: finished
0709 12:26:21.711859 (+876127us) write_op.cc:301] APPLY: starting
0709 12:26:21.722431 (+ 10572us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:21.778738 (+ 56307us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:21.778789 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:21.869515 (+ 90726us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:21.893938 (+ 24423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:26:21.894012 (+    74us) write_op.cc:312] APPLY: finished
0709 12:26:21.894609 (+   597us) log.cc:844] Serialized 119 byte log entry
0709 12:26:22.543753 (+649144us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:26:22.746599 (+202846us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:22.754661 (+  8062us) write_op.cc:454] Released schema lock
0709 12:26:22.784123 (+ 29462us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:22.885018 (+100895us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:22.885954 (+   936us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11299,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":835370,"apply.run_cpu_time_us":27023,"apply.run_wall_time_us":1097159,"num_ops":17,"prepare.queue_time_us":261706,"prepare.run_cpu_time_us":27368,"prepare.run_wall_time_us":1780844,"raft.queue_time_us":582717,"raft.run_cpu_time_us":46725,"raft.run_wall_time_us":783163,"replication_time_us":2169494,"spinlock_wait_cycles":134803968,"thread_start_us":3634807,"threads_started":5,"wal-append.queue_time_us":2104013,"wal-append.run_cpu_time_us":65468,"wal-append.run_wall_time_us":2694585}]]}
I20260709 12:26:25.262362  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.246553 10368 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:25.266880  7777 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.003s (SENT)
I20260709 12:26:25.258364 10698 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.269289 10698 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:25.271567 10170 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.273195 10170 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:25.275569  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95470.7us
I20260709 12:26:25.276744  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.261968 10757 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.279474 10754 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.283764 10572 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:25.284335 10738 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:26:25.285454  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46706us
I20260709 12:26:25.286415  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.286674 10416 ts_tablet_manager.cc:1403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 1.257s	user 0.038s	sys 0.042s
I20260709 12:26:25.287591  7626 leader_election.cc:400] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:26:25.288072  7626 leader_election.cc:403] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:26:25.267470  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:25.295259  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.324486  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 59320.2us
I20260709 12:26:25.283056  7559 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.328454  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:25.329397  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.266569 10801 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:25.234967 10796 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.331859  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:25.332628  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46351.2us
I20260709 12:26:25.325394  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:25.331423  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44681.5us
I20260709 12:26:25.289475 10368 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:25.340014 10443 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:25.325373 10738 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.340054  7559 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.327858 10757 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:25.329244 10528 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.344129 10741 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:25.332244 10756 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
W20260709 12:26:25.357116  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.358179 10636 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.358470 10706 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.358826 10636 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.341917 10754 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.344373 10794 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:25.341027 10595 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:25.363768  7729 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:25.351579 10757 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.350728 10738 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.365212 10453 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:25.340744 10796 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.365856  7736 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:25.367744 10754 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.368815  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:25.369583  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 88013.8us
I20260709 12:26:25.368872  7559 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:25.364171 10365 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:26:25.371954  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:25.406731  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.407579  7730 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:25.408000 10453 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:25.377941  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 43864.7us
I20260709 12:26:25.371623  7735 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:25.396896  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.372922  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.398537  7728 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:25.401933  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.402488  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144301us
I20260709 12:26:25.409168  7559 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.395671 10528 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.417974  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.419135  7737 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:25.426023  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:25.426769  7736 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:25.432583 10758 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:25.434755 10595 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:25.407385 10788 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.409152 10453 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:25.449445  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52702.1us
I20260709 12:26:25.423797 10789 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.451283  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:25.439658 10741 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:25.466398  7777 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.031s (SENT)
I20260709 12:26:25.467790  7730 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:25.465946 10416 raft_consensus.cc:359] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:25.470865  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.471606 10416 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: FOLLOWER
I20260709 12:26:25.472626 10738 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.466183 10781 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 12:26:25.473738 10803 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:26:25.473572 10770 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:25.473878 10783 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:25.475157 10480 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:25.475456 10805 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:26:25.475267 10606 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:25.477571 10792 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:25.478175  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:25.478389  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.478734 10802 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:26:25.479707 10606 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:25.481855 10606 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:25.446384 10801 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.481232 10595 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.481343 10754 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.480561  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.482700  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:25.483052  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.466609 10799 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:26:25.425544 10528 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.446137 10443 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:25.491933  7733 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:25.493505  7559 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.478277  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:25.502466  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:25.499352 10443 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.507571  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.508004 10757 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:25.508441  7661 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:25.509972  7655 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:25.510268 10528 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.500312 10572 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:25.512096 10508 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:25.512570 10636 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.513441 10741 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.513990 10741 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.515793  7653 raft_consensus.cc:2376] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9. Re-sending same reply.
I20260709 12:26:25.516420 10741 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:25.517233  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69616.3us
I20260709 12:26:25.512197  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:25.519984 10796 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.472638 10416 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.521309  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.522171  7733 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:25.522681 10416 ts_tablet_manager.cc:1434] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.235s	user 0.010s	sys 0.002s
I20260709 12:26:25.523824  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:25.525316  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44948.9us
I20260709 12:26:25.525671  7660 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:25.527459  7559 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:25.534421  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.449901 10788 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.465337 10789 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.484222 10801 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.555390  7559 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 2, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.548050  7656 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:25.507121 10443 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.544204 10170 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:25.559010  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.560103  8412 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:25.533121 10508 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:25.540246 10738 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:25.558518  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.577821  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:25.594576  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.575116  7559 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.629340  7559 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 3, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:25.625984  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64055.5us
I20260709 12:26:25.627395  7656 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:26:25.632356  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 181781us
I20260709 12:26:25.564970 10508 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.638316  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.535951 10368 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.640900  7559 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.564744 10170 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.652071  7656 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:25.574656 10794 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:25.576188 10738 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:25.653591  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 184847us
I20260709 12:26:25.576663  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.564888 10636 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 12:26:25.671593  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.677490  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:25.692720  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.645311 10508 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:25.692916  8415 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:25.693624  8412 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:25.701577  7732 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:25.703140  7729 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:25.638069  8406 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:25.706889  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.709156  7730 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 3.
I20260709 12:26:25.701726  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:25.706694 10453 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.716323  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.716758 10453 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.717271  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:25.717546  7656 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:25.718389  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:25.718616 10764 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.719285 10764 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.719121  7559 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.720667  7653 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:25.721599 10764 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.721262  7559 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 3, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.724571 10453 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.716579  8413 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:25.726999  7737 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:25.728722 10801 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.696903 10814 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.731040 10814 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.733608 10814 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:25.565711 10606 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.670229 10738 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:25.735888  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 102318us
W20260709 12:26:25.746467  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.740253  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 214580us
W20260709 12:26:25.765661  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 127803us
W20260709 12:26:25.768564  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.768390 10170 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.776937  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.750420  7559 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.707772 10794 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.790233  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.795581  7662 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:25.741981 10443 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:25.799613  7559 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 1 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:25.767596 10801 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.812911  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:25.814071  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47225.7us
I20260709 12:26:25.814610  8412 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:25.816139 10443 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.817665 10480 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:25.782848 10368 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:25.818117 10480 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:25.810032  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.819591  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.821798  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:25.809528 10819 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:25.823633  7656 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:25.824155 10819 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.824821 10819 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:25.792604 10768 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:26:25.817231 10443 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.832744  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:25.836992  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:25.792274 10820 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:25.849777  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66981.6us
I20260709 12:26:25.819151 10480 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.856233 10788 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:25.857051 10788 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.857689 10788 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.855562 10453 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:25.859829 10453 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:25.860281 10453 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.861333 10453 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:25.857089  7656 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:25.849872  7736 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:25.858193 10698 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:26:25.869531  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 345797us
I20260709 12:26:25.786595 10734 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:26:25.843956 10783 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:25.876691  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.883997  7656 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
W20260709 12:26:25.888329  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51520.1us
I20260709 12:26:25.891747  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:25.882483  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.716s (SENT)
I20260709 12:26:25.894965  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.865726 10706 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:25.899616  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:25.866575 10480 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:25.909493  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.912541  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:25.913709  8415 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:25.866187 10698 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.831923 10801 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:25.930806  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.923501  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.890200 10794 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:25.890324 10802 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.883708 10783 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:25.889657 10811 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:25.952322  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:25.907194 10814 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:25.973096  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:25.957800  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75358.7us
W20260709 12:26:25.941993 10807 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:25.979156  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:25.940971 10754 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:25.967805  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 84923.3us
W20260709 12:26:25.969816  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.907867 10706 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:25.985584  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 192275us
I20260709 12:26:25.931710 10636 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:26:25.961745 10819 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.005723 10817 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:26.020597  8415 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:26.008368  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:25.964442 10453 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:25.974910 10811 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.940120 10792 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:25.967263 10572 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.982925 10814 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.013031  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:25.983573 10794 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:26.029160  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.839s (SENT)
I20260709 12:26:26.061401  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:25.998982 10758 raft_consensus.cc:2764] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:26:26.110986  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47076.5us
I20260709 12:26:26.116006  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:26.119438  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:26.121474 10606 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.033342 10818 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:26:26.124519 10818 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.125152 10818 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.060364 10814 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.068481 10698 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.113849 10796 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:26.130216 10829 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:26.130872 10829 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:26.131345 10829 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.114113 10757 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.132642 10757 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.130270 10796 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:26.135344 10754 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:26.135515 10796 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.136852 10836 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:26.137259 10796 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.141528 10835 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:26.012816 10816 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.144s
I20260709 12:26:26.148283 10823 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:26.148965 10823 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:26.150465  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.151482  8411 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:26.153069  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.532s (SENT)
I20260709 12:26:26.152742 10783 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.154655 10783 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.154768  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.160089 10706 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.162238 10788 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:26.220023  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.219471  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.164358 10794 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.066987 10572 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:25.965205 10802 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.061964 10528 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:26.177211  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61279.2us
W20260709 12:26:26.240919  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 285541us
I20260709 12:26:26.020009 10820 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:26.018610 10817 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.243979 10817 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:26.035094 10811 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.027966 10770 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:26.209920  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 180813us
I20260709 12:26:26.026887 10789 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:26.200076  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 207108us
W20260709 12:26:26.239351 10830 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:26:26.247036  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 301652us
I20260709 12:26:26.267565  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:26.268602  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:26.190234  7656 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:26.150377 10823 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.234459  8411 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:26.243294 10698 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.188841 10819 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:26.241883 10528 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:26.284402  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.282454  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.208895 10754 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:26.282202  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.304174  7659 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:26.285508  7661 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:26.301364  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.280702 10826 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.246s	user 0.003s	sys 0.001s
I20260709 12:26:26.314779  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.316992  7656 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:26.324903  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 338096us
I20260709 12:26:26.282147 10636 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:26.282403 10758 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:26.281684 10706 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:26.353463  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.310804 10819 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:26.277854 10595 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:26:26.355280  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.367723  7733 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.357230  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.328478 10823 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.366008  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:26.366187  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109035us
I20260709 12:26:26.368955  7737 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:26.369112  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.368916  7659 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:26.338688 10801 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.332973 10757 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.334425  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:26.341033 10636 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:26.340801 10758 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:26.389173  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.343605 10741 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:26.347255 10840 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:26.402031  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67708.9us
I20260709 12:26:26.403296  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:26.398216 10636 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:26.401412 10758 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:26.438720  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.412513 10801 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.458479  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.410167 10528 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.401189 10757 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:26:26.465806  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:26.476910  7808 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:26.477339  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:26.487753  7806 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:26.477775  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:26.489807  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.491493  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.479038  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:26.493932  7809 raft_consensus.cc:2368] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7.
W20260709 12:26:26.485852  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.496599  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.497267  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.497701  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.475291  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:26.499516  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:26:26.459004 10508 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.715s	user 0.005s	sys 0.008s
W20260709 12:26:26.472520 10480 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.003s
W20260709 12:26:26.511407  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.449918 10837 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:26.515656  8411 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:26.424088 10842 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:26.512336  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:26.467509 10843 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:26:26.518965  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:26.528034  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.528371  7805 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:26.273464 10770 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.408282 10840 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.285897 10820 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.461370 10754 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:26.534037  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.468516 10572 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:26.465308 10528 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.527689  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:26.528039 10595 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:26.544661  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.546535  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:26.549469  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.553787  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.840s (SENT)
I20260709 12:26:26.468115 10802 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.590263  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:26.558691  7660 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:26.573127  8415 raft_consensus.cc:2410] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:26.499298 10801 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.581722  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:26.592109  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 188956us
I20260709 12:26:26.592407  7659 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:26.543102  7733 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:26.593338  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.544394 10832 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:26:26.585742  8411 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:26.595815  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.558372  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.596647  7732 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:26.545861 10837 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:26.601969  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.528828 10508 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.607013  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.607903  8412 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:26.613111  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 259209us
I20260709 12:26:26.614334  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:26.615028  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.617300  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:26.559041 10365 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:26.589241 10595 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:26.630410  7661 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:26.600756  8405 pending_rounds.cc:85] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 5
I20260709 12:26:26.636122  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.685098  7732 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.595645 10842 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.600533 10824 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.643275  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:26.635847 10365 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.625794 10734 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.669589 10845 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:26.645141 10794 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:26.719992  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:26.659794  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:26.671393  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 117677us
I20260709 12:26:26.677377  7733 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:26.559046 10819 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.629880  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.637432 10170 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.690136  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:26.690510  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.731943  7735 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:26.732944  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.671685  7672 tablet_copy_service.cc:342] P be9c043c6d424ba1ab8def55dc9bb3e2: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8 received from {username='slave'} at 127.0.0.1:40408
I20260709 12:26:26.735404  7672 tablet_copy_service.cc:434] P be9c043c6d424ba1ab8def55dc9bb3e2: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-59f766c3d13d4360bcdeb1d5f0f318a8 on tablet 59f766c3d13d4360bcdeb1d5f0f318a8 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:26.693666  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.713083 10365 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:26.704577 10851 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:26:26.705693  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1808) took 1172 ms (client timeout 1532 ms). Trace:
W20260709 12:26:26.706709  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63586.9us
I20260709 12:26:26.630167 10781 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:26:26.660257 10850 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:26.686061 10814 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.007s
I20260709 12:26:26.714423  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.719244  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.793617  7736 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.794821  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.795689  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:26.694478  8406 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:26.723572  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.798166  7730 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:26.691883 10453 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.004s
I20260709 12:26:26.725520  7737 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.728229  7729 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:26.649887 10572 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:26:26.660964 10817 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:26.698822  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.738636  7734 raft_consensus.cc:1217] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[6.5-6.5]   Dedup: 6.5->[]
I20260709 12:26:26.741173  7731 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:26.741333  7661 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:26.802862  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:26.742133  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.743799  7659 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:26.740435  7653 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:26.744735  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:26.745116 10853 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:26:26.746368 10859 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:26:26.747767 10847 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:26:26.716181 10845 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.768757  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:26.742002  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 562) took 1173 ms. Trace:
I20260709 12:26:26.773648 10818 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:26.781399  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.798774 10789 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:26.765981 10770 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.686208  8405 pending_rounds.cc:107] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.6
I20260709 12:26:26.763602  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.853094  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.708381 10824 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:26.751904  7660 rpcz_store.cc:269] 0709 12:26:25.533733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.534067 (+   334us) service_pool.cc:224] Handling call
0709 12:26:26.705431 (+1171364us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400436cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:26.705616 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352755968}
I20260709 12:26:26.725849 10794 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:26.765055  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.695480 10754 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:26.791352  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 160473us
I20260709 12:26:26.863358  8379 leader_election.cc:400] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7.
I20260709 12:26:26.863828  8379 leader_election.cc:403] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:26.737993 10836 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:26.695359 10842 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.811365  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.811594  7804 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:26.812985  7734 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:26.721895 10734 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.820607  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.867621  7732 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:26.867719  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:26.822114  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.824585  8409 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:26.830281  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.831439  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.831609  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.790221 10840 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:26.742439 10170 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:26.845599  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:26.848570  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.873109  7731 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:26.851958  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.853204  7735 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:26.768065 10801 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:26.856174  8405 raft_consensus.cc:2917] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Aborting config change with OpId 5.6: Aborted: Op aborted by new leader
W20260709 12:26:26.862468 10855 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:26.898442  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.891129  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.901404  7734 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:26.848067  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.979s (SENT) [suppressed 48 similar messages]
I20260709 12:26:26.853953  7733 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:26.867098  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.868873  7730 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.865880  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.916347  7654 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:26.865880  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:26.917459  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69447.4us
I20260709 12:26:26.917903  7660 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:26.869442  7736 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:26.865818 10794 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.928791  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.929677  7729 raft_consensus.cc:2410] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:26:26.930632  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.931411  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:26.932040  7734 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.869522  7659 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:26.841645  8408 rpcz_store.cc:276] 0709 12:26:25.558834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.558988 (+   154us) service_pool.cc:224] Handling call
0709 12:26:25.559471 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.635166 (+ 75695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.635212 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.653084 (+ 17872us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.934906 (+281822us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:25.934979 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.935102 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.731869 (+796767us) raft_consensus.cc:1685] finished
0709 12:26:26.731891 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.732037 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97389,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9408,"thread_start_us":170420,"threads_started":1,"wal-append.queue_time_us":358557}
I20260709 12:26:26.866241 10836 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:26.800484 10814 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.846349 10845 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.886119 10838 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7.
I20260709 12:26:26.886873  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:26.887437 10841 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 12:26:26.887719 10858 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:26.988935 10858 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.989467 10858 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:26.990185 10862 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 12:26:26.992164 10706 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.001s
I20260709 12:26:26.933558  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:26.994346  7735 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.936944  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.730968 10595 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.941460  7737 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:26.943003  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.007567  7654 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:26.943665  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:27.008489  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 391269us
I20260709 12:26:26.945043  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.010176  7660 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:26:27.010512  7660 pending_rounds.cc:85] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 4
I20260709 12:26:27.010751  7660 pending_rounds.cc:107] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:26:27.010998  7660 raft_consensus.cc:2889] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.737347 10837 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.948549  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:26.897132  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.953963  7736 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:26.954356 10866 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:26.959062 10842 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:26.954684  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:26.872308 10789 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:26.960944 10734 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:26.961747 10833 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:26.963449 10864 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:27.027808  8413 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:26:27.028239  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:27.028622  7737 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:26.942548 10836 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.976678  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.029331  7654 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:27.029994  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:26.928115  8405 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.981020 10636 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
W20260709 12:26:26.932305 10783 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.688s	user 0.015s	sys 0.000s
I20260709 12:26:26.907701 10840 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 12:26:26.981655 10865 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 12:26:26.986987 10824 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:26.879653 10818 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:26:26.909260 10508 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:26.992686  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.001482  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.061357  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.939343  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:26.871244 10741 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:26:26.999095 10757 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.013543  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.079317  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.018133  7729 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:27.079741  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.960458 10738 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.178s	user 0.007s	sys 0.011s
I20260709 12:26:27.019793  7660 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:26:27.080349  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.022493  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:26.964748 10863 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:26:26.968940 10814 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:27.029305  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.032111  7735 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:27.037016  7659 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:27.038055  7662 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:26.954746 10817 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:27.021008 10706 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.108453  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.086627  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.109614  7731 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:27.087680  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:27.110548  7660 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:27.055306  7626 connection.cc:256] RPC call timeout handler was delayed by -1.3874938330000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
I20260709 12:26:27.101869  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.102998  7812 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:27.102926  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.112138  7808 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:27.104394  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.107159  7729 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:27.062486  7661 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:27.114656  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:27.115093  7626 connection.cc:256] RPC call timeout handler was delayed by -1.2254938330002005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 12:26:27.062346 10595 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:27.115383  7810 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:27.119468  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:27.085157 10796 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.942s	user 0.016s	sys 0.000s
W20260709 12:26:27.115711  7626 connection.cc:256] RPC call timeout handler was delayed by -1.2227938330001962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 12:26:27.128458  7626 connection.cc:256] RPC call timeout handler was delayed by -1.1219938330000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.227s for a response
I20260709 12:26:27.105547  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:27.148803  7655 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:27.118338  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.120002  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.120543  7811 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:27.127393  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.117949  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.149947  7729 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:27.150338  7806 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
I20260709 12:26:27.168381  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.055475 10636 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.105248 10606 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.114990 10770 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.176972  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 156565us
I20260709 12:26:27.193821  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.207072  7808 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:27.050076 10833 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.230016  7660 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:27.090227 10738 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.232105  7732 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:27.092640 10757 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:27.187175 10829 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.055s	user 0.009s	sys 0.008s
I20260709 12:26:27.170867  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.119377 10706 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:27.297354  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.237704  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.308961  7811 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
W20260709 12:26:27.309942  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.310119  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:27.271122 10882 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:26:27.263453  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.270864  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.313409  8413 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:27.314831  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.315972  7808 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:27.320413  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.321362  7659 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:27.320472  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.322357  7655 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:27.322180  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:27.323892  8411 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:27.332559 10754 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:27.333251 10754 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.333743 10754 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:27.193528 10606 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.338125 10606 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:27.342614  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.343359  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:27.321375  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 149246us
W20260709 12:26:27.390053  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 154876us
I20260709 12:26:27.392228  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:27.400074  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.540s (SENT)
W20260709 12:26:27.400954  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.235297 10758 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.005s
I20260709 12:26:27.406186  7806 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:27.393649 10365 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.004s
I20260709 12:26:27.434420  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:27.416659  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.155s (SENT)
I20260709 12:26:27.436798  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:27.439810  7807 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:26:27.440351  7807 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 6
I20260709 12:26:27.440623  7807 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:26:27.440918  7807 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:27.441875 10365 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.450708  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.452164  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.453413  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.454468 10706 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:27.455530  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.456908  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.464030  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:27.465034  7656 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:27.231976 10770 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.469897  8413 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:27.472425  7809 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:27.474268  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.475760  7809 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:26:27.476913  7734 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.002s
I20260709 12:26:27.480391  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:27.482565  7808 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:27.493671  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:27.494911  7808 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:26:27.495101  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 151866us
I20260709 12:26:27.434546 10170 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:27.499295  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:27.504078  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 87490us
W20260709 12:26:27.505508  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.158s (SENT)
I20260709 12:26:27.506047  7702 leader_election.cc:400] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:26:27.510504  7702 leader_election.cc:403] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:27.507078  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:27.514395  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:27.465560 10814 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:27.482831 10876 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.392069 10829 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.429993 10758 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.528496  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:27.530298 10874 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:26:27.500463 10741 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.505687 10595 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:27.531634 10770 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:27.519470 10170 raft_consensus.cc:2764] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:27.560763  7656 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:27.561743  7803 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:27.562264  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 241377us
I20260709 12:26:27.564836  8408 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:27.565315  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.567196  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.568986  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.570767  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.571859  7803 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:27.573318  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.574239  7803 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:27.575953  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.577776  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:27.578154  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 568) took 1802 ms (client timeout 1577 ms). Trace:
W20260709 12:26:27.578967  8406 rpcz_store.cc:269] 0709 12:26:25.776085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.791614 (+ 15529us) service_pool.cc:224] Handling call
0709 12:26:27.577767 (+1786153us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54004ba854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:27.577986 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905389312}
W20260709 12:26:27.586494 10869 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.754s	user 0.004s	sys 0.003s
I20260709 12:26:27.597406 10875 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:27.618579  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.633890  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.592464 10572 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.617053  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.630131  7662 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:26:27.643891  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 94177.4us
I20260709 12:26:27.673848  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:27.516103 10817 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:26:27.646118  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:27.688761  7660 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:27.646780  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 95078.2us
I20260709 12:26:27.541731 10595 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.603045  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:27.690285  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.690517  7661 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:27.664749  7735 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:27.531754 10876 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.643368 10528 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:27.662736 10875 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:27.544061 10741 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.710340  7660 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:27.706959  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:27.695478 10595 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:27.724466  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:27.699003 10528 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:27.748531 10528 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:27.748778 10858 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.000s
I20260709 12:26:27.675640 10572 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.719656 10783 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.749593  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.781356  7806 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:27.717530 10636 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:27.724465 10738 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:27.788311  7660 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:27.778615  7807 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:27.788141  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.716382 10754 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.733047 10757 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:27.742720 10833 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:27.793175  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:27.794353  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 288880us
I20260709 12:26:27.797181  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 487) took 1190 ms. Trace:
W20260709 12:26:27.811048  7810 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.691s	user 0.013s	sys 0.000s
I20260709 12:26:27.811923  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:26:27.814102  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.817138  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:27.819644  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:27.820770  7806 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:27.823772 10819 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:27.825785  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101370us
I20260709 12:26:27.827211  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:27.830334  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:27.835907  7810 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:27.837136  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:27.837373  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:27.797760  7805 rpcz_store.cc:276] 0709 12:26:26.606406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.606600 (+   194us) service_pool.cc:224] Handling call
0709 12:26:26.638008 (+ 31408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.638474 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.638519 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.645378 (+  6859us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.810922 (+165544us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:26.943246 (+132324us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.008134 (+ 64888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.008225 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.585130 (+576905us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:27.786479 (+201349us) raft_consensus.cc:1685] finished
0709 12:26:27.796872 (+ 10393us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.797066 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26213,"prepare.queue_time_us":238953,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":78036,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":30354,"spinlock_wait_cycles":67894016,"thread_start_us":218657,"threads_started":2,"wal-append.queue_time_us":159550}
I20260709 12:26:27.856410  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:27.861016  7807 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:27.856837  8407 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:27.785020 10839 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:27.871728  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.872800  7728 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:27.874827  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.796782 10365 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:27.815291 10801 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:27.882620  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:27.888652  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:27.889873  7660 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:27.780794 10783 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.891251  7806 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:27.888660  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:27.781208 10858 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.817113 10879 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:26:27.834379 10880 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.832301 10368 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:27.864316 10170 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:27.803283 10849 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:27.753026 10875 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.835183 10814 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:26:27.864758 10819 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:27.912004  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:27.907831  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:27.950832  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:27.961643  7660 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:27.887841 10757 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.890081 10817 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:27.960371  7730 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:27.896477  7653 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:27.895314 10839 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:27.867245 10606 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:26:27.866969 10770 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 12:26:27.945101 10829 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:27.986992  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 149927us
I20260709 12:26:27.909451 10706 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:27.878407 10892 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:26:27.912533 10781 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:26:27.948657 10365 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:26:27.867698 10837 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.012439  7810 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:27.986238 10875 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:28.009812  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 142924us
I20260709 12:26:27.985631 10888 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.036152  7661 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:27.950043 10368 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:27.940212 10893 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:26:28.039746 10837 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:27.992137 10849 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:28.011893 10817 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:27.991396 10896 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:27.940872 10880 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.057971  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:28.033900 10781 consensus_queue.cc:1243] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:26:28.050320 10881 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:26:27.897056 10792 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.165s
I20260709 12:26:27.896574 10818 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.162s
W20260709 12:26:28.081980  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 170854us
W20260709 12:26:28.063102 10900 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:28.057816 10741 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.068084 10817 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.116747  7656 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:26:28.091210  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.081751 10814 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:28.030601 10606 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:28.077412  7662 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:28.057730 10849 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:28.080539 10886 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 12:26:27.998364 10757 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.112882  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.099416  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2053) took 1314 ms. Trace:
I20260709 12:26:28.141548  7733 rpcz_store.cc:276] 0709 12:26:26.784961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.929621 (+144660us) service_pool.cc:224] Handling call
0709 12:26:26.948270 (+ 18649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.950483 (+  2213us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:27.057311 (+106828us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.576397 (+519086us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b18003fe6e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:26:27.779829 (+203432us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.782979 (+  3150us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.783243 (+   264us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:27.783302 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.783408 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.098972 (+315564us) raft_consensus.cc:1685] finished
0709 12:26:28.099103 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.099331 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11685,"prepare.queue_time_us":141710,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":43369,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":25271,"spinlock_wait_cycles":18716544,"thread_start_us":272137,"threads_started":2,"wal-append.queue_time_us":914026}
I20260709 12:26:28.128539 10606 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:28.105147 10572 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.144424 10606 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.155753 10783 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.120494 10817 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:28.053148 10888 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:28.192209 10903 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:26:28.108218 10839 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.279618  8406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:28.280652  8406 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.281136  8406 raft_consensus.cc:614] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:28.283088  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:28.080104 10896 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.250694 10528 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.287547 10896 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.135330 10849 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:28.208330 10845 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.224s	user 0.012s	sys 0.002s
I20260709 12:26:28.211553  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 828) took 1200 ms. Trace:
I20260709 12:26:28.123170 10814 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:28.289714  8409 rpcz_store.cc:276] 0709 12:26:27.011145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.011353 (+   208us) service_pool.cc:224] Handling call
0709 12:26:27.032194 (+ 20841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.032529 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.032554 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.071449 (+ 38895us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.313051 (+241602us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:27.313114 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.313197 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.206750 (+893553us) raft_consensus.cc:1685] finished
0709 12:26:28.206779 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.211461 (+  4682us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":241061,"threads_started":1,"wal-append.queue_time_us":432275}
I20260709 12:26:28.233076  7662 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:26:28.234524 10811 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.966s	user 0.010s	sys 0.007s
I20260709 12:26:28.299664  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.300836  7656 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:28.235875  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 571) took 1165 ms. Trace:
I20260709 12:26:28.297042  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:28.236611 10794 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.288s	user 0.011s	sys 0.005s
I20260709 12:26:28.240305 10368 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:28.241266 10887 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 12:26:28.214788 10836 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.161s	user 0.009s	sys 0.005s
I20260709 12:26:28.245703 10606 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.247591 10837 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:26:28.369954  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.371101  7656 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 3.
I20260709 12:26:28.276854  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:28.278293 10858 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.279078  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:28.290344  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:28.373164  7733 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:28.372740  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 332522us
I20260709 12:26:28.299842 10792 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.291180  7660 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:26:28.250527 10595 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.002s
I20260709 12:26:28.298111  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:28.251183 10889 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.372s	user 0.001s	sys 0.003s
W20260709 12:26:28.251492 10891 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.467s	user 0.005s	sys 0.000s
I20260709 12:26:28.202895 10880 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.305842  7661 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:28.311297  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:28.401650  7732 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:28.307596  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:28.322060  7734 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:28.301307  8414 rpcz_store.cc:276] 0709 12:26:27.069925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.076486 (+  6561us) service_pool.cc:224] Handling call
0709 12:26:27.076895 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.077360 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.077391 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.077887 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.500966 (+423079us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:27.501029 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.501153 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.235478 (+734325us) raft_consensus.cc:1685] finished
0709 12:26:28.235505 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.235696 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2102,"raft.queue_time_us":6235,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":11528,"thread_start_us":422569,"threads_started":1,"wal-append.queue_time_us":592661}
I20260709 12:26:28.362707  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:28.363175  8412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.011s	sys 0.053s
W20260709 12:26:28.367123  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60375.9us
W20260709 12:26:28.403928  8412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.011s	sys 0.053s
W20260709 12:26:28.291113  7804 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.833s	user 0.007s	sys 0.000s
I20260709 12:26:28.404474  7804 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:28.374868  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.405047  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1988) took 1120 ms. Trace:
I20260709 12:26:28.405669  7804 rpcz_store.cc:276] 0709 12:26:27.285312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.456614 (+171302us) service_pool.cc:224] Handling call
0709 12:26:28.404893 (+948279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:28.372658  7728 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:28.384100  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86053.8us
I20260709 12:26:28.406515  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.306126 10911 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.388456  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:28.408525  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:28.414999  7731 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:28.416654  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.299727 10845 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.402459 10741 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:28.369992 10836 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.433071  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.434985 10757 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.493232  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 566) took 1418 ms. Trace:
I20260709 12:26:28.435843  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:28.496279  8407 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:28.395792 10528 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:28.436048  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.437137  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:28.498023  8409 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:28.412427 10858 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.396158 10595 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.439697 10781 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:28.440253 10910 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:26:28.441519 10606 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:28.414790 10814 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.469205 10741 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.369680 10368 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:26:28.308446 10811 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.443640  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.391429 10792 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.307191 10794 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:28.451845  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:28.465317  7804 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:28.467051  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2064) took 1472 ms. Trace:
I20260709 12:26:28.471377  8414 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:28.466799  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:28.478681  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.493860  7658 rpcz_store.cc:276] 0709 12:26:27.074457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.078538 (+  4081us) service_pool.cc:224] Handling call
0709 12:26:27.121280 (+ 42742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.122088 (+   808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.122140 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.171473 (+ 49333us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.667536 (+496063us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:27.667599 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.667712 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.492886 (+825174us) raft_consensus.cc:1685] finished
0709 12:26:28.492920 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.493137 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6090,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15581,"thread_start_us":474381,"threads_started":1,"wal-append.queue_time_us":640774}
I20260709 12:26:28.503151  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:28.511377  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75412.4us
I20260709 12:26:28.442288 10818 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.851300 10817 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.003s
I20260709 12:26:28.430444 10911 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:28.591142  7654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.012s	sys 0.029s
W20260709 12:26:28.901598  7654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.015s	sys 0.052s
W20260709 12:26:28.843899  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.009s	sys 0.010s
W20260709 12:26:28.902685  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.009s	sys 0.015s
W20260709 12:26:28.903074  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 605158us
W20260709 12:26:28.664384 10850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.017s	sys 0.029s
I20260709 12:26:28.842111 10907 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:28.585641  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.068s	sys 0.089s
W20260709 12:26:28.628528 10823 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 2.251s	user 0.006s	sys 0.020s
I20260709 12:26:28.677147 10758 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.600942  7810 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:28.911810  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2000) took 1140 ms. Trace:
I20260709 12:26:28.501466 10781 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.593380  7736 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:28.725577 10892 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:28.600443  7812 raft_consensus.cc:2368] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:28.827858  8407 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:28.492406 10896 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:26:28.560369 10606 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:28.561779  7804 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:26:28.931922  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2001) took 1160 ms. Trace:
I20260709 12:26:28.932328  7804 rpcz_store.cc:276] 0709 12:26:27.771950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.554715 (+782765us) service_pool.cc:224] Handling call
0709 12:26:28.561139 (+  6424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.931578 (+370439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.931856 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:28.667199 10894 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:26:28.674883 10875 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.000s
I20260709 12:26:28.643152 10888 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.686533 10794 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.687948 10811 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:28.501734 10858 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.688745 10836 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.662108 10912 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.838313  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.682138 10879 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
I20260709 12:26:28.572681 10770 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:28.645648  7733 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:28.680811 10792 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:28.829030 10915 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.310s	user 0.002s	sys 0.005s
I20260709 12:26:28.644234 10910 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.683050 10845 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.573194 10837 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:28.515092 10741 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:28.684211 10789 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:28.642784 10820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.006s	sys 0.038s
I20260709 12:26:28.519312  7730 rpcz_store.cc:276] 0709 12:26:26.994770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.089070 (+ 94300us) service_pool.cc:224] Handling call
0709 12:26:27.106366 (+ 17296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.963212 (+856846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.455066 (+491854us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5400348854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:28.466979 (+ 11913us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963538176}
I20260709 12:26:28.685065 10595 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.874544  8414 raft_consensus.cc:606] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:28.853147  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.683342 10914 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:28.891021  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 412455us
I20260709 12:26:29.037328  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:28.859848 10899 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.286s	user 0.000s	sys 0.002s
W20260709 12:26:28.876562 10839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.010s	sys 0.026s
W20260709 12:26:28.904906  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.019s	sys 0.022s
W20260709 12:26:28.909600  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.081s	sys 0.113s
W20260709 12:26:29.046041  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.019s	sys 0.022s
W20260709 12:26:29.046932  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 749925us
I20260709 12:26:28.972247 10879 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:26:29.047992  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:28.912638  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 98308.4us
I20260709 12:26:29.050469  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:28.912258  7810 rpcz_store.cc:276] 0709 12:26:27.771387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.519191 (+747804us) service_pool.cc:224] Handling call
0709 12:26:28.597228 (+ 78037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.911520 (+314292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.911741 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:28.923908 10911 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.900164 10734 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:29.064940 10734 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:29.065466 10734 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.064981 10888 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:29.066253 10888 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:28.908891 10850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.025s	sys 0.046s
I20260709 12:26:28.914146 10796 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.067282 10888 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:29.068149 10917 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:26:29.067963 10796 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.911859 10921 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.913887 10781 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.069365 10921 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.895138 10818 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.072080 10921 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.072382 10818 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:28.899694 10817 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.034143 10836 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:29.074810 10817 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.074759 10836 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.077158 10836 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:28.958590  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:28.960633 10840 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.078608 10840 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.079053 10840 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:29.079756 10920 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 12:26:29.079582 10734 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.081673  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:28.942410 10925 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.083369 10875 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.083865 10875 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.083570 10925 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.084338 10875 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.084859 10781 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.086333  7805 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:29.032655 10820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.340s	user 0.006s	sys 0.040s
I20260709 12:26:29.092934  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:28.939435 10907 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:28.956400 10606 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.956022 10770 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:29.031883 10794 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:29.047993  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:29.052623 10935 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:29.049839  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:28.965224 10912 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:29.187265  7812 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1982) took 1988 ms (client timeout 1784 ms). Trace:
I20260709 12:26:28.984082 10910 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:29.045727 10839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.010s	sys 0.026s
I20260709 12:26:28.916736 10928 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.087939  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:29.199808 10734 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.201759 10781 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.098203 10914 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:29.202543 10921 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.955425  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 488739us
I20260709 12:26:29.202965 10914 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.205063  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:29.097940 10916 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:29.206085  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:29.105298 10850 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.206600 10814 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:29.207037 10850 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:29.039996 10894 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:29.207783  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2181) took 1700 ms. Trace:
I20260709 12:26:29.207872 10734 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:29.208829 10824 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:29.209246  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 821) took 2708 ms. Trace:
I20260709 12:26:29.209479 10850 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.210240 10850 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.034428 10837 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.095661  7811 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:29.086269 10925 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.032259 10923 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.100075  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 141594us
I20260709 12:26:28.933784 10892 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.106803  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:29.209702  8405 rpcz_store.cc:276] 0709 12:26:26.501022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.506081 (+  5059us) service_pool.cc:224] Handling call
0709 12:26:26.580941 (+ 74860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.169543 (+588602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.182846 (+ 13303us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.389882 (+207036us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.789254 (+399372us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:28.029803 (+240549us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.052278 (+ 22475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.065748 (+ 13470us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.208958 (+1143210us) raft_consensus.cc:1685] finished
0709 12:26:29.208991 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.209158 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74741,"prepare.queue_time_us":198770,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":51321,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":53594,"thread_start_us":380972,"threads_started":2,"wal-append.queue_time_us":1073806}
W20260709 12:26:29.123045  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79308us
I20260709 12:26:29.141947 10754 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:29.152513 10835 raft_consensus.cc:670] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:29.154295  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 683) took 1652 ms. Trace:
I20260709 12:26:29.154647 10706 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:29.127058  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:29.158511 10876 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.159896 10758 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:26:29.164834 10901 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.145s	user 0.000s	sys 0.006s
W20260709 12:26:29.186124  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136299us
I20260709 12:26:29.219547 10734 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.187916  7812 rpcz_store.cc:269] 0709 12:26:27.107983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.633426 (+525443us) service_pool.cc:224] Handling call
0709 12:26:29.096442 (+1463016us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54003723d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:29.096679 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2143327616}
W20260709 12:26:29.191035  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 143149us
I20260709 12:26:29.192050 10911 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:29.196058  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.385456  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.226622  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2062) took 2232 ms. Trace:
I20260709 12:26:29.386770  7736 rpcz_store.cc:276] 0709 12:26:26.994055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.079574 (+ 85519us) service_pool.cc:224] Handling call
0709 12:26:27.108784 (+ 29210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.912993 (+1804209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.226101 (+313108us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540037cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:29.226449 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3413663616}
I20260709 12:26:29.216310  7737 pending_rounds.cc:85] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:26:29.388346  7737 pending_rounds.cc:107] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:26:29.388649  7737 raft_consensus.cc:2889] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:29.388715  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:29.310197  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.248835  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:29.229987  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 575) took 1348 ms (client timeout 1577 ms). Trace:
I20260709 12:26:29.251835  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.248631  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.391546  8409 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:29.349941 10905 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:29.208410  7653 rpcz_store.cc:276] 0709 12:26:27.507799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.521978 (+ 14179us) service_pool.cc:224] Handling call
0709 12:26:27.525080 (+  3102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.993316 (+468236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.207479 (+1214163us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540033e0d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:29.207711 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724566400}
I20260709 12:26:29.250648  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.258121  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2464) took 1691 ms. Trace:
I20260709 12:26:29.400902  7805 rpcz_store.cc:276] 0709 12:26:27.566295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.515885 (+949590us) service_pool.cc:224] Handling call
0709 12:26:28.564868 (+ 48983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.086808 (+521940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.257617 (+170809us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:29.257991 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198813440}
I20260709 12:26:29.323772  7809 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:29.113081 10170 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:29.186826 10820 raft_consensus.cc:993] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
W20260709 12:26:29.335247  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 186381us
I20260709 12:26:29.100003 10933 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:29.106325 10770 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.336977  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:29.428227  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91386.6us
W20260709 12:26:29.344445  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 1990) took 2050 ms (client timeout 1516 ms). Trace:
I20260709 12:26:29.348242  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 494) took 1089 ms. Trace:
W20260709 12:26:29.242514 10849 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.954s	user 0.010s	sys 0.002s
I20260709 12:26:29.349993  8411 rpcz_store.cc:276] 0709 12:26:27.502381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.502695 (+   314us) service_pool.cc:224] Handling call
0709 12:26:27.526161 (+ 23466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.886542 (+1360381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.153981 (+267439us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:29.154218 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246266624}
I20260709 12:26:29.196030 10896 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:29.107650 10936 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:29.209482 10935 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:29.227043 10819 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:29.219372 10528 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.378726  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:29.051453 10368 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:26:29.238384 10912 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.235635 10910 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.251293  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:29.206640 10916 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:29.212409 10858 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.209290 10824 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:29.206108 10914 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:29.383913  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.339097 10942 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:29.511608 10942 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.509078 10754 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.515048  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2183) took 1870 ms. Trace:
I20260709 12:26:29.515165 10754 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.518380 10754 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.515837  7654 rpcz_store.cc:276] 0709 12:26:27.644942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.670816 (+ 25874us) service_pool.cc:224] Handling call
0709 12:26:27.693330 (+ 22514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:27.710199 (+ 16869us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:28.902067 (+1191868us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.328244 (+426177us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:29.344147 (+ 15903us) log.cc:844] Serialized 52 byte log entry
0709 12:26:29.344305 (+   158us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:29.344582 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.356491 (+ 11909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.356619 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.514726 (+158107us) raft_consensus.cc:1685] finished
0709 12:26:29.514760 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.514959 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5187,"prepare.queue_time_us":344440,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":133,"raft.queue_time_us":40478,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":21281,"spinlock_wait_cycles":21606272,"thread_start_us":1528634,"threads_started":2,"wal-append.queue_time_us":480494}
W20260709 12:26:29.390758  8413 rpcz_store.cc:269] 0709 12:26:27.882155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.882313 (+   158us) service_pool.cc:224] Handling call
0709 12:26:29.229655 (+1347342us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54004ba854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:29.229876 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079849216}
I20260709 12:26:29.391168  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:29.106505 10606 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.390889  8412 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:29.377580 10758 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:29.532874  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.190557 10794 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:29.406145  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 96077.4us
I20260709 12:26:29.211091 10850 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.508589 10824 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:29.416234  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:29.419754  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:29.250505 10840 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.221403 10921 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.424827  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:29.434633  7804 rpcz_store.cc:276] 0709 12:26:28.258901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.942162 (+683261us) service_pool.cc:224] Handling call
0709 12:26:28.942515 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.942746 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:29.105022 (+162276us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.347088 (+242066us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.347826 (+   738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.347890 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:29.347939 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.348006 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.348148 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21849,"prepare.queue_time_us":15337,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":130,"raft.queue_time_us":25616,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":174,"thread_start_us":378121,"threads_started":2}
I20260709 12:26:29.384989 10911 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.559868 10934 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:29.560283 10911 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.560657 10934 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:29.431524  7811 rpcz_store.cc:269] 0709 12:26:27.294322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.309773 (+ 15451us) service_pool.cc:224] Handling call
0709 12:26:29.344251 (+2034478us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:29.344370 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4043574016}
I20260709 12:26:29.392532 10781 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.442169  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.436828 10839 raft_consensus.cc:1081] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:26:29.447860  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:29.404189 10170 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:29.400067 10905 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:29.463173  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70610.1us
I20260709 12:26:29.464730  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:29.465095 10819 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:29.465792  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 87236.6us
I20260709 12:26:29.466388 10595 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.465343 10896 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:29.464277  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 112066us
I20260709 12:26:29.450901 10849 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:29.236037 10928 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.538288 10858 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.450997 10936 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.338842 10923 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.508302 10875 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.473644  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:29.567401  7659 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:29.392561 10734 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.444001 10836 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:29.478165  7662 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:29.580209  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:29.479112  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 1995) took 2182 ms. Trace:
I20260709 12:26:29.510344  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:29.510695 10837 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.607301 10942 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.580461 10595 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.607710 10936 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:29.215813  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:29.625504 10892 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:29.525560  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 141728us
I20260709 12:26:29.525365  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:29.627555  7654 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.627877  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.585631 10850 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:29.373567 10706 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.569260  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.632231  8411 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:29.473343  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
W20260709 12:26:29.632943 10953 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.083s	user 0.003s	sys 0.000s
I20260709 12:26:29.633306  8405 raft_consensus.cc:606] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.471360 10950 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:29.536298  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 492) took 1278 ms. Trace:
I20260709 12:26:29.564101 10781 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.634778  7806 rpcz_store.cc:276] 0709 12:26:28.258275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.712621 (+454346us) service_pool.cc:224] Handling call
0709 12:26:28.734944 (+ 22323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.735549 (+   605us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:28.735594 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.110468 (+374874us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:26:29.111208 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.383892 (+272684us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:29.383943 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.384018 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.520266 (+136248us) raft_consensus.cc:1685] finished
0709 12:26:29.520296 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.536182 (+ 15886us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":215120,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5253,"spinlock_wait_cycles":53944192,"thread_start_us":272166,"threads_started":1,"wal-append.queue_time_us":396296}
I20260709 12:26:29.496222 10796 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.481626 10907 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.553253  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:29.533653  8406 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:29.610344 10858 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.640206 10858 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:29.445315 10933 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.523586 10910 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:29.473902 10528 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.473196  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:29.611171  7805 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:29.572606  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.586253  7731 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:29.587406  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:29.698638  7736 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:29.587790 10819 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.563643 10758 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.586999  7734 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:29.597486 10928 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.584410 10170 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.565066 10606 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:29.600579  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2068) took 1730 ms (client timeout 1774 ms). Trace:
I20260709 12:26:29.618400 10923 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.593765 10849 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.580647 10888 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:26:29.620992 10879 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:29.624442 10595 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.636564 10781 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.520637 10951 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:26:29.565668 10840 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.642114  8409 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:29.585286  7656 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:29.587733  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:29.539669  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.618762 10734 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.614441  7808 rpcz_store.cc:276] 0709 12:26:27.296231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.501569 (+205338us) service_pool.cc:224] Handling call
0709 12:26:27.501984 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.542614 (+ 40630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.542646 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.543364 (+   718us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.838096 (+294732us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:27.838140 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.838229 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.163990 (+1325761us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:29.471728 (+307738us) raft_consensus.cc:1685] finished
0709 12:26:29.471754 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.478999 (+  7245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47876,"raft.queue_time_us":20766,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":11824,"spinlock_wait_cycles":33792896,"thread_start_us":168423,"threads_started":1,"wal-append.queue_time_us":387777}
I20260709 12:26:29.650022  7803 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:29.604708  7700 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.577s (SENT)
I20260709 12:26:29.573072  7810 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:29.652732  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:29.605890  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.744539  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 139879us
W20260709 12:26:29.574882 10741 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.002s
W20260709 12:26:29.675366  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 122243us
I20260709 12:26:29.678279  7659 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:29.686739  7654 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:29.703495  8408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:29.648941 10875 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:29.642168 10921 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.802918  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:29.714489  7735 rpcz_store.cc:269] 0709 12:26:27.870179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.870366 (+   187us) service_pool.cc:224] Handling call
0709 12:26:29.600332 (+1729966us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002486d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:29.600487 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3209161600}
I20260709 12:26:29.660874 10801 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:29.563169 10934 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:29.704600 10796 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.740483  7729 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:29.802747  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.728277 10758 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:29.804414  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 265002us
I20260709 12:26:29.696938 10925 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:26:29.739444 10170 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.708482 10792 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:26:29.813323  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.814473 10796 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.644814 10905 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.581516 10896 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.842166  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:29.843297  8412 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:29.839396  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:29.809473 10948 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:29.849822  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.850570  8409 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:29.810457 10811 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:29.857458  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.813787 10935 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:29.828452 10796 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.837704 10606 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:29.875342  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:29.575384 10916 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.807462 10741 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:29.841500 10794 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.868472  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.821784 10921 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:26:29.856247 10905 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:29.889884  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 416951us
I20260709 12:26:29.881477 10951 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:29.896447  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:29.920401  7735 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:29.856899 10811 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.888969 10875 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:29.906291 10706 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.828620 10923 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:26:29.843937 10907 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:26:29.845782 10950 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:29.913470  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:29.864176 10876 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:29.930699  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:29.949187  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:29.890794 10606 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:29.931784 10949 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:26:29.939363 10892 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:29.940505 10959 raft_consensus.cc:993] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:26:29.914738 10754 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.943099  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:29.946727  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 245405us
I20260709 12:26:29.966678  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:29.955055  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152977us
I20260709 12:26:29.927155 10811 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:29.885619 10839 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:29.926422 10875 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:29.906836 10935 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:29.969748  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 354965us
I20260709 12:26:29.968278  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:29.972975  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:29.975871  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:29.977115  8411 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:29.978989  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:29.988114  8411 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:29.937565 10907 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:29.953822 10925 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.942360 10706 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.995182  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:29.962859 10837 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:29.966377 10947 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:30.070920  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:30.072572  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 90156.8us
W20260709 12:26:29.999734  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.073240  8414 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:29.999889  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:30.077080 10944 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:26:30.070628 10734 raft_consensus.cc:1081] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:26:29.998150 10933 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:29.952385 10950 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.995463 10892 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:26:30.070816 10845 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:30.071826 10967 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:26:29.971546 10875 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:30.129504 10899 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:26:30.098455 10837 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.082283 10925 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:29.972347 10937 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:30.080139 10960 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 12:26:29.965713 10754 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.098356  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 103272us
I20260709 12:26:30.010926  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:30.097589 10968 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:26:30.129417  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.975401 10934 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:30.095940 10770 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
I20260709 12:26:30.130658  7702 leader_election.cc:400] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:30.184144  7702 leader_election.cc:403] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.074903  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:29.981598 10819 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:29.998225 10858 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:30.128217 10824 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.004s
W20260709 12:26:30.131381 10917 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:30.071031 10801 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:26:30.129568 10958 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:26:29.996457 10912 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.167s
I20260709 12:26:30.094846 10757 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:26:30.128306 10914 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:30.173897 10837 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:30.130232 10947 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:30.183158  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:30.176994 10706 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:30.191711  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.160458 10845 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.202972  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72539.8us
I20260709 12:26:30.182114 10754 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:30.215561  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 204728us
I20260709 12:26:30.183123 10934 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.226717  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:30.223387 10947 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:30.204807 10824 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:30.231881  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48800.9us
I20260709 12:26:30.155845 10606 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.201700 10968 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:30.187418 10933 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:26:30.247443  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.248298  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:30.248868  7654 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:30.207540 10741 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:30.196430 10770 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.250728  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.231998 10971 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:30.232709 10845 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:30.254006  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.255679  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.279866  7659 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:30.256896  7662 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:30.282536  7661 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:30.284289  8407 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:30.284626  7729 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:30.266595 10950 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:30.237313 10910 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:30.267835 10770 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:30.243826 10781 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:30.269260 10911 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.001s
I20260709 12:26:30.271672 10933 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:30.272567 10972 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:26:30.225335 10801 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:30.237890 10934 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:30.278746 10971 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.248831 10967 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:26:30.245838 10968 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:30.267330  7655 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:30.278867  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:30.268905  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.303503  7654 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:30.341274  7653 pending_rounds.cc:85] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 1
I20260709 12:26:30.305557 10734 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:30.316220 10849 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:30.320235  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:30.321097  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.320937 10959 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:30.320993 10925 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:30.323211 10966 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:26:30.305585 10933 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:30.324630  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.330693  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121942us
I20260709 12:26:30.341737  7653 pending_rounds.cc:107] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:26:30.353186  7653 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:30.337817  8406 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.001s
I20260709 12:26:30.304780 10910 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.343264  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.369835  7654 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:30.356002  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 357176us
I20260709 12:26:30.357733  7657 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:30.373799  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.375285  7729 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:30.378100  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 839) took 1129 ms. Trace:
I20260709 12:26:30.339550 10971 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:30.378787  8413 rpcz_store.cc:276] 0709 12:26:29.248657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.533073 (+284416us) service_pool.cc:224] Handling call
0709 12:26:29.542446 (+  9373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.542697 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:30.155914 (+613217us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b180032de80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:30.156109 (+   195us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.377592 (+221483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.377655 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:30.377705 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.377780 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.377994 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3944,"prepare.queue_time_us":297003,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"spinlock_wait_cycles":14030592,"thread_start_us":514127,"threads_started":2}
I20260709 12:26:30.380096  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:30.388597  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.360011  8406 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:30.349702 10849 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:30.395316 10910 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:30.397188  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:30.399520  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79404.4us
I20260709 12:26:30.401187  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:30.355517 10595 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.001s
I20260709 12:26:30.406785  7729 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:30.340451 10858 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:30.354666 10757 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:30.310495 10781 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.308115 10911 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.415985  7657 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:30.416646  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 416821us
I20260709 12:26:30.416033  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:30.424772  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:30.426163  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 199527us
I20260709 12:26:30.401043 10706 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:30.493270  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72468.9us
I20260709 12:26:30.409879 10835 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.398694 10979 raft_consensus.cc:2764] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:26:30.501216  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.489845  7734 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:30.490767  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.425772 10923 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:30.494045  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78194.9us
I20260709 12:26:30.553267  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.554114  7736 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:30.423367 10770 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.496271  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.550637  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:30.496866  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:30.501631  8408 raft_consensus.cc:606] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:30.552497  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:30.457769  7729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:53564
I20260709 12:26:30.502206  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.558233  7729 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:30.558804  7729 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:30.559820  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.560535  7728 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:30.423817  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 835) took 1176 ms. Trace:
I20260709 12:26:30.503322  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.506579  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.509176  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.562093  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.562361  7812 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:30.509300  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.563537  7806 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:30.509423  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.510293  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.511112  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.512323  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.567046  7811 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.513140  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.568521  7809 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.513855  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.514624  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.527501  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:30.569872  7807 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.570232  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:30.528926  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99973.5us
I20260709 12:26:30.575310  7654 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:30.577338  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.424955 10595 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.581022  7659 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:30.583765  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.591830  7731 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:30.545141 10988 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:30.595901 10849 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.596521 10988 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.573457  8406 rpcz_store.cc:276] 0709 12:26:29.247271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.250920 (+  3649us) service_pool.cc:224] Handling call
0709 12:26:30.416849 (+1165929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.600162 10910 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.600831  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.600771 10910 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.601822  7735 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:30.602572 10811 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 12:26:30.602806  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.616151  7735 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:30.603765  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.603477 10811 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.604142 10888 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:30.607652  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.609045  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.629505  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:30.630306  7803 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.610777  7808 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:30.610489 10875 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.612583 10911 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:30.632414  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.612854 10528 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:30.617987  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:30.621755  7655 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:30.634088 10528 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:30.634668 10528 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:30.622958  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.555060 10734 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:30.626268  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.551801 10963 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:30.631028  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.640173  7728 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:30.642663  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 145855us
I20260709 12:26:30.550853 10757 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:30.664535  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:30.619688 10986 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:26:30.628459 10966 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:26:30.647313  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.667786  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.678900  7803 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:30.648181 10758 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.820s	user 0.009s	sys 0.004s
I20260709 12:26:30.686851  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.629212 10973 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:30.691213  7735 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:30.690016  7729 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.698531  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:30.599139 10988 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:30.697245  7812 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.551743 10835 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.573722 10923 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.580120 10770 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.638582 10948 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:30.434303 10968 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.715418  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.789490  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.635739 10528 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.700258  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.883760  7805 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:30.721577  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.884977  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.787187  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 574) took 1399 ms. Trace:
I20260709 12:26:30.885704  7655 rpcz_store.cc:276] 0709 12:26:29.388132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.388307 (+   175us) service_pool.cc:224] Handling call
0709 12:26:30.786864 (+1398557us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b540028df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:30.787073 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018295680}
I20260709 12:26:30.886240  7728 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:30.784894  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:30.816545  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.677455 10963 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:30.666057 10824 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:26:30.893949  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:30.700917 10936 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.076s	user 0.006s	sys 0.008s
I20260709 12:26:30.896917  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.936203  7805 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:30.898059  7810 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.900379  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:30.902889  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.937268  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:30.937970  7777 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.207s (SENT)
I20260709 12:26:30.938700  7735 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:30.938551  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:30.905895  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.817668 10992 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.686945  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:30.688143 10979 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:30.906637  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:30.968798  7655 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:30.894376  7659 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:30.837450 10966 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:26:30.954309  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.989674  7805 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:30.675619 10734 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:30.955420  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:30.958770  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:30.956513 10741 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:30.954355 10936 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:30.960501 10969 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:30.961592 10819 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:26:30.961813  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.964803  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:30.959290 10988 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:31.005690  7728 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:30.799736 10910 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:30.687201 10858 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.005693 10819 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:31.016144 10994 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:26:31.017083 10819 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:30.900812 10170 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.072s	user 0.015s	sys 0.005s
I20260709 12:26:31.018779 11000 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.019342 11000 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.699569 10706 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.023758 10954 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:30.816869 10973 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:30.748761 10758 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.683893 10888 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:30.687760 10982 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:30.967763 10801 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:26:30.992750  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:30.995296  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.061450  7735 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:30.995719  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:31.063129  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67541.5us
I20260709 12:26:30.951876 10968 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:31.069316 10981 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:31.070420 10981 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:31.070904 10981 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:30.997031  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:31.072455  7805 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:31.073457 10888 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.073652  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:31.078908 10801 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:31.079495 10801 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:31.081039 10801 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:26:30.978600  7653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.017s	sys 0.012s
W20260709 12:26:31.083686  7653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.018s	sys 0.012s
I20260709 12:26:31.084168 10982 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.025599  7808 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:30.964179 10916 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:31.087630 10916 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.090833 10770 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.074542  7805 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:31.034507  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:31.091943 11002 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:31.092733  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:31.093758 10998 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:31.094476  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:31.094761  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:31.095647  7657 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:31.100169  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.101007 10985 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.101817 10985 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.104548  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:31.113101  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:31.118366  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59578 (request call id 3) took 1468 ms. Trace:
I20260709 12:26:30.899632 10757 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.120759  7806 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:31.062050 10758 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:31.042165 10980 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
W20260709 12:26:31.042514 10837 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.804s	user 0.022s	sys 0.000s
I20260709 12:26:31.044559 10875 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:31.044982 10796 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.004s
I20260709 12:26:31.045915 10811 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:31.048604 10845 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.000s
W20260709 12:26:31.049136 10962 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.953s	user 0.002s	sys 0.000s
W20260709 12:26:31.051869 10754 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.809s	user 0.016s	sys 0.005s
I20260709 12:26:31.052019  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:31.173215  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.856380 10850 consensus_queue.cc:1175] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 7 status { last_received { term: 7 index: 5 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:31.052989  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:31.001899 10979 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:30.998586 10781 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.066624  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:31.186635  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120080us
I20260709 12:26:31.011765 10734 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.033670  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 833) took 1787 ms. Trace:
I20260709 12:26:30.967460 10992 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.031523 10963 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.132719  7807 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:31.059041 10936 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.102730  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.135673  7811 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:31.118947  8408 rpcz_store.cc:276] 0709 12:26:29.650183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.650441 (+   258us) service_pool.cc:224] Handling call
0709 12:26:31.118059 (+1467618us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:31.118229 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1707727232}
I20260709 12:26:31.024686 10954 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:30.944288 10948 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.083014 10801 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:31.162560  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:31.164675  7662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.012s	sys 0.042s
W20260709 12:26:31.226220  7662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.012s	sys 0.042s
I20260709 12:26:31.059137 11000 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.187810  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:31.228452  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.006s	sys 0.000s
W20260709 12:26:31.228907  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.006s	sys 0.000s
I20260709 12:26:31.229169  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:31.219949 10835 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.195573  8415 rpcz_store.cc:276] 0709 12:26:29.245954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.252157 (+  6203us) service_pool.cc:224] Handling call
0709 12:26:29.252650 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:29.623812 (+371162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.623861 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:29.624696 (+   835us) log.cc:844] Serialized 52 byte log entry
0709 12:26:29.884319 (+259623us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:29.900302 (+ 15983us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.930475 (+ 30173us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.930797 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.322441 (+391644us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:30.376401 (+ 53960us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:30.376502 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.033422 (+656920us) raft_consensus.cc:1685] finished
0709 12:26:31.033446 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.033605 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10393,"prepare.queue_time_us":60522,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":8670,"raft.queue_time_us":17532,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":9025,"spinlock_wait_cycles":863197952,"thread_start_us":259115,"threads_started":1,"wal-append.queue_time_us":446528}
I20260709 12:26:31.146648 10837 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.213848 10961 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:31.214512 10954 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.215760  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:31.171062 10947 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.002s
I20260709 12:26:31.218436 10992 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:31.194576 10905 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.267s	user 0.009s	sys 0.006s
I20260709 12:26:31.220579  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:31.151939 10741 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:31.136018 10982 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.091073 10916 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.162230 10796 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.162598 10928 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.235570  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:31.235869 10781 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:31.236083 10875 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:31.205454 10811 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:31.232498  7732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.003s	sys 0.004s
I20260709 12:26:31.210835 10979 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.173439 10845 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.198356 10754 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.036938 10969 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:26:31.455070  7803 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.744s	user 0.014s	sys 0.000s
I20260709 12:26:31.493961  7803 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:31.243347  7655 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:31.309926  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.272171  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:31.239974  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0362258340001063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 12:26:31.413779  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.499575  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.308660  8409 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:26:31.497078  7732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.004s	sys 0.009s
W20260709 12:26:31.499239  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 227092us
I20260709 12:26:31.358412  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.557634  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:31.497102  7735 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:31.498646  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9993258340000466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 12:26:31.559360  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9912258340000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 12:26:31.559803  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9822258340000189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 12:26:31.560129  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9813258340000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:26:31.560411  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9660258339999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 12:26:31.560694  7702 connection.cc:256] RPC call timeout handler was delayed by -1.795025834000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 12:26:31.561007  7702 connection.cc:256] RPC call timeout handler was delayed by -1.66452583399996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 12:26:31.331542 10606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.004s	sys 0.004s
I20260709 12:26:31.529345  7658 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:31.499122  7662 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:31.242684 10951 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.314s	user 0.014s	sys 0.001s
W20260709 12:26:31.232956 10896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.000s	sys 0.007s
W20260709 12:26:31.532775 11003 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:26:31.353518 10876 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.533938 10796 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.297449 10758 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:31.536790  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:31.534808 10905 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:31.543902 10985 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:31.551256  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.552951 10971 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.130s	user 0.011s	sys 0.001s
W20260709 12:26:31.554608 11008 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:26:31.458545 10907 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.022s	sys 0.053s
I20260709 12:26:31.560757 10811 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:31.244695 10967 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.892s	user 0.006s	sys 0.006s
W20260709 12:26:31.561461  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5331258340002023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 12:26:31.652899  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4926258340001368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
W20260709 12:26:31.653499  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4917258340001354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 12:26:31.653927  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4377258340000481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
W20260709 12:26:31.654338  7702 connection.cc:256] RPC call timeout handler was delayed by -1.398125833999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 12:26:31.241322  7728 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:31.348902  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.000s	sys 0.009s
W20260709 12:26:31.651890  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 414847us
W20260709 12:26:31.567446  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66534.5us
I20260709 12:26:31.657104 11002 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.657582 10837 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.657720 11002 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.659268 11009 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:31.564800  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:31.668219  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 103501us
I20260709 12:26:31.669543  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:31.571815  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:31.533759 11001 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:26:31.495779 10781 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.500192 10875 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.240626 10961 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:31.233668 10973 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.690330 10948 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.578747  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:31.691092 10948 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:31.691589 10948 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.692127 10948 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:31.693145 10754 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:31.357378 10933 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.694965 10933 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:31.696251 11014 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:26:31.703514 10905 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:31.708695  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1873) took 1273 ms. Trace:
I20260709 12:26:31.709322  7659 rpcz_store.cc:276] 0709 12:26:30.435505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.536485 (+1100980us) service_pool.cc:224] Handling call
0709 12:26:31.708575 (+172090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.242363 10987 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:31.570890 10896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.260s	user 0.000s	sys 0.024s
W20260709 12:26:31.563341 10606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.005s	sys 0.010s
W20260709 12:26:31.627439  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.723774  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.634979  7806 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.013s
W20260709 12:26:31.724412  7806 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.016s	sys 0.013s
I20260709 12:26:31.573074 10936 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.656132  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.005s	sys 0.010s
W20260709 12:26:31.562132  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.568526  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2202) took 1060 ms. Trace:
I20260709 12:26:31.727861  7662 rpcz_store.cc:276] 0709 12:26:30.508625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.235212 (+726587us) service_pool.cc:224] Handling call
0709 12:26:31.568451 (+333239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.573326 10951 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:31.273180 10947 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.559404 10928 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.684890 10781 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:31.685366 10875 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:31.686374 10961 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.711331  7655 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:31.716737  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:31.622387 10876 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.720388  7728 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:31.651844 10925 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:31.648458 10971 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:31.728528  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:31.969254  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 241832us
I20260709 12:26:31.969731  7654 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:31.970168  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1878) took 1408 ms. Trace:
I20260709 12:26:31.970700  7659 rpcz_store.cc:276] 0709 12:26:30.561457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.710329 (+1148872us) service_pool.cc:224] Handling call
0709 12:26:31.970076 (+259747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.971988  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:31.977721  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.972132  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.791s (SENT) [suppressed 17 similar messages]
W20260709 12:26:31.932005  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.980180  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.902292 11019 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:26:31.902953  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1863) took 1562 ms. Trace:
I20260709 12:26:31.982548  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:31.983633  7737 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:26:31.884341  7736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.001s	sys 0.020s
W20260709 12:26:31.984885  7736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.003s	sys 0.022s
W20260709 12:26:31.767716  7808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.009s
W20260709 12:26:31.985826  7808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.011s	sys 0.015s
W20260709 12:26:31.735651  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 327776us
I20260709 12:26:31.974886 10961 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:31.997779  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:31.708077 10967 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5.
I20260709 12:26:31.782330  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:31.988238  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.776578  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2077) took 1288 ms. Trace:
I20260709 12:26:32.007741  7662 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:32.008641  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2201) took 1500 ms. Trace:
I20260709 12:26:32.009339  7662 rpcz_store.cc:276] 0709 12:26:30.508099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.781928 (+1273829us) service_pool.cc:224] Handling call
0709 12:26:32.008553 (+226625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.010758  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:31.786252 10968 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.000s
I20260709 12:26:32.005386  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2020) took 1483 ms. Trace:
I20260709 12:26:32.018460  7805 rpcz_store.cc:276] 0709 12:26:30.522132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.126238 (+604106us) service_pool.cc:224] Handling call
0709 12:26:31.137286 (+ 11048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.138622 (+  1336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.138674 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.172333 (+ 33659us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.634608 (+462275us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:31.634702 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.634832 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.005126 (+370294us) raft_consensus.cc:1685] finished
0709 12:26:32.005148 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.005308 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6212,"thread_start_us":397487,"threads_started":1,"wal-append.queue_time_us":544150}
I20260709 12:26:32.012395  7662 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:32.014212  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.007869  7732 rpcz_store.cc:276] 0709 12:26:30.488099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.491340 (+  3241us) service_pool.cc:224] Handling call
0709 12:26:30.552604 (+ 61264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.552827 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:30.731188 (+178361us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.724853 (+993665us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.776005 (+ 51152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.776112 (+   107us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:31.776181 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.776302 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.776488 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18480,"prepare.queue_time_us":231616,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":85772,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":56174,"thread_start_us":1123315,"threads_started":3,"wal-append.queue_time_us":956464}
I20260709 12:26:31.982602  7653 rpcz_store.cc:276] 0709 12:26:30.340361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.340502 (+   141us) service_pool.cc:224] Handling call
0709 12:26:30.341009 (+   507us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.355336 (+ 14327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.355373 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.356269 (+   896us) log.cc:844] Serialized 52 byte log entry
0709 12:26:31.057470 (+701201us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4800080358. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:26:31.084214 (+ 26744us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:31.084424 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.084556 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.833030 (+748474us) raft_consensus.cc:1685] finished
0709 12:26:31.833061 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.902841 (+ 69780us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":914,"raft.queue_time_us":8575,"raft.run_cpu_time_us":3443,"raft.run_wall_time_us":682979,"spinlock_wait_cycles":11380480,"thread_start_us":616578,"threads_started":1,"wal-append.queue_time_us":858619}
I20260709 12:26:32.016232  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.017012  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:32.024271  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.024499  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1879) took 1452 ms. Trace:
I20260709 12:26:32.026533  8415 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:31.933012 10837 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.026494  7662 rpcz_store.cc:276] 0709 12:26:30.572533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.010359 (+1437826us) service_pool.cc:224] Handling call
0709 12:26:32.024412 (+ 14053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.029210  8414 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:32.028898  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:31.933128 10992 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.029875  7662 raft_consensus.cc:2376] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8. Re-sending same reply.
I20260709 12:26:32.030812  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.030658  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:32.032407  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.030337  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.065136  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:32.075282  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.078174  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.080520  7732 raft_consensus.cc:2368] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
W20260709 12:26:32.083388  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52841.8us
I20260709 12:26:32.087956  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:32.088852  8406 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:32.074051 10741 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.089274  8406 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:32.089740 10741 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.090232  8406 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:31.977932 10925 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.093276 10925 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.061458 10837 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.080631  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.103463  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:32.107620  7804 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:32.110191  7656 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:32.110343  7805 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:32.035362 10981 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.888s	user 0.008s	sys 0.004s
I20260709 12:26:32.121052  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 505) took 1505 ms. Trace:
I20260709 12:26:32.124637 10837 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:32.125291 11015 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:26:32.125914 10741 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.126699 10971 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.126827 10754 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.126974 10814 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.109560 10770 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:32.127597 10845 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:32.127732 10814 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:32.128160 10770 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.017469 11002 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:31.768676 10936 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.118373  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.125092 10792 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.824348 10811 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:32.128607 10801 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.895s	user 0.007s	sys 0.006s
I20260709 12:26:32.121263  7730 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:32.123744  7811 rpcz_store.cc:276] 0709 12:26:30.615819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.207180 (+591361us) service_pool.cc:224] Handling call
0709 12:26:31.219579 (+ 12399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.219792 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:31.652085 (+432293us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.112080 (+459995us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.120607 (+  8527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.120676 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:32.120736 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.120834 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.120978 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8219,"prepare.queue_time_us":447124,"prepare.run_cpu_time_us":179,"prepare.run_wall_time_us":177,"raft.queue_time_us":95911,"thread_start_us":808699,"threads_started":2}
W20260709 12:26:32.119310  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 143363us
I20260709 12:26:32.117487  7737 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:32.134649  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2092) took 1441 ms. Trace:
I20260709 12:26:32.135370  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:31.722492 11009 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.114892  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.157552  7653 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:31.726681 10987 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.159544 10814 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.165776  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2021) took 1643 ms. Trace:
W20260709 12:26:32.096261 11017 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:26:32.110862 10896 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:32.155618  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.173522  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.166833  7808 rpcz_store.cc:276] 0709 12:26:30.522499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.127524 (+605025us) service_pool.cc:224] Handling call
0709 12:26:31.128199 (+   675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.128885 (+   686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.128929 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.129659 (+   730us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.986411 (+856752us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:31.986471 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.986585 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.165371 (+178786us) raft_consensus.cc:1685] finished
0709 12:26:32.165405 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.165663 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24418,"raft.queue_time_us":5215,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2834,"thread_start_us":764306,"threads_started":1,"wal-append.queue_time_us":990021}
I20260709 12:26:32.152382  7732 rpcz_store.cc:276] 0709 12:26:30.692974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.100836 (+1407862us) service_pool.cc:224] Handling call
0709 12:26:32.134061 (+ 33225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.134276 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.134301 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.134352 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:32.134387 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.134449 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.134572 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.120083 10979 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:32.137372 10934 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.791s	user 0.013s	sys 0.003s
I20260709 12:26:32.099542 10820 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:32.212363  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 696) took 2639 ms. Trace:
W20260709 12:26:32.209618  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58994.4us
I20260709 12:26:32.097543 10992 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.217202  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 602) took 1327 ms. Trace:
I20260709 12:26:32.335472  7728 rpcz_store.cc:276] 0709 12:26:30.889508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.201880 (+1312372us) service_pool.cc:224] Handling call
0709 12:26:32.216494 (+ 14614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.216714 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.216757 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.216826 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:32.216876 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.216955 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.217106 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.119949 10528 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:26:32.127597 10754 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:32.259032  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.263974  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:32.269711  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.270714 10948 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
W20260709 12:26:32.271549  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2012) took 1759 ms (client timeout 1571 ms). Trace:
W20260709 12:26:32.346953  7804 rpcz_store.cc:269] 0709 12:26:30.512503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.513614 (+  1111us) service_pool.cc:224] Handling call
0709 12:26:32.271314 (+1757700us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.271455 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3351389952}
W20260709 12:26:32.273306  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1858) took 2015 ms (client timeout 2022 ms). Trace:
I20260709 12:26:32.138481 10981 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.334852  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:32.349537  7656 rpcz_store.cc:269] 0709 12:26:30.257990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.258196 (+   206us) service_pool.cc:224] Handling call
0709 12:26:32.273067 (+2014871us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54000f3054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.273215 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3847674624}
I20260709 12:26:32.234989  8407 rpcz_store.cc:276] 0709 12:26:29.573171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.573346 (+   175us) service_pool.cc:224] Handling call
0709 12:26:29.575417 (+  2071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.284886 (+709469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.212064 (+1927178us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540046cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:32.212290 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629547904}
W20260709 12:26:32.241971  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52050us
I20260709 12:26:32.258726 10979 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:32.311506  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.334676  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1861) took 2007 ms. Trace:
I20260709 12:26:32.348991  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.311784 10947 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.312443 10936 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.312682  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.358377  7732 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:32.145325 11016 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:32.278276 10934 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.315006  7809 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:32.362205  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.363106  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.364430  7808 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:32.276868  7811 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:32.329545  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.367578  8413 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:32.337738  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.338656  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.368702  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.369372  7805 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:32.369638  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1892) took 1136 ms. Trace:
I20260709 12:26:32.201094 10969 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.154116 10792 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.354660  7804 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.352892  7661 rpcz_store.cc:276] 0709 12:26:30.327919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.328982 (+  1063us) service_pool.cc:224] Handling call
0709 12:26:30.355975 (+ 26993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.371751 (+ 15776us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:30.790589 (+418838us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b18003a3d80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:26:31.237907 (+447318us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.981421 (+743514us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.984965 (+  3544us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.985147 (+   182us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:31.985183 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.985258 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.319717 (+334459us) raft_consensus.cc:1685] finished
0709 12:26:32.334340 (+ 14623us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.334576 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125363,"prepare.queue_time_us":223907,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":280966,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":37334,"spinlock_wait_cycles":23973632,"thread_start_us":814055,"threads_started":2,"wal-append.queue_time_us":866302}
I20260709 12:26:32.214897 10896 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.370028  7656 rpcz_store.cc:276] 0709 12:26:31.233474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.350845 (+1117371us) service_pool.cc:224] Handling call
0709 12:26:32.369566 (+ 18721us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.387603  7803 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:32.389315  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.389930  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.395125  7728 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
W20260709 12:26:32.390311  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52715.4us
I20260709 12:26:32.352481 10979 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:32.398838  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.399940  7653 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:32.400949  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1868) took 1992 ms (client timeout 2588 ms). Trace:
W20260709 12:26:32.401530  7653 rpcz_store.cc:269] 0709 12:26:30.408823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.114452 (+1705629us) service_pool.cc:224] Handling call
0709 12:26:32.400848 (+286396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.405993  7807 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:32.408607  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.409559  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:32.412009 11050 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:32.414841 11050 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.414956 10770 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.415310 11050 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:32.415879 10754 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.412779  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.380853 10967 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:32.418447  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 439855us
I20260709 12:26:32.421540  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.422513  7661 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:32.423170  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1891) took 1232 ms. Trace:
I20260709 12:26:32.423566  7661 rpcz_store.cc:276] 0709 12:26:31.190516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.391877 (+1201361us) service_pool.cc:224] Handling call
0709 12:26:32.423081 (+ 31204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.418612  7656 raft_consensus.cc:2376] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
I20260709 12:26:32.417658  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:32.425668  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1885) took 1237 ms. Trace:
I20260709 12:26:32.426007  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1889) took 1236 ms. Trace:
I20260709 12:26:32.426429  7656 rpcz_store.cc:276] 0709 12:26:31.188565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.391883 (+1203318us) service_pool.cc:224] Handling call
0709 12:26:32.425571 (+ 33688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.426939  7653 rpcz_store.cc:276] 0709 12:26:31.189876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.403678 (+1213802us) service_pool.cc:224] Handling call
0709 12:26:32.425902 (+ 22224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.346956 10948 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.437690  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.411206  7804 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:32.429998  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.383447 10969 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:32.433308  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.441576  7812 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:26:32.435635  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.613s (SENT)
I20260709 12:26:32.442984  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:32.439803 10876 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.440461  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.447396  7728 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:26:32.434341 10933 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.737s	user 0.014s	sys 0.000s
W20260709 12:26:32.460964  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.230s (SENT)
I20260709 12:26:32.451382  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.463728  7804 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:26:32.452363  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 464475us
I20260709 12:26:32.445916  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:32.380134 11016 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.420617 11042 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:32.437638 10770 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.430845 10754 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:32.471170  7659 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:32.477155  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.441054 10914 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.452929 10936 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:32.472100  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:32.474543  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.485101  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:32.485806  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1893) took 1252 ms. Trace:
I20260709 12:26:32.486210  7661 rpcz_store.cc:276] 0709 12:26:31.233789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.448359 (+1214570us) service_pool.cc:224] Handling call
0709 12:26:32.485722 (+ 37363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.441722 10987 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.466374  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.471553  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:32.481446  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.482787  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.493079  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.478936  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.494674  7804 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:32.489290  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.496789  7728 raft_consensus.cc:2410] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:26:32.469575 10925 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:32.502426 10706 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:32.502614  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.503038 10706 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.503585  7661 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:32.503594 10706 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:32.504441  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1887) took 1315 ms. Trace:
I20260709 12:26:32.504822  7661 rpcz_store.cc:276] 0709 12:26:31.189245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.487150 (+1297905us) service_pool.cc:224] Handling call
0709 12:26:32.504288 (+ 17138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.489512 10754 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:32.507804 11036 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:26:32.472431 10951 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.513278  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.483369 10528 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:32.492451 10928 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:32.526541  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.541136  7732 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:32.499686 10987 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.538678  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 509) took 1921 ms. Trace:
I20260709 12:26:32.556696  7812 rpcz_store.cc:276] 0709 12:26:30.616965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.515304 (+1898339us) service_pool.cc:224] Handling call
0709 12:26:32.537902 (+ 22598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.538173 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.538209 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.538270 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:32.538319 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.538428 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.538590 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:32.529845 11040 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.312s	user 0.000s	sys 0.002s
W20260709 12:26:32.530506 11033 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.318s	user 0.001s	sys 0.001s
I20260709 12:26:32.524148  7661 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:32.543936 11025 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:26:32.544965 11039 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:26:32.545761  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2478) took 1665 ms (client timeout 2083 ms). Trace:
I20260709 12:26:32.484549 10770 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.512619 10757 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:32.548151  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.550633 11016 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.551716 11009 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.490963 10914 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.566136  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1886) took 1377 ms. Trace:
I20260709 12:26:32.590770  7661 rpcz_store.cc:276] 0709 12:26:31.188917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.505483 (+1316566us) service_pool.cc:224] Handling call
0709 12:26:32.566037 (+ 60554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.591181  7812 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:32.592129  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2477) took 1711 ms. Trace:
I20260709 12:26:32.592514 10914 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.592619  7812 rpcz_store.cc:276] 0709 12:26:30.880514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.566686 (+1686172us) service_pool.cc:224] Handling call
0709 12:26:32.590886 (+ 24200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.591808 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.592050 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.593508  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 704) took 1167 ms. Trace:
I20260709 12:26:32.593930  8415 rpcz_store.cc:276] 0709 12:26:31.426045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.426634 (+   589us) service_pool.cc:224] Handling call
0709 12:26:31.488789 (+ 62155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.027178 (+538389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.593247 (+566069us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54007a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:32.593438 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236218240}
I20260709 12:26:32.595685  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.520782 10951 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:32.584059  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.568084  7809 rpcz_store.cc:269] 0709 12:26:30.880672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.104170 (+223498us) service_pool.cc:224] Handling call
0709 12:26:32.545541 (+1441371us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400373cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.545687 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415776512}
I20260709 12:26:32.592043  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.603072  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2200) took 2095 ms. Trace:
I20260709 12:26:32.603574  7661 rpcz_store.cc:276] 0709 12:26:30.507630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.591731 (+2084101us) service_pool.cc:224] Handling call
0709 12:26:32.602991 (+ 11260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.529357 10528 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.571020  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2093) took 1744 ms. Trace:
W20260709 12:26:32.600490  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 238367us
W20260709 12:26:32.605898  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 133923us
I20260709 12:26:32.616716  7809 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:32.617488  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2484) took 1722 ms. Trace:
I20260709 12:26:32.618240  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.617887  7809 rpcz_store.cc:276] 0709 12:26:30.895127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.615973 (+1720846us) service_pool.cc:224] Handling call
0709 12:26:32.616453 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.617205 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.617426 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.621357  7809 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:26:32.622328  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2485) took 1726 ms. Trace:
I20260709 12:26:32.624766  7809 rpcz_store.cc:276] 0709 12:26:30.895413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.620651 (+1725238us) service_pool.cc:224] Handling call
0709 12:26:32.621057 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:32.621901 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.622235 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.628566 11042 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:32.616315  7737 rpcz_store.cc:276] 0709 12:26:30.826626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.151446 (+1324820us) service_pool.cc:224] Handling call
0709 12:26:32.151770 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.151986 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:32.357186 (+205200us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.570514 (+213328us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.570593 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:32.570655 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.570753 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.570934 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3053,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"thread_start_us":417596,"threads_started":2}
I20260709 12:26:32.634603  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.636267  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.637507  7737 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:32.627141  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.638935  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.598066 10992 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.645406  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.548919 10979 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.562273 10987 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.676046  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.680250  7737 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:32.622983 10528 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:32.676353  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49318.4us
W20260709 12:26:32.688303  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70126.7us
I20260709 12:26:32.692262  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:32.693850  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.695770  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.700735  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.639467 11009 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:32.599251 10757 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.708495  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:32.709172 11009 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:32.683137 10845 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:32.707763 11049 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:26:32.714305  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2088) took 2197 ms. Trace:
I20260709 12:26:32.723930  7736 rpcz_store.cc:276] 0709 12:26:30.534773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.218098 (+683325us) service_pool.cc:224] Handling call
0709 12:26:31.218499 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:31.218905 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.218935 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:31.230453 (+ 11518us) log.cc:844] Serialized 52 byte log entry
0709 12:26:31.985355 (+754902us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:31.985411 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.985504 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.713986 (+728482us) raft_consensus.cc:1685] finished
0709 12:26:32.714011 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.714211 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3495,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":17270,"thread_start_us":676803,"threads_started":1,"wal-append.queue_time_us":876784}
I20260709 12:26:32.690037 10934 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.721645  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 508) took 2104 ms. Trace:
I20260709 12:26:32.730281  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.727737  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.674196 10706 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.728884  7804 rpcz_store.cc:276] 0709 12:26:30.616733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.510414 (+1893681us) service_pool.cc:224] Handling call
0709 12:26:32.533830 (+ 23416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.534179 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:32.543396 (+  9217us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.721002 (+177606us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.721100 (+    98us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:32.721175 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.721296 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.721514 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19430,"prepare.queue_time_us":10433,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"thread_start_us":157619,"threads_started":1}
I20260709 12:26:32.707374 10981 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.713083 11000 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:32.710954 10792 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.713992 10896 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:32.748606  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.749997 10757 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.791957 10896 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:32.792711 10757 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:32.754623 10770 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.736670 10845 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:32.757359 11042 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.744881 11065 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:32.759233  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:32.800784  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:32.760035 11054 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:32.801863  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:32.760346  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:32.803658  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1881) took 2218 ms (client timeout 1791 ms). Trace:
W20260709 12:26:32.804173  7655 rpcz_store.cc:269] 0709 12:26:30.585661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.992421 (+406760us) service_pool.cc:224] Handling call
0709 12:26:32.803439 (+1811018us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540021b6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.803567 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419311360}
I20260709 12:26:32.804978  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.806183  7732 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:32.769567 10706 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:32.733301 10781 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.728s	user 0.012s	sys 0.002s
I20260709 12:26:32.808967 10781 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:32.772068 10979 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.775632 10987 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.777146  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.812318  7736 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:32.762538 10875 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.000s
W20260709 12:26:32.814702  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.778641 11009 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.783676 11000 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:32.809417  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:32.810889  7729 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.828465  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.816782  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.830147  7661 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
W20260709 12:26:32.777601 10961 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.004s
I20260709 12:26:32.829063  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:32.845112  7736 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:26:32.796023 11046 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.481s	user 0.004s	sys 0.000s
I20260709 12:26:32.825534 10875 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:32.827472  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116299us
I20260709 12:26:32.824482  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:32.850986  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:32.711915 10992 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:32.848331  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.852540  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:32.853101  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 511) took 1336 ms. Trace:
I20260709 12:26:32.856356  7804 rpcz_store.cc:276] 0709 12:26:31.517219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.741702 (+1224483us) service_pool.cc:224] Handling call
0709 12:26:32.852557 (+110855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.852757 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.852783 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.852827 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:32.852857 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.852911 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.853031 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.854760  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.854970  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53564 (request call id 598) took 2404 ms. Trace:
I20260709 12:26:32.858512  7732 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7.
I20260709 12:26:32.858212  7729 rpcz_store.cc:276] 0709 12:26:30.431658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.457549 (+ 25891us) service_pool.cc:224] Handling call
0709 12:26:32.810756 (+2353207us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b4c0001fb80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc0001eea20 00007fc0001ffa29 00007fc00020013f 00007fc0001c42ab 00007fc0001f1f3c 00007fc0001f23fe 00007fc00047f88b 00007fc001e7e495 00007fc001ecf30f 00007fc001f19a55 00007fc006ee9cd0
0709 12:26:32.828595 (+ 17839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79154560}
I20260709 12:26:32.862946  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.862741  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:32.871951  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.873404  7729 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:32.874797  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:32.878719 10951 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.880378 11057 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:32.885972  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2199) took 2379 ms (client timeout 1908 ms). Trace:
I20260709 12:26:32.860042 11065 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:32.932011  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 507) took 2315 ms. Trace:
I20260709 12:26:32.936859  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:32.932468  7803 rpcz_store.cc:276] 0709 12:26:30.616440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.494845 (+878405us) service_pool.cc:224] Handling call
0709 12:26:31.528629 (+ 33784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.391897 (+863268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.931063 (+539166us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b5400331b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:32.931924 (+   861us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974791936}
I20260709 12:26:32.939718  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.854250 10987 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 12:26:32.893226  7777 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.555s (SENT)
I20260709 12:26:32.853266 10796 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:32.887022  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1870) took 2463 ms (client timeout 1835 ms). Trace:
I20260709 12:26:32.946403  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:32.946609  7654 rpcz_store.cc:269] 0709 12:26:30.423607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.594246 (+170639us) service_pool.cc:224] Handling call
0709 12:26:32.886741 (+2292495us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540021bbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.886900 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3066987776}
I20260709 12:26:32.948112  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.948915  7654 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:32.949172  7803 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:32.949862  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2198) took 2443 ms. Trace:
W20260709 12:26:32.950366  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57192.8us
W20260709 12:26:32.898552  7658 rpcz_store.cc:269] 0709 12:26:30.506959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.037984 (+531025us) service_pool.cc:224] Handling call
0709 12:26:32.885704 (+1847720us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540028c654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.885863 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980274560}
I20260709 12:26:32.900678  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:32.952158  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:32.900818  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48358.9us
I20260709 12:26:32.901260 10992 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:32.902086  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1875) took 2401 ms. Trace:
I20260709 12:26:32.958400  7661 rpcz_store.cc:276] 0709 12:26:30.501278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.867450 (+2366172us) service_pool.cc:224] Handling call
0709 12:26:32.901472 (+ 34022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.901687 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.901715 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.901764 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:32.901803 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.901867 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.902015 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.863860 10845 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:32.905669 11055 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 12:26:32.969327  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:32.975207  7729 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
W20260709 12:26:32.910203 10820 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.003s
I20260709 12:26:32.910705 11000 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.057057  7803 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:33.057785  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2490) took 1497 ms. Trace:
I20260709 12:26:33.058125  7803 rpcz_store.cc:276] 0709 12:26:31.560729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.056418 (+1495689us) service_pool.cc:224] Handling call
0709 12:26:33.056779 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.057501 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.057727 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.059556  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:32.913273  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1883) took 1725 ms (client timeout 1555 ms). Trace:
W20260709 12:26:33.064538  7659 rpcz_store.cc:269] 0709 12:26:31.187914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.997374 (+809460us) service_pool.cc:224] Handling call
0709 12:26:32.913013 (+915639us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54000d07d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.913164 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071809792}
I20260709 12:26:32.914036  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2483) took 2019 ms. Trace:
I20260709 12:26:32.911011 10754 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:32.924360  8415 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:32.926009  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:32.927969 10781 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:32.929826 10875 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:32.944010  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2075) took 2450 ms (client timeout 2588 ms). Trace:
W20260709 12:26:33.084458  7730 rpcz_store.cc:269] 0709 12:26:30.487493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.490420 (+  2927us) service_pool.cc:224] Handling call
0709 12:26:32.937343 (+2446923us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54000d2ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:32.937477 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3394127232}
I20260709 12:26:33.084932  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:33.085767  7731 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:33.085815  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:33.086807  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:33.089715 10754 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:33.090380 10754 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.093155 10754 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:32.954123  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:33.094372  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 140357us
I20260709 12:26:33.095485 11058 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.095623  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:33.095553  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.097218  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:32.950349  7654 rpcz_store.cc:276] 0709 12:26:30.506581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.948191 (+2441610us) service_pool.cc:224] Handling call
0709 12:26:32.948665 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.949500 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.949783 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.098095 10979 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:33.098701 10979 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.099426 10896 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.100237 10896 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:32.955817  7732 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:33.099375  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:33.101545  7731 raft_consensus.cc:2376] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
I20260709 12:26:33.101627 10979 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.101596  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:33.102388  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1872) took 2670 ms. Trace:
I20260709 12:26:33.102605  7732 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:33.103175  7731 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:33.103909  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2095) took 2115 ms. Trace:
I20260709 12:26:32.889271  7804 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:33.010644  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:33.103963  7732 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:32.958474 10796 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.907089 10967 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:32.963017 10992 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:33.054520  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84147.6us
W20260709 12:26:33.055130  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58353.3us
I20260709 12:26:33.071839  8408 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:26:33.073586  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:40408
I20260709 12:26:33.074236  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:33.075802  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 585) took 2461 ms. Trace:
I20260709 12:26:33.088812  7810 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:33.104382  7731 rpcz_store.cc:276] 0709 12:26:30.988321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.102582 (+2114261us) service_pool.cc:224] Handling call
0709 12:26:33.102924 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.103620 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.103849 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:33.105652  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 95127.8us
I20260709 12:26:33.106117  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 603) took 2060 ms. Trace:
I20260709 12:26:32.962322 10925 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:33.106994  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:33.146551  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.066896  7812 rpcz_store.cc:276] 0709 12:26:30.894739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.593729 (+1698990us) service_pool.cc:224] Handling call
0709 12:26:32.594131 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.594536 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:32.594931 (+   395us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.732632 (+137701us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.733215 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.733387 (+   172us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:32.733736 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.750120 (+ 16384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.750217 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.913777 (+163560us) raft_consensus.cc:1685] finished
0709 12:26:32.913804 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.913969 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35352,"prepare.queue_time_us":5210,"prepare.run_cpu_time_us":288,"prepare.run_wall_time_us":10613,"raft.queue_time_us":58317,"raft.run_cpu_time_us":484,"raft.run_wall_time_us":19211,"thread_start_us":117857,"threads_started":1,"wal-append.queue_time_us":271909}
I20260709 12:26:33.102829  7656 rpcz_store.cc:276] 0709 12:26:30.432159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.812592 (+2380433us) service_pool.cc:224] Handling call
0709 12:26:33.101712 (+289120us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:33.101925 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.102089 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.102111 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.102151 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:33.102178 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.102230 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.102334 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26498304}
I20260709 12:26:33.206478  7732 rpcz_store.cc:276] 0709 12:26:31.045440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.103462 (+2058022us) service_pool.cc:224] Handling call
0709 12:26:33.103780 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.105851 (+  2071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.106046 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.208644  7655 raft_consensus.cc:2376] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5. Re-sending same reply.
W20260709 12:26:33.033422  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.052139  7658 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:33.211848  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2206) took 2700 ms. Trace:
I20260709 12:26:33.212491  7658 rpcz_store.cc:276] 0709 12:26:30.511602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.951977 (+2440375us) service_pool.cc:224] Handling call
0709 12:26:33.051785 (+ 99808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.211425 (+159640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.211744 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.953754  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:33.157125  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1869) took 2738 ms (client timeout 1547 ms). Trace:
I20260709 12:26:32.957815 10801 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.156508  7728 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:33.086998  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:33.072343 10820 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.220753  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 604) took 1991 ms. Trace:
I20260709 12:26:33.215054  7729 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:33.222218  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1900) took 1008 ms. Trace:
I20260709 12:26:33.143914  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2087) took 2627 ms. Trace:
I20260709 12:26:33.133690  7737 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:33.214468  7657 rpcz_store.cc:269] 0709 12:26:30.418722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.503046 (+ 84324us) service_pool.cc:224] Handling call
0709 12:26:33.156866 (+2653820us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:33.157034 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148080384}
I20260709 12:26:33.211289  7659 rpcz_store.cc:276] 0709 12:26:30.604996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.065779 (+2460783us) service_pool.cc:224] Handling call
0709 12:26:33.066148 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.066368 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.066407 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.066457 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:33.066501 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.066559 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.066690 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.223534  7729 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:33.216405  7661 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:26:33.223838  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 270119us
I20260709 12:26:33.240597  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2345) took 1164 ms. Trace:
I20260709 12:26:33.241117  7661 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.221906  7728 rpcz_store.cc:276] 0709 12:26:31.228944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.156011 (+1927067us) service_pool.cc:224] Handling call
0709 12:26:33.156342 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.220398 (+ 64056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.220677 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.244764  7655 rpcz_store.cc:276] 0709 12:26:32.213377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.805090 (+591713us) service_pool.cc:224] Handling call
0709 12:26:33.146273 (+341183us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:33.222097 (+ 75824us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44039424}
I20260709 12:26:33.246070  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 588) took 2554 ms. Trace:
I20260709 12:26:33.246590  7656 rpcz_store.cc:276] 0709 12:26:30.691535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.209479 (+2517944us) service_pool.cc:224] Handling call
0709 12:26:33.245467 (+ 35988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.245695 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.245722 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.245776 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:33.245826 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.245884 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.246007 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.248385  7730 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:33.249140  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2343) took 1422 ms. Trace:
I20260709 12:26:33.249490  7730 rpcz_store.cc:276] 0709 12:26:31.826417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.220105 (+1393688us) service_pool.cc:224] Handling call
0709 12:26:33.248140 (+ 28035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.248859 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.249082 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.251260  7658 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:33.252182  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1890) took 2062 ms. Trace:
I20260709 12:26:33.252636  7658 rpcz_store.cc:276] 0709 12:26:31.190207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.213534 (+2023327us) service_pool.cc:224] Handling call
0709 12:26:33.239971 (+ 26437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.251787 (+ 11816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.252106 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556416}
W20260709 12:26:33.240808  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 584) took 1227 ms (client timeout 1526 ms). Trace:
I20260709 12:26:33.243371  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:33.243279  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1896) took 1569 ms. Trace:
W20260709 12:26:33.257704  8408 rpcz_store.cc:269] 0709 12:26:32.013274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.013437 (+   163us) service_pool.cc:224] Handling call
0709 12:26:32.925756 (+912319us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:33.240718 (+314962us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63456640}
I20260709 12:26:33.259214  7659 rpcz_store.cc:276] 0709 12:26:31.681917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.241656 (+1559739us) service_pool.cc:224] Handling call
0709 12:26:33.242187 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.242621 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.242685 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.242779 (+    94us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:33.242886 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.242994 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.243176 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.227183  7731 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:33.241569  7729 rpcz_store.cc:276] 0709 12:26:32.075972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.222865 (+1146893us) service_pool.cc:224] Handling call
0709 12:26:33.223315 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.240251 (+ 16936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.240518 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.262511  7729 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:33.264011  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2348) took 1102 ms. Trace:
I20260709 12:26:33.264297  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1903) took 1050 ms. Trace:
I20260709 12:26:33.264878  7655 rpcz_store.cc:276] 0709 12:26:32.214331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.246603 (+1032272us) service_pool.cc:224] Handling call
0709 12:26:33.247151 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.263879 (+ 16728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.263915 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.263963 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:33.264000 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.264080 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.264223 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16176}
I20260709 12:26:33.266135  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2341) took 1552 ms. Trace:
I20260709 12:26:33.264575  7729 rpcz_store.cc:276] 0709 12:26:32.161287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.261762 (+1100475us) service_pool.cc:224] Handling call
0709 12:26:33.262236 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.263491 (+  1255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.263825 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:33.142591 11050 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.002s
I20260709 12:26:33.112489 10796 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:33.269076  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:33.306191  7812 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:33.306600  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2350) took 1144 ms. Trace:
I20260709 12:26:33.308871  7731 rpcz_store.cc:276] 0709 12:26:31.713296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.226414 (+1513118us) service_pool.cc:224] Handling call
0709 12:26:33.226891 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.265730 (+ 38839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.266053 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.307274  7729 rpcz_store.cc:276] 0709 12:26:32.161980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.305345 (+1143365us) service_pool.cc:224] Handling call
0709 12:26:33.305912 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.306159 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.306185 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.306238 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:33.306271 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.306341 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.306520 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.310839 11058 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.150336 10905 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:33.312371  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69176.5us
I20260709 12:26:33.220816 10992 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:33.313916  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:33.302286  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2101) took 1593 ms. Trace:
I20260709 12:26:33.239259  7733 rpcz_store.cc:276] 0709 12:26:30.516212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.107986 (+591774us) service_pool.cc:224] Handling call
0709 12:26:31.108358 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.572871 (+464513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.572917 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.572965 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:31.572994 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.143634 (+1570640us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540037e954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:33.143696 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.143840 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067594624}
I20260709 12:26:33.267657  7657 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:33.322798  7732 rpcz_store.cc:276] 0709 12:26:31.709029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.224378 (+1515349us) service_pool.cc:224] Handling call
0709 12:26:33.301363 (+ 76985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.301738 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.301778 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.301883 (+   105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:33.301928 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.301985 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.302179 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.323969  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1894) took 2089 ms. Trace:
I20260709 12:26:33.324389  7657 rpcz_store.cc:276] 0709 12:26:31.234073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.239443 (+2005370us) service_pool.cc:224] Handling call
0709 12:26:33.239796 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.323658 (+ 83862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.323910 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.268466  7730 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:33.219854 10925 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:33.348961  7733 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:33.321724  7661 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.306298 10923 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:33.326427  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.329452  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.250833 10836 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.264711 11059 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:33.113952 10979 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:33.348819 10811 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:33.347684  7736 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:33.321446 10801 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:33.360185  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.363829  7661 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:33.364244  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2370) took 1030 ms. Trace:
I20260709 12:26:33.319514 10905 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.320134 11050 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.388118  7731 rpcz_store.cc:276] 0709 12:26:32.333458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.310012 (+976554us) service_pool.cc:224] Handling call
0709 12:26:33.344516 (+ 34504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.359461 (+ 14945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.359506 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.363730 (+  4224us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:33.363785 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.363902 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.364151 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13818,"raft.queue_time_us":13980,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":4154}
I20260709 12:26:33.393858  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.353233 10925 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.361347 11059 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.374863  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.377465  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.400420  7809 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:33.379772 10835 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:33.388543  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.535s (SENT)
W20260709 12:26:33.392534  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.327037  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2347) took 1166 ms. Trace:
I20260709 12:26:33.402536  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.403519  7655 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:33.402923  7730 rpcz_store.cc:276] 0709 12:26:32.160737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.250359 (+1089622us) service_pool.cc:224] Handling call
0709 12:26:33.268245 (+ 17886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.325840 (+ 57595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.326971 (+  1131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:33.402221  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.548s (SENT)
I20260709 12:26:33.404152  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.362515 10979 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.365329 11042 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.398137  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.411720  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.406147  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40408 (request call id 584) took 2783 ms. Trace:
I20260709 12:26:33.413250  7661 rpcz_store.cc:276] 0709 12:26:30.604514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.960142 (+2355628us) service_pool.cc:224] Handling call
0709 12:26:33.388084 (+427942us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":291}
I20260709 12:26:33.406502  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.396486  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.383906  8406 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:33.405495 10781 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.384461 10811 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:33.405310 11059 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.418946  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.387732 10801 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.419302  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.417689 10835 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:33.420775 10781 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:33.362141 10836 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.408077 10979 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.416457 10875 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.431206  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.445415  7657 raft_consensus.cc:2376] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7. Re-sending same reply.
I20260709 12:26:33.432461  7730 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:33.395366 10968 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:26:33.438788  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 341652us
I20260709 12:26:33.439306  7658 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:33.435964 10811 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:33.437083  7736 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:33.408968 11016 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:33.438858 10801 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:33.444504  7734 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:33.460727  7734 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:33.462785  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.460911  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:33.442209 10781 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.464334  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.440004 10835 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:33.442386 10923 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.467658  8408 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:33.491827  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.468793  7730 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:33.464645  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:33.473500  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.461764  7734 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.495682  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:33.482350  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:33.483530  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:33.485193  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:33.497934  7810 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7. Re-sending same reply.
I20260709 12:26:33.498656  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:33.499960  7804 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:33.522310  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.523102  7661 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:33.522310  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.525246  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:33.495286  8414 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:33.463363 10978 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:33.466079 10757 raft_consensus.cc:2764] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:33.537197  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.540858  8414 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.542268  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.474566 10781 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.538560  7730 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:33.463150 10976 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:33.466419 11058 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:26:33.545557  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.553390  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.554339  8407 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:33.546315 10835 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.547780  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2352) took 1385 ms. Trace:
I20260709 12:26:33.548290 10836 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.521665 11050 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:33.528985 10967 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:33.543118  7661 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:33.556628  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.557248  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2089) took 3020 ms. Trace:
I20260709 12:26:33.564262  7735 rpcz_store.cc:276] 0709 12:26:30.538688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.559185 (+1020497us) service_pool.cc:224] Handling call
0709 12:26:31.578493 (+ 19308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:31.625990 (+ 47497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.626031 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:31.651239 (+ 25208us) log.cc:844] Serialized 52 byte log entry
0709 12:26:32.111460 (+460221us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:32.111524 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.111624 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.555543 (+1443919us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b240010c508. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee73fb0 00007fc001aea488 00007fc001aeabe8 00007fc001ac2398 00007fc001abd76a 00007fc001abcbb6
0709 12:26:33.556842 (+  1299us) raft_consensus.cc:1685] finished
0709 12:26:33.556868 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.557167 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41717,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":32637,"spinlock_wait_cycles":8379264,"thread_start_us":402628,"threads_started":1,"wal-append.queue_time_us":781104}
I20260709 12:26:33.559681  7737 rpcz_store.cc:276] 0709 12:26:32.162735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.693434 (+530699us) service_pool.cc:224] Handling call
0709 12:26:33.547534 (+854100us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400144bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:33.547694 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009153792}
I20260709 12:26:33.566556  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.567873  7737 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:33.568557  7737 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:33.569587  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.561199  8411 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:33.527103 10923 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.533780 11063 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:26:33.569845  7737 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.582238  7737 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.582603  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.589941  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.591265  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:33.592514  7735 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:33.591571  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:33.594980  7661 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:33.591202 11016 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:33.577842 11050 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:33.566550 10967 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:33.542835 10978 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.579174 10770 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:33.599735  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6470250580000538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:26:33.612129  7735 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:33.603858  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:33.611304  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6353250580000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 12:26:33.613044  7661 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.604290 10937 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:26:33.610544 11051 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 12:26:33.602888 11069 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:26:33.610726 10967 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:33.618824 10978 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:33.619987 11068 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:33.637568  7661 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:33.588282 10835 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:33.579735 10757 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
W20260709 12:26:33.623952  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6065250579999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 12:26:33.652093  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5471250579998923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 12:26:33.652504  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3311250579999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 12:26:33.652781  7702 connection.cc:256] RPC call timeout handler was delayed by -1.317625057999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 12:26:33.653055  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3032250579999527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:26:33.653082  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:33.653442  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1682250580001892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:26:33.653733  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0584250580000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 12:26:33.653915  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 501) took 3039 ms. Trace:
I20260709 12:26:33.654479  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:33.657866  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:33.648037 10595 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.654709  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:33.650982  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.660338  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:33.654426  7807 rpcz_store.cc:276] 0709 12:26:30.614614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.206896 (+592282us) service_pool.cc:224] Handling call
0709 12:26:31.207339 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.406581 (+1199242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.653611 (+1247030us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003f3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:33.653808 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756093696}
I20260709 12:26:33.670065  7661 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:33.674176  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.689355  8407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:33.670033 10757 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:33.680307 10595 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.728240  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 316574us
I20260709 12:26:33.707755  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.682957 11050 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:33.726867  7737 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:26:33.747159  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.748260  8412 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:33.741436 10528 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.032s	user 0.007s	sys 0.009s
I20260709 12:26:33.756702  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:33.772846 10923 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.772284 10796 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.773627 10923 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:33.775069 10845 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:33.720129 10595 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.784999  8408 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:33.794505  8407 raft_consensus.cc:606] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:33.795269  8407 raft_consensus.cc:614] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:33.798343 11042 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:33.767853 10528 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.802987 10528 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:33.772147  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.799712  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.804224  8405 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:33.805522  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:33.805950  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.807317  8408 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:33.813982 10992 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.828099  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 590) took 1019 ms. Trace:
I20260709 12:26:33.828794  8415 rpcz_store.cc:276] 0709 12:26:32.808875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.809054 (+   179us) service_pool.cc:224] Handling call
0709 12:26:33.827765 (+1018711us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:33.827946 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57874176}
I20260709 12:26:33.835281 10836 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:33.836568 10836 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:33.837342 10948 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.839471 10839 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:33.840647 10839 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:33.842651  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.843371  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.846578 10849 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.857748 10595 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.861202  8414 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:33.863137  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.109566  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2019) took 3587 ms. Trace:
W20260709 12:26:34.109910  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 246797us
I20260709 12:26:34.112694  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.114066  8408 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:33.865082  8413 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:33.874102  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 577) took 3922 ms. Trace:
W20260709 12:26:34.058522  7656 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.017s	sys 0.042s
I20260709 12:26:34.060642  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.060114  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.062309  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.064420  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:34.073606  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.003s	sys 0.014s
W20260709 12:26:34.073969  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.076717  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.076546  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 417190us
I20260709 12:26:34.086046  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2113) took 1203 ms. Trace:
I20260709 12:26:34.091615  7734 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:34.092583  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2115) took 1157 ms. Trace:
W20260709 12:26:34.096572  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.097761  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 329397us
I20260709 12:26:34.100410  7803 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:34.048034  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:34.118189  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.119959  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:33.763170 10937 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.167586 10937 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:34.168300 10937 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.168778 10937 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.172556  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:33.929764 10836 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:33.920518 10839 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:33.783581 10845 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:33.781909 10875 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:26:34.110236  7806 rpcz_store.cc:276] 0709 12:26:30.521696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.121758 (+600062us) service_pool.cc:224] Handling call
0709 12:26:31.122076 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.122323 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:31.724812 (+602489us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.108134 (+2383322us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b180004afe0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:26:34.108481 (+   347us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.108645 (+   164us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:34.108775 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.108973 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.109273 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":683,"prepare.queue_time_us":534953,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":130474,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9171,"spinlock_wait_cycles":7795584,"thread_start_us":995137,"threads_started":3,"wal-append.queue_time_us":534037,"wal-append.run_cpu_time_us":7605,"wal-append.run_wall_time_us":1092809}
W20260709 12:26:33.870597 10888 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 12:26:33.954934 10905 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
I20260709 12:26:34.153339  7733 rpcz_store.cc:276] 0709 12:26:32.882690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.350627 (+467937us) service_pool.cc:224] Handling call
0709 12:26:33.354577 (+  3950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.354842 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:33.730642 (+375800us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.731140 (+   498us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.084688 (+353548us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.084843 (+   155us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:34.085042 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.085234 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.085701 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352413,"prepare.queue_time_us":481583,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":66724,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8786,"thread_start_us":334932,"threads_started":1}
I20260709 12:26:33.948230 10921 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:33.983744 10925 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.002s
I20260709 12:26:33.966181 10967 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:33.866781 10801 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:33.987589 10979 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.007s
I20260709 12:26:33.974527 10992 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:34.054801  7559 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 283) took 22860 ms (client timeout 15000 ms). Trace:
I20260709 12:26:33.779286 11051 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:33.981774 11059 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.003s
I20260709 12:26:33.946560 11072 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:26:34.128130  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.007s
W20260709 12:26:34.134300  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 464922us
I20260709 12:26:34.143730 10942 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:34.126005  7660 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.005s	sys 0.003s
I20260709 12:26:34.150619 11047 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:34.116508  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.153628  7736 rpcz_store.cc:276] 0709 12:26:32.935247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.972276 (+ 37029us) service_pool.cc:224] Handling call
0709 12:26:34.091936 (+1119660us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002ab554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:34.092272 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321350016}
W20260709 12:26:34.159482  7656 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.018s	sys 0.043s
I20260709 12:26:34.160876  8415 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:34.158428  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.163601  7657 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:34.163322  8409 rpcz_store.cc:276] 0709 12:26:29.952175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.952351 (+   176us) service_pool.cc:224] Handling call
0709 12:26:29.983762 (+ 31411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.557107 (+1573345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.873712 (+2316605us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:33.874014 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043804416}
W20260709 12:26:34.165005  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.004s	sys 0.014s
W20260709 12:26:34.165819  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 118266us
I20260709 12:26:34.192807  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:34.198333  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.007s	sys 0.007s
I20260709 12:26:34.182814  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.182981  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:34.212738  8411 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:26:34.183781  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.196754 11051 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.213979  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.206410  7658 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:34.214965  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.176014  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.214453 11051 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:34.211400  7660 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.005s	sys 0.003s
I20260709 12:26:34.216288  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 606) took 1230 ms. Trace:
I20260709 12:26:34.185679 10845 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.221199 10910 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.197610 11059 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.224362  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2498) took 1391 ms. Trace:
I20260709 12:26:34.224720  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2486) took 3325 ms. Trace:
I20260709 12:26:34.224831  7803 rpcz_store.cc:276] 0709 12:26:32.832514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.058935 (+226421us) service_pool.cc:224] Handling call
0709 12:26:33.069452 (+ 10517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.161812 (+1092360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.223880 (+ 62068us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001347d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:34.224259 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2368940288}
I20260709 12:26:34.225849 11059 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:34.227258 10907 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:34.227825  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:34.228583  8415 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:34.228933  8415 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:34.230226  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.230295  8415 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.195674 10801 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.233582  8408 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:34.233830 10796 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.235919  7811 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.238085  7735 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:34.175180 10819 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.239822  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.239991  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:34.225353  7809 rpcz_store.cc:276] 0709 12:26:30.899013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.625660 (+1726647us) service_pool.cc:224] Handling call
0709 12:26:32.675754 (+ 50094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.401198 (+725444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.223862 (+822664us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:34.224516 (+   654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665702144}
I20260709 12:26:34.242683  7736 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:34.228605 10907 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:34.194516 10979 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.245604 11016 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.218899  7731 rpcz_store.cc:276] 0709 12:26:32.967793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.394282 (+426489us) service_pool.cc:224] Handling call
0709 12:26:33.395766 (+  1484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.076479 (+680713us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:34.077301 (+   822us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.155335 (+ 78034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.155413 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:34.155490 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.197608 (+ 42118us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003b56d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:34.197684 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.198011 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173,"prepare.queue_time_us":49850,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2220,"raft.queue_time_us":31451,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":237,"spinlock_wait_cycles":1562805504,"thread_start_us":45050,"threads_started":1,"wal-append.queue_time_us":11496}
I20260709 12:26:34.241766  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.247716  8412 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.248024  7655 raft_consensus.cc:2410] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:34.248950 11016 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:34.249539 11016 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:34.247211 11092 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:34.251780 11092 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:34.186353  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:34.253064  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 129640us
W20260709 12:26:34.253700  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 649228us
W20260709 12:26:34.254124  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77372.4us
I20260709 12:26:34.255131  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.256250  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:34.198064 11072 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:34.261168 11072 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.262259 10948 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.185359 10770 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:34.263360  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.000s	sys 0.010s
W20260709 12:26:34.264174  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.000s	sys 0.011s
I20260709 12:26:34.264899 10839 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:34.266214  7657 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:34.193769 10967 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:34.267553  7659 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:26:34.268977 10975 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:34.269563 10975 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.270181 10975 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.189141 10905 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.280138  8415 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.280612 10942 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:34.282603 10905 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:34.270249  7806 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:34.283577  7653 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.189590 10921 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:34.285827 10921 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.263471 10770 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:34.288558 10770 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.274530  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.290650  7805 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:34.181668 11057 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:34.291898 11057 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.249548  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:34.251856 10896 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:34.275884  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.254602  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:34.289103  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.190582 10925 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.253614 10836 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:34.243651 11047 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.274273  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.288830 11092 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:34.244828 10907 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:34.191915 10781 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.009s
W20260709 12:26:34.298563  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.304108 10921 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.250943  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2106) took 1498 ms. Trace:
I20260709 12:26:34.308405  7809 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:34.301853  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47618.1us
I20260709 12:26:34.306926  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.321945 11058 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:34.322089  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.324759  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2118) took 1180 ms. Trace:
I20260709 12:26:34.325186  7734 rpcz_store.cc:276] 0709 12:26:33.144131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.443957 (+299826us) service_pool.cc:224] Handling call
0709 12:26:33.444343 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.089360 (+645017us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b5400167720. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e97a1d 00007fc001e7e1a4 00007fc001f00447 00007fc001f18fe7 00007fc001f1a7de 00007fc001f24b49 00007fc001f25416 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419
0709 12:26:34.324029 (+234669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.324072 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.324165 (+    93us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:34.324200 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.324277 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.324648 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38598,"raft.queue_time_us":39366,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":277,"spinlock_wait_cycles":222958848}
I20260709 12:26:34.303854 10734 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:34.327206 10734 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:34.319278  7729 rpcz_store.cc:276] 0709 12:26:32.752534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.311617 (+559083us) service_pool.cc:224] Handling call
0709 12:26:33.311965 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.350633 (+ 38668us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:33.350680 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.362157 (+ 11477us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.250190 (+888033us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:34.250348 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.250508 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.250598 (+    90us) raft_consensus.cc:1685] finished
0709 12:26:34.250621 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.250850 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37194,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":6517,"thread_start_us":802561,"threads_started":1,"wal-append.queue_time_us":346729}
I20260709 12:26:34.309020 11057 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.328732 11092 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:34.329260 11092 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:34.328853 10734 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.302731 11088 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:34.196234  7559 rpcz_store.cc:269] 0709 12:26:11.194397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.194584 (+   187us) service_pool.cc:224] Handling call
0709 12:26:34.054337 (+22859753us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:26.731150 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:26.748687 (+ 17537us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:26.748750 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:26:26.748767 (+    17us) tablet.cc:662] Decoding operations
0709 12:26:26.927478 (+178711us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:26.931045 (+  3567us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:26.941331 (+ 10286us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:26:27.459892 (+518561us) tablet.cc:701] Row locks acquired
0709 12:26:27.460036 (+   144us) write_op.cc:260] PREPARE: finished
0709 12:26:27.460259 (+   223us) write_op.cc:270] Start()
0709 12:26:27.460443 (+   184us) write_op.cc:276] Timestamp: P: 1783599987460166 usec, L: 0
0709 12:26:27.460467 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:27.461265 (+   798us) log.cc:844] Serialized 5525 byte log entry
0709 12:26:28.963107 (+1501842us) op_driver.cc:464] REPLICATION: finished
0709 12:26:29.574285 (+611178us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1f32a 00007fc001ea4f14 00007fc001ea4ec3 00007fc001ea4e38 00007fc001ea4df3 00007fc001ea40dc 00007fc001ecde16 00007fc001eb5cbb 00007fc001ea039d 00007fc001ea3d75
0709 12:26:29.712907 (+138622us) write_op.cc:301] APPLY: starting
0709 12:26:29.723652 (+ 10745us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:29.869544 (+145892us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:29.883615 (+ 14071us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:31.082293 (+1198678us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:31.082813 (+   520us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:26:31.082888 (+    75us) write_op.cc:312] APPLY: finished
0709 12:26:31.208990 (+126102us) log.cc:844] Serialized 101 byte log entry
0709 12:26:31.627165 (+418175us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.666630 (+ 39465us) write_op.cc:454] Released schema lock
0709 12:26:31.724136 (+ 57506us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:31.903247 (+179111us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.933365 (+ 30118us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":287582,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":648173,"apply.run_cpu_time_us":28793,"apply.run_wall_time_us":2151348,"num_ops":14,"prepare.queue_time_us":284400,"prepare.run_cpu_time_us":28848,"prepare.run_wall_time_us":970640,"raft.queue_time_us":375908,"raft.run_cpu_time_us":90265,"raft.run_wall_time_us":633992,"replication_time_us":1493867,"spinlock_wait_cycles":8295680,"thread_start_us":1282792,"threads_started":5,"wal-append.queue_time_us":746805,"wal-append.run_cpu_time_us":26755,"wal-append.run_wall_time_us":3166739}]]}
I20260709 12:26:34.319082 10896 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:34.301044  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:34.291095 10770 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.307613 10836 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.308260 11002 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:34.344871  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.308442 10796 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:34.347591  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.349376  7662 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:34.335512 10896 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.330696 11092 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:34.359251  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58331.4us
I20260709 12:26:34.360764  7811 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:34.343724 10836 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.364326  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 587) took 3682 ms. Trace:
I20260709 12:26:34.344027 11002 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:34.381690  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74820.7us
I20260709 12:26:34.395290  7658 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:34.357786 11072 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:34.399432  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 140359us
W20260709 12:26:34.407359  7654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.009s	sys 0.017s
W20260709 12:26:34.407881  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 221877us
I20260709 12:26:34.408811 10796 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.409727  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.411490 11010 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.412237 11010 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.413882  7655 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.415181  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:34.419275  7700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.021s	sys 0.010s
I20260709 12:26:34.417481 10978 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:34.419811  7700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.021s	sys 0.010s
I20260709 12:26:34.419876  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:34.420490  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 887326us
I20260709 12:26:34.419607  8409 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.421921  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.422796 11047 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:34.422972  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.424422 10907 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:34.426630 10811 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.960s	user 0.011s	sys 0.008s
I20260709 12:26:34.426707  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2346) took 2292 ms. Trace:
I20260709 12:26:34.382382 11088 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:34.429174  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:34.430177 11063 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:34.430881 10981 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:34.381992 10979 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:34.431499 10910 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:34.431990 10978 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:34.432907  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1895) took 3196 ms (client timeout 1744 ms). Trace:
I20260709 12:26:34.433825  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2112) took 1589 ms. Trace:
I20260709 12:26:34.371830  7660 rpcz_store.cc:276] 0709 12:26:30.664084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.113964 (+2449880us) service_pool.cc:224] Handling call
0709 12:26:33.155763 (+ 41799us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:33.156207 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.156235 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:33.156866 (+   631us) log.cc:844] Serialized 52 byte log entry
0709 12:26:33.516106 (+359240us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:34.250390 (+734284us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.332288 (+ 81898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.339771 (+  7483us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.343470 (+  3699us) raft_consensus.cc:1685] finished
0709 12:26:34.343490 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.345761 (+  2271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14568,"prepare.queue_time_us":260004,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":3922,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":15473,"thread_start_us":1017674,"threads_started":2,"wal-append.queue_time_us":323363}
I20260709 12:26:34.437055 11045 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.438074 10905 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.441001  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:34.411525 10967 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.408314  7809 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:34.407971  7654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.009s	sys 0.018s
I20260709 12:26:34.419513 11090 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:26:34.443434 10849 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.445914  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.447885  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.449242  7733 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:34.451967 11010 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.336233 10781 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.462955  7653 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:34.303963  7558 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:34.470436  7728 rpcz_store.cc:276] 0709 12:26:32.134180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.245697 (+1111517us) service_pool.cc:224] Handling call
0709 12:26:33.311645 (+ 65948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.312060 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.312090 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.312698 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.101580 (+788882us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18004024c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:34.425914 (+324334us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b54000fe9a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001e996b8 00007fc001f2292b 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c
0709 12:26:34.426008 (+    94us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:34.426074 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.426215 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.426263 (+    48us) raft_consensus.cc:1685] finished
0709 12:26:34.426279 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.426474 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7152,"raft.queue_time_us":876,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":7315,"spinlock_wait_cycles":150766720,"thread_start_us":56157,"threads_started":1,"wal-append.queue_time_us":924023}
W20260709 12:26:34.468969  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 183910us
I20260709 12:26:34.397440 11002 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:34.428762  8406 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.990s	user 0.011s	sys 0.007s
I20260709 12:26:34.350164 10975 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.431134  8409 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:34.335415 11016 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.457497 10839 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:34.485258  7662 rpcz_store.cc:269] 0709 12:26:31.236643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.034947 (+798304us) service_pool.cc:224] Handling call
0709 12:26:34.432514 (+2397567us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54002198d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:34.432788 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5054036736}
W20260709 12:26:34.491693  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.540s (SENT)
I20260709 12:26:34.470372 10925 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:34.522761  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93944.3us
I20260709 12:26:34.523320  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1906) took 2170 ms. Trace:
I20260709 12:26:34.524376  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.524824 10979 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.525589 10801 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:34.526374 10801 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:34.527029  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 79361.7us
I20260709 12:26:34.527731 10801 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.529141  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:34.526367  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:34.533231  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.534651  7558 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:34.536229  7660 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:34.536777  7558 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 6, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 6 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.539886  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:34.541440  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63184us
I20260709 12:26:34.545974  8412 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:34.490109 10811 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.486100  7732 rpcz_store.cc:276] 0709 12:26:32.844236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.323913 (+479677us) service_pool.cc:224] Handling call
0709 12:26:33.350538 (+ 26625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.359059 (+  8521us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:33.686656 (+327597us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.951168 (+264512us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b48002001d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:26:34.335954 (+384786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.350926 (+ 14972us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:34.375111 (+ 24185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.403632 (+ 28521us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.433575 (+ 29943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44675,"prepare.queue_time_us":407525,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":68135,"raft.run_cpu_time_us":5146,"raft.run_wall_time_us":598845,"spinlock_wait_cycles":5574016,"thread_start_us":741844,"threads_started":2,"wal-append.queue_time_us":451912}
I20260709 12:26:34.551250 10948 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:34.552203 10923 consensus_queue.cc:1175] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "992d6df95ef74f7f8b12534534a45501" responder_term: 10 status { last_received { term: 9 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:34.509588  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 86717.9us
I20260709 12:26:34.491650 11045 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.518420 10907 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.518239 11089 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:34.571506  8413 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 8. Ops: []
I20260709 12:26:34.531280 11016 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:34.512703  8415 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:34.558638  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 217225us
I20260709 12:26:34.542802 11092 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.538044  7660 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:26:34.554452 10925 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.576100  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1913) took 1074 ms. Trace:
I20260709 12:26:34.594153  7653 rpcz_store.cc:276] 0709 12:26:33.495590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.552953 (+ 57363us) service_pool.cc:224] Handling call
0709 12:26:34.516027 (+963074us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54003985d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:34.570429 (+ 54402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676510976}
W20260709 12:26:34.591174  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48948.1us
I20260709 12:26:34.558454 10796 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.591871  7655 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:34.603732  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1911) took 1101 ms. Trace:
W20260709 12:26:34.606050  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.608744  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.609781  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.611829  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 166158us
I20260709 12:26:34.571188  8406 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:26:34.612804  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 585) took 2599 ms (client timeout 1734 ms). Trace:
I20260709 12:26:34.613034  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:34.613296  8406 rpcz_store.cc:269] 0709 12:26:32.013685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.016413 (+  2728us) service_pool.cc:224] Handling call
0709 12:26:33.383622 (+1367209us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b58006764d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e7e0aa 00007fc001f00447 00007fc001f18fe7 00007fc001f1a7de 00007fc001f2627c 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c
0709 12:26:34.612643 (+1229021us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411189632}
I20260709 12:26:34.615561 11047 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:34.611748  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.523835  7656 rpcz_store.cc:276] 0709 12:26:32.352570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.267882 (+915312us) service_pool.cc:224] Handling call
0709 12:26:33.314390 (+ 46508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.314641 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:34.212789 (+898148us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.500236 (+287447us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.509035 (+  8799us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:34.522845 (+ 13810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.522989 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.523203 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25694,"prepare.queue_time_us":409799,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":17536,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":13883,"thread_start_us":1101552,"threads_started":2,"wal-append.queue_time_us":316335}
I20260709 12:26:34.546000 10905 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.585675 10845 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.584508 10819 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:34.576849 11057 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.583137 10811 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.605698 11010 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:34.604442  7661 rpcz_store.cc:276] 0709 12:26:33.495011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.302069 (+807058us) service_pool.cc:224] Handling call
0709 12:26:34.302710 (+   641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.303193 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:34.303963 (+   770us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.595581 (+291618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.595632 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:34.595702 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.595777 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.595901 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15155,"prepare.queue_time_us":1169,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":42171,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"thread_start_us":271313,"threads_started":2,"wal-append.queue_time_us":242038}
I20260709 12:26:34.631981  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1909) took 1138 ms. Trace:
W20260709 12:26:34.633546  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.635855  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.621057 11059 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.648691 10528 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:26:34.650171 11058 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:34.650462  7812 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.651746  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.654317  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:34.655715  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:34.656642 11100 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:34.656494  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:34.650211  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123993us
I20260709 12:26:34.658489  7804 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:34.655761 10987 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:34.661485 10987 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:34.661880  7733 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.663352 10969 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.663555  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:34.664438  8406 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.664141 10969 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.664867  7732 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:34.662775 10987 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.637121 10967 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:34.667207  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:34.667654  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:34.669951  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.819s (SENT)
I20260709 12:26:34.671427  7809 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:34.671840  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.672959  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 586) took 4067 ms. Trace:
W20260709 12:26:34.675482  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.675642 11089 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:34.676302 10975 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:34.668092  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:34.628834 11063 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:34.677477 11063 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:34.675715  7558 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 5, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.640597  7659 rpcz_store.cc:276] 0709 12:26:33.493907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.268638 (+774731us) service_pool.cc:224] Handling call
0709 12:26:34.269099 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.269446 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:34.295818 (+ 26372us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.589179 (+293361us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.623344 (+ 34165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.623428 (+    84us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:34.623502 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.623602 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.631883 (+  8281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66258,"prepare.queue_time_us":25741,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":138,"raft.queue_time_us":116615,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":278,"thread_start_us":254527,"threads_started":2,"wal-append.queue_time_us":206078}
I20260709 12:26:34.681990  7558 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:34.655743  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:34.683288  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:34.683660  7728 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:34.684012  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68879.7us
I20260709 12:26:34.669739  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.689963  7811 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:34.673413  7654 rpcz_store.cc:276] 0709 12:26:30.605324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.098657 (+2493333us) service_pool.cc:224] Handling call
0709 12:26:33.099027 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.099303 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.099332 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.099853 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.486662 (+386809us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:34.505371 (+1018709us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.619982 (+114611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.637950 (+ 17968us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.643857 (+  5907us) raft_consensus.cc:1685] finished
0709 12:26:34.643878 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.672858 (+ 28980us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59049,"prepare.queue_time_us":289128,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":60906,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":15481,"thread_start_us":1349158,"threads_started":2,"wal-append.queue_time_us":555960}
I20260709 12:26:34.626322 11047 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:34.634464 11088 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:26:34.702370  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.666955 10969 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.709466 11002 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.711175  7558 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 3, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:34.713696  7558 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 4 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.712225  7730 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:34.700121  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:34.718283  7728 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:34.625553 10925 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:34.720541  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:34.720746  7558 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:34.722242 10925 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:34.726486  7807 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:34.722577  7558 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 6, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:26:34.729835  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.731956 11002 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:34.727325 11047 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.732950 10781 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:34.733166  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:34.734145  7730 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.714547 11010 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.738142 11010 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.740047  7558 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:34.741694 10528 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:34.742357 10528 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:34.742950  7558 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.743984  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:34.752418  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.753933  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.755125 10910 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.755431 11042 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:34.755658 10796 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:34.756526 10796 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:34.757700 10796 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.743672 10528 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.759742  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:34.760257 10528 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.760619  7811 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:34.760838  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66327.7us
I20260709 12:26:34.744508 10905 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:34.787693  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67875.8us
I20260709 12:26:34.640713 11051 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.747920 10819 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:34.748884 11002 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:34.697000 10979 raft_consensus.cc:2764] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:26:34.789671  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2212) took 1126 ms. Trace:
W20260709 12:26:34.793161  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45407.8us
I20260709 12:26:34.794653  7730 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:34.794170  7558 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 1 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:34.813694  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:34.808647  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.750967 10770 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:34.822099  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:34.824649  7657 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.813078 11089 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:34.829754  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:34.819957 10975 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:34.824504  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:34.795110 11072 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 12:26:34.826259 10937 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.006s
I20260709 12:26:34.843961  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:34.843598  7804 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:34.846439  7811 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:34.849454  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:34.849989 10845 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:34.851289  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:34.860386 11045 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.861649  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:34.862609  7661 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.837783 10811 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:26:34.879379 10925 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:34.856612 10770 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.853516 10937 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.889897  8406 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:34.890661 10937 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:34.853574 11089 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:34.903357  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:34.918007  7558 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:26:34.920519  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51647.3us
I20260709 12:26:34.926240  8413 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:34.927299  7653 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:34.856420  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:26:34.928633  7809 raft_consensus.cc:606] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:34.929059  7809 raft_consensus.cc:614] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:34.899766 10770 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:34.929615  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 81269.1us
W20260709 12:26:34.934353 11114 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.329s	user 0.005s	sys 0.000s
I20260709 12:26:34.935036 10781 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.935465 11045 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:34.936017 10801 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.925755  7558 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:34.933808  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:34.837381 10987 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:26:34.901108 10975 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:34.950774  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:34.935074  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:34.951741 11111 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.332s	user 0.003s	sys 0.000s
I20260709 12:26:34.953125 11089 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:34.916404 10925 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:34.955958 10967 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.865211 11088 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:35.005270 10781 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:35.005733 10781 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:35.007725  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.008797 10811 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:34.958357  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:34.900036 10896 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.004118 11045 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:26:35.007889  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74150.6us
I20260709 12:26:35.010216  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:34.965739  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.016970  7659 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.008842  7654 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:35.043210  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.986531 11002 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.009503 10811 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:35.063669  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.064028  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:35.063669  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.066944  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:35.065001  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:35.014051 11058 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:35.028894 10987 raft_consensus.cc:2764] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:26:35.071123  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.074565  7662 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:35.076433  7734 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:35.016147 11045 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:35.091049  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.099378  7653 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:35.006870 10781 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:34.984483 10836 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.115442  7654 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:35.118425  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.119338  7661 raft_consensus.cc:2376] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7. Re-sending same reply.
W20260709 12:26:35.107867  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 108135us
I20260709 12:26:35.121109  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.121840  7653 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.126341  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:35.050261 10967 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.133498  8415 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:26:35.137042  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178813us
I20260709 12:26:35.108889 11051 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.096522 11002 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.165781  7733 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:26:35.157591  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.161566 10967 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.163328  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:35.180171  7653 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:35.098917 11063 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.182686  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.185019  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.189555  7729 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:35.188737 10875 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.200100  7655 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:26:35.171577 10781 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.200588  7655 pending_rounds.cc:85] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 2
I20260709 12:26:35.189628 11057 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.224753 11126 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:26:35.225211 11117 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:26:35.256800  7654 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:35.193742 10875 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.264118  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.245901  7659 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:35.212976  7655 pending_rounds.cc:107] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:26:35.266330  7655 raft_consensus.cc:2889] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.123143 11058 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:35.260196  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 130368us
I20260709 12:26:35.138043 10975 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:35.277870 11127 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:26:35.299482  7661 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:35.300189  7661 pending_rounds.cc:85] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 5
I20260709 12:26:35.300499  7661 pending_rounds.cc:107] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:35.300858  7661 raft_consensus.cc:2889] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:35.307806 10528 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.009s
I20260709 12:26:35.309209 10875 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:35.310563  7654 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:34.806716  7658 rpcz_store.cc:276] 0709 12:26:33.663631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.338413 (+674782us) service_pool.cc:224] Handling call
0709 12:26:34.369754 (+ 31341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.788861 (+419107us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:34.788920 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.789007 (+    87us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:34.789184 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.789340 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.789562 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18334,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":74915}
I20260709 12:26:35.312498  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 603) took 1243 ms. Trace:
I20260709 12:26:35.313112  7662 rpcz_store.cc:276] 0709 12:26:34.069700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.550164 (+480464us) service_pool.cc:224] Handling call
0709 12:26:34.550705 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.082120 (+531415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.312148 (+230028us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54004353d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:35.312413 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159522560}
I20260709 12:26:35.314534  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:35.317644  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61151.8us
I20260709 12:26:35.164402 10925 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.327417 10845 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:35.328233 10845 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.333431  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:35.332966 11130 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:35.334816  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:35.335503 11130 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:35.336869 11130 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:35.336436 11125 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:26:35.339505  7735 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:35.344732  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2207) took 1695 ms. Trace:
I20260709 12:26:35.344265  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:35.351333  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:35.339978 10836 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.342293 11058 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.332538 10969 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:35.330917 10845 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:35.354230  7657 rpcz_store.cc:276] 0709 12:26:33.649860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.330019 (+680159us) service_pool.cc:224] Handling call
0709 12:26:34.330604 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.827418 (+496814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.344476 (+517058us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54000d02d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:35.344655 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125912320}
I20260709 12:26:35.369506  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:35.418449  7731 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:35.418624  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74445.3us
I20260709 12:26:35.284814 10849 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.430820  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 134014us
W20260709 12:26:35.429318  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59930us
I20260709 12:26:35.344486 10896 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.429625  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.212015 10811 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.445454 10849 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.371764 10969 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:35.298246 11057 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.462895 10968 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:35.464954 11057 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.466351 11063 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.467255  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:35.247826 11045 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.470700  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:35.472234 10969 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.473575 10969 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:35.459409 11047 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:35.475178 11047 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.475674 11047 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.460239 10981 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:26:35.460886 11128 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 12:26:35.461477 10910 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:35.430073 10942 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:35.472874 11002 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:35.453796  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.459848 11130 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.497928  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:35.501554  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:35.510953  7811 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:35.505611  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.508762  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.457574 10875 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.490852 11016 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.912s	user 0.008s	sys 0.005s
I20260709 12:26:35.480881 11058 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:35.511184  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:35.516659  7804 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:35.513607  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.520367  7735 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:35.512042  7733 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:35.517653  8415 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEARNER]: Advancing to term 7
I20260709 12:26:35.525173  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:35.529649  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:35.525167  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:35.502306 11063 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:35.481315 10845 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:26:35.546845  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:35.539057  8413 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.003s
I20260709 12:26:35.541018  7807 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:35.538847  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:35.549140  8413 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:35.550938  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:35.542769 10978 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:35.581660  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52123.7us
I20260709 12:26:35.592057  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:35.582088 11063 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:35.586355 11072 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.589494  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.507287 11057 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:26:35.596797  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.527611 10875 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.560541 10907 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:35.545172 11088 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:26:35.590235  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:35.591159 10978 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:35.605027 10845 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.640810  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.624727  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:35.625078  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:35.615715 11016 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:26:35.625883 10836 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:35.627498 11051 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.608050 11002 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:35.628376 11139 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:35.629355 10910 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:35.631521 11057 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:35.633054 11088 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:35.621461  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.640462 10978 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:35.662343  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 215841us
W20260709 12:26:35.670840  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 129911us
I20260709 12:26:35.609759 10975 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:26:35.673998  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48989.6us
W20260709 12:26:35.681748  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57102.7us
I20260709 12:26:35.689005  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.690073  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:35.616737 11072 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:35.692981  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.010s	sys 0.010s
W20260709 12:26:35.701233  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.011s	sys 0.011s
I20260709 12:26:35.700886  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:35.704515  8415 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEARNER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:35.693177  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:35.757771  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51243.7us
I20260709 12:26:35.677953 11057 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:35.672816 11139 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:35.797995  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109007us
I20260709 12:26:35.678282 11088 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:35.799019  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:35.808017  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 118033us
I20260709 12:26:35.809999 10925 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:35.814059 11045 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:35.701455 10987 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:35.682272 10942 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 12:26:35.672823 11051 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.815928  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:35.757426 10907 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:35.682509 11058 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:35.798897 11151 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:35.705713 10849 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.843124  7804 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:35.715034 10796 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.877691  7658 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:35.850220  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.843781 10925 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:35.885367  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.854737  7662 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:35.816634 10875 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:35.759390  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58600us
I20260709 12:26:35.854259  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:35.857519  7657 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:35.833889  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.873504  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:35.873277  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:35.895546  7660 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:35.757772 11016 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:35.708149 10845 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.890103  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:35.891009  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:35.898725  7730 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:35.899768  7812 raft_consensus.cc:2410] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:26:35.904651  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61120.4us
I20260709 12:26:35.910145  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:35.924109  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:35.871085 11151 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:35.894342 10925 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:35.886966 10811 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.955652 10811 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:35.955974 11155 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:35.940619 10911 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:35.906730 11016 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:35.940892 11133 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.941937  8408 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:35.959651  7730 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:35.942112 11051 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:35.942905 11140 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:26:35.946869 11047 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:35.948184 11072 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:35.949515 11136 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.950126 11144 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:26:35.938524 11088 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:35.952279  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46802.1us
I20260709 12:26:35.988319  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:35.952502 10907 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:35.939716  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:35.953728 10979 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.940917  7807 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:35.934085 11151 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:35.930480 11089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.055s
I20260709 12:26:35.975934  7812 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:35.977020  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.027410  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:36.028396  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51478.9us
I20260709 12:26:36.028388  8412 raft_consensus.cc:2376] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7. Re-sending same reply.
W20260709 12:26:35.969550 10781 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.005s
I20260709 12:26:35.989491  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:36.033996  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:36.052220  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:36.054499  8415 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:36.060861  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.066962  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:36.032712 11089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.017s	sys 0.056s
I20260709 12:26:36.072978  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.089865  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:35.933756 11057 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:36.091974 11144 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:36.104272 10910 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:36.087126 10781 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:35.989751 11133 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.106614 10975 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:36.125438 10836 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:26:36.071404 11072 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:36.117121  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.139035 11059 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:36.071179 11161 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:36.146489  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.148015  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:36.171808  7700 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.734s (SENT)
I20260709 12:26:36.088899 11156 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:36.169548 10942 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:26:36.181741  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.145079 11088 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:36.068019 11136 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.127550 11143 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 12:26:36.193092  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 299984us
W20260709 12:26:36.196646  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.100194 10979 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.197171  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.092352 11058 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.209909  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.185271 11072 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.192813 10910 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.222644  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.193485 11156 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:36.194597 10942 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:36.351657  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 179869us
I20260709 12:26:36.362761  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.201344 11161 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.342783  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 391374us
I20260709 12:26:36.345799  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:36.149386 11016 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.201035 10811 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:36.406478  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.404429  8407 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 12:26:36.414113  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68413.6us
I20260709 12:26:36.415791  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:36.366212 10819 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:36.433631  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.477476  7729 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:36.477294  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 404387us
I20260709 12:26:36.207182 10925 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.480299  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:36.490468  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 344121us
I20260709 12:26:36.428676 11164 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.492453  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.492508 11164 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.493431  7734 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:36.495328 11051 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:36.496526 11051 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.497013 11016 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:36.439509 11047 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:36.499749 10845 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:36.501713 11163 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 12:26:36.514513  7660 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 12:26:36.516362  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.428197 11089 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:36.545454  7804 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:36.438241 10819 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:36.523794  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.501699 11051 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.631901 11051 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:36.580657  7660 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:36.634366 11160 raft_consensus.cc:2764] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:26:36.634696 11045 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.785s	user 0.010s	sys 0.003s
I20260709 12:26:36.549827 10849 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.390709 11155 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.635890  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.635797 10849 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.549477 10836 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:26:36.636499 11155 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.638649 10849 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.628942  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:36.639938  7734 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:36.641508  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.636579  7737 raft_consensus.cc:2376] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
I20260709 12:26:36.428938 10910 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.646147 10910 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:36.648175 10979 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.655768  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:36.407325 11161 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:36.524685 10978 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.003s
I20260709 12:26:36.660518 10978 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:36.673988 11150 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.568s	user 0.001s	sys 0.000s
W20260709 12:26:36.688468  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.537568 11170 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.663919 11154 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:36.754145 10979 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:36.756049  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 239805us
I20260709 12:26:36.755472  7734 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:36.546373 10801 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:36.759502  7804 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:36.765226  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:36.748705 10896 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:36.773017  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.777117  7728 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:36.748736 10781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.031s	sys 0.083s
I20260709 12:26:36.759460 10910 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.759174 10845 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:36.793159  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 386375us
I20260709 12:26:36.773283  8406 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:36.797657  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 381950us
I20260709 12:26:36.773891 11143 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:36.806341  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:36.808661  7655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.010s	sys 0.009s
W20260709 12:26:36.809090  7655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.010s	sys 0.009s
W20260709 12:26:36.782485 10781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.032s	sys 0.083s
I20260709 12:26:36.789480 10801 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:36.796391 11042 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:36.794744 11164 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.811190 11133 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.783259 10819 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.776526 10896 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.830724  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:36.828083  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.814702 10781 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:36.843050  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.825831 11051 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.770388 11170 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.860152  7728 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 4: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:36.826231 11130 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:36.860738  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 337036us
I20260709 12:26:36.799779 10910 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.799580 11154 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.835675 10845 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:36.842137 10849 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.858703  7734 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:36.867516  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61279.8us
I20260709 12:26:36.842295 11173 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.846230 11164 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.876209  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:36.852310 10925 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:36.856231 11136 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:36.884948  7734 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:36.859699 11072 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:36.882994  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.521s (SENT)
I20260709 12:26:36.864616 10819 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:36.891938  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:36.779569 11156 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:36.879529 11159 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.588s	user 0.003s	sys 0.002s
I20260709 12:26:36.904589  7657 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:36.876722 11166 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.909076  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.001554  7660 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:37.020566  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:37.005376  7654 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:37.009210  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.021685  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.014050  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.015583  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.029397  7661 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:37.032150  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156114us
I20260709 12:26:37.042021  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:36.888788 11045 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.135s
I20260709 12:26:36.882341 11143 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:26:36.885396 10849 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.001935 11059 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.880149 11051 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:36.899348 11136 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:37.068383  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.068784  7737 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:37.025732 11072 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:36.907994 11155 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.079917  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.087680  7656 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:37.095604  7654 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
W20260709 12:26:37.045614 10969 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.572s	user 0.013s	sys 0.006s
I20260709 12:26:37.001179 11047 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 12:26:36.905966 11164 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.011453 10836 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:26:37.097337  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.108695  7736 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
W20260709 12:26:37.098425  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 216450us
I20260709 12:26:37.064625 10978 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.076689 11045 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:26:37.083050 10979 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.118010 11182 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:26:37.118399 10836 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:26:37.119026  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.551s (SENT)
I20260709 12:26:37.019505 10801 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.125257  7658 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:37.117736 11147 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.266s	user 0.007s	sys 0.010s
W20260709 12:26:37.134313 11177 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
I20260709 12:26:37.135644 10896 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.120512 10845 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.120500 11072 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.136832 10975 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:37.134980 11151 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.042s	user 0.006s	sys 0.006s
I20260709 12:26:37.136953 11161 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.127693 11193 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.139312 11183 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:37.139835 11172 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 12:26:37.141258 11188 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.141744 11181 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:37.106674 10925 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:26:37.086531 10968 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:26:37.099146 11136 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.127027 11154 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:37.144038 11185 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:37.145816  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.147784  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.140129 11176 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.253s
I20260709 12:26:37.080232 10781 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.159088  7659 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:37.173825  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.142309 11190 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:37.165869 10796 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.279s	user 0.009s	sys 0.006s
I20260709 12:26:37.179340  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.101658 11042 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:26:37.086616 11059 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.158329 10975 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:37.207756  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:37.207962  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:37.155493 10979 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.218396  7736 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:37.177853 11178 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.208464 10910 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.215896 10796 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:37.224303  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.237838  7737 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 4.
W20260709 12:26:37.226364  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.226775  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.229592 11184 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:26:37.229758 10975 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:36.889240 11173 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.044756 11166 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.168589 11151 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:37.253788  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 143821us
I20260709 12:26:37.181775 11188 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.259927  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.219601 11051 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.173106 11183 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.172585 11193 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:37.254516  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.558s (SENT) [suppressed 39 similar messages]
W20260709 12:26:37.266445  7627 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.615s (SENT)
I20260709 12:26:37.266878  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.263062  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:37.248134 11180 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:37.256729 10975 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.275574  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:37.279075  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196963us
I20260709 12:26:37.280038  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.264052 10910 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.281790  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:37.271991 11016 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.774s	user 0.016s	sys 0.000s
I20260709 12:26:37.274309 11171 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.276269 11051 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:37.314152  7729 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:37.315070  7777 leader_election.cc:400] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
I20260709 12:26:37.300204 10925 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:37.322798  7777 leader_election.cc:403] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:26:37.324049 10801 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
I20260709 12:26:37.323750  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.324995  7729 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:37.326051  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.327680  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.328997  7728 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:37.330008  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.327728 11195 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:37.332508 11195 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.334998 11139 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.336556 11166 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.337392 11195 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.336369 11139 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:37.303190 10528 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:37.346875  7728 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:26:37.319967  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53549.6us
I20260709 12:26:37.348768  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.350565  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.351616  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.246904 11178 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.353018  7656 pending_rounds.cc:85] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 1
I20260709 12:26:37.352279  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:37.372108  7656 pending_rounds.cc:107] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:26:37.357923  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.372610  7656 raft_consensus.cc:2889] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.311594 11143 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.379590  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.673s (SENT)
I20260709 12:26:37.383482  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.384420  8412 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:37.380507 11042 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.305686 10910 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.388365  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:37.389357  7729 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.353957 10849 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.356086 10528 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.390976  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.391516  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.377791 10836 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:37.400480 11130 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.004s
W20260709 12:26:37.401580 11199 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:26:37.411298 11196 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:26:37.381971 11188 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.416635  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:37.417613 10819 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.002s
I20260709 12:26:37.406157 10836 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:37.418646 11088 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.225s	user 0.005s	sys 0.006s
I20260709 12:26:37.418919  7731 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:37.402742 10528 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:37.404645 11045 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.422178 11045 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.426461 11130 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.427436 11187 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.427064 11130 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.428104 11187 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.428831  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:37.428012  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.387737 10835 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:37.383495 11164 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.394928 10968 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.429925 11059 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:37.448388  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.464475  7728 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:37.399590 11042 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.445176  7809 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:37.459435  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:37.462350  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.462920  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:37.418646 10836 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:37.473011  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.473726  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.474498  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:37.474540 11042 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:37.475405  7807 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:37.475553  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.474598  7805 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:37.476931  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.477941  7728 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:26:37.430188 11130 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.480140  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:37.482124 10925 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:37.486029  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:37.316479 11171 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.473764  8415 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:37.488215 11042 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:37.488849 11042 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.471388 11072 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.490118  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.497865  7728 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:37.404583 10849 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:37.480343 11178 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.491089  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 612) took 1628 ms. Trace:
I20260709 12:26:37.514070  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.501013  7803 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:37.382627 11143 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.507933  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:37.514842  7728 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:37.497294 11042 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:37.488972 11171 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.517311  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:37.528290  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:37.530009  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:37.533362  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 176258us
I20260709 12:26:37.533587  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.512846  7655 rpcz_store.cc:276] 0709 12:26:35.862768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.862886 (+   118us) service_pool.cc:224] Handling call
0709 12:26:35.877363 (+ 14477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.905518 (+ 28155us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:36.092294 (+186776us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.822669 (+730375us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.830443 (+  7774us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.830620 (+   177us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:36.830884 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.020391 (+189507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.025256 (+  4865us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.490792 (+465536us) raft_consensus.cc:1685] finished
0709 12:26:37.490823 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.491011 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33009,"prepare.queue_time_us":292037,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":98,"raft.queue_time_us":166624,"raft.run_cpu_time_us":626,"raft.run_wall_time_us":25537,"thread_start_us":1017518,"threads_started":3,"wal-append.queue_time_us":752785}
I20260709 12:26:37.538892  7811 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:37.452014 10835 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:37.540987  7627 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.726s (SENT)
I20260709 12:26:37.506878 11136 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.544668  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.546075  7729 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:37.548669  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.468075 10968 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.551033  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.549446  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:37.564390  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:37.568383  7811 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:37.570469  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.572571  7806 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:37.578493  7729 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:26:37.578670  7808 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:37.540326 10801 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:26:37.579463  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.579468 10801 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.585950 10801 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:37.586367 10528 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.589246 10528 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:37.590070  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.566781 11164 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:37.585131 10910 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:37.558857 11072 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.558735 10925 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:37.579113  7729 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:26:37.591166  7662 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.604487 10968 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:37.604732  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 526) took 1737 ms (client timeout 1953 ms). Trace:
I20260709 12:26:37.605368  7729 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:37.605223  7809 rpcz_store.cc:269] 0709 12:26:35.867813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.894715 (+ 26902us) service_pool.cc:224] Handling call
0709 12:26:37.604510 (+1709795us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:37.604632 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095934336}
W20260709 12:26:37.607251  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66298.2us
W20260709 12:26:37.608045  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.759s (SENT)
I20260709 12:26:37.608117 11201 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.608547  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.609398 11088 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:37.608758 11201 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.594565  7728 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:37.611691 11201 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.612633  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.606827  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.604310  7803 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:37.598017 11176 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:37.671921 11176 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.672523 11176 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:37.679790  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.819s (SENT)
I20260709 12:26:37.598677  7736 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:37.603822  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.682085  7811 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:37.597150 10835 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.685175  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.549268 11202 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:37.686411  7728 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:37.696779  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:37.595789 11143 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.701941  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:37.598368 11193 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:37.704324  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.598389 11170 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.598232 11183 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.680394 10925 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:37.693892 10845 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.708217  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3938056659999347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 12:26:37.722110  7702 connection.cc:256] RPC call timeout handler was delayed by -2.358705665999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 12:26:37.722445  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3479056659998605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
W20260709 12:26:37.722810  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0734056659998714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 12:26:37.723918  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0086056660002214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
I20260709 12:26:37.702818 11164 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:37.698696 11202 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:37.708916 10911 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.719003  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:37.724344  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9348056660001021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 12:26:37.722573  7728 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:37.722636  7662 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:37.715520 11201 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:37.596803 11072 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.738271 11072 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.738199  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.739356 11202 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.741314  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:37.728734  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5469056659999296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.258s for a response
I20260709 12:26:37.717163 11139 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:37.743073  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.730000  7811 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:37.733366  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2149) took 1876 ms. Trace:
W20260709 12:26:37.728612  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115168us
I20260709 12:26:37.744449  7736 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:37.746924  7731 rpcz_store.cc:276] 0709 12:26:35.857415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.857968 (+   553us) service_pool.cc:224] Handling call
0709 12:26:37.733185 (+1875217us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54000721d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:37.733304 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3524480256}
I20260709 12:26:37.745293  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.748948  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:37.748993  7702 connection.cc:256] RPC call timeout handler was delayed by -1.44520566600022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26930000000000004s for a response
I20260709 12:26:37.744560  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:37.750728  7627 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.829s (SENT)
I20260709 12:26:37.751359  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.760061 11173 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.756253 11190 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:37.765010  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.731151 11201 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:37.767150  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.768989  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:37.769165  7662 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:37.765710  7654 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:37.770534  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.779917  7812 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:37.781329  7812 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:37.783727  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.784512  7660 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:37.777107  7803 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:37.771591  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 12:26:37.777052 11212 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:26:37.731303 10845 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.791960  7811 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:26:37.796419  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49191us
I20260709 12:26:37.782925 11201 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:37.797372  7812 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:37.798272  7803 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:37.787624 11167 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:37.800828  7734 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:37.751405 11139 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:37.797129  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.836s (SENT)
I20260709 12:26:37.806576  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.804601  7734 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:37.809252 10801 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:37.809857  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.838s (SENT)
I20260709 12:26:37.810396 11190 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:37.811051  7734 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.804656  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.804468 11167 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:37.813057 11042 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:37.745368 10911 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:37.814486  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64401.3us
I20260709 12:26:37.831357  7734 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.837832  7809 raft_consensus.cc:2368] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9.
I20260709 12:26:37.823451 11167 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.836647  7654 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:37.841106  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.589s (SENT)
W20260709 12:26:37.850334 11222 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:26:37.850718 11139 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.791595 10925 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:37.855935  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:37.858398  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.633s (SENT)
W20260709 12:26:37.858989  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.859329 10801 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:37.859047  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.859880  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.861119  7656 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:37.861176 11190 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:37.862171  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.862833  7656 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:37.864475  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.949s (SENT)
I20260709 12:26:37.868176  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:37.868749  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.870541  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.870602  7654 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:37.875418  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:37.876471  7660 raft_consensus.cc:2410] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:37.874904  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.880039  7734 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:26:37.882292  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:37.897207  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.898579  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.883635  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.884377  8405 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:37.884635  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.902482  8408 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:37.885454  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.903725  8415 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:37.886523  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:37.888449  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.905697  8411 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:37.889963  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.891065  7806 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:37.879190 10987 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:37.879475 11190 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:37.909438  8409 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:37.911428  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.911973  7656 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:37.912611  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.912804  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.913524  7734 pending_rounds.cc:85] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
W20260709 12:26:37.899648  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50983.2us
I20260709 12:26:37.913691  7812 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:37.899204  7811 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:37.913870  7734 pending_rounds.cc:107] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:26:37.905758  7660 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:37.889058 10528 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.913311  7653 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:37.914179  7734 raft_consensus.cc:2889] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.895680  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.852298 11176 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.916527  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.918673  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:37.908695 11209 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.487s	user 0.003s	sys 0.000s
I20260709 12:26:37.922030  7662 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:37.922621  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.922081  7803 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:26:37.925436 11042 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.926671  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:37.926827  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:37.927722  7811 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:37.925558  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:37.932057  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67699.4us
I20260709 12:26:37.937001  7806 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:37.940219  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:37.933638  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.074s (SENT)
I20260709 12:26:37.942619  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:37.941362  7812 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:37.904281 11202 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:26:37.973237  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:37.916705 11186 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:26:37.950506 10987 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:37.922415 10528 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.954950 10845 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:37.963065 11190 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:37.984946  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51351.4us
W20260709 12:26:37.990640  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.083s (SENT)
I20260709 12:26:37.992440 11173 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.995806 11173 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:37.998634 11173 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:37.991907 11201 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.000926 10528 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:38.002024 11201 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.005838  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.006796  7808 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:37.994012 10987 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:38.010133 11174 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.173s	user 0.002s	sys 0.000s
I20260709 12:26:37.987989 10911 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:38.010572  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:38.016865  8411 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.015291  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:38.017067  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48534.7us
W20260709 12:26:38.019109 11002 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.093s	user 0.018s	sys 0.005s
I20260709 12:26:38.019034 10801 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:38.019685 11002 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.110s	user 0.019s	sys 0.005s
I20260709 12:26:38.019838 10801 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.020610 10801 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:38.021643  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.155s (SENT)
I20260709 12:26:38.022210  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.038206  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2146) took 2332 ms. Trace:
I20260709 12:26:38.039060  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.038113  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.052436  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:38.038838  7735 rpcz_store.cc:276] 0709 12:26:35.689488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.689700 (+   212us) service_pool.cc:224] Handling call
0709 12:26:35.798752 (+109052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.832652 (+ 33900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.832687 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.858428 (+ 25741us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.809284 (+1950856us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b180043b2e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:37.949307 (+140023us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:37.949349 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.949419 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.011117 (+ 61698us) raft_consensus.cc:1685] finished
0709 12:26:38.011142 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.021418 (+ 10276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75345,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":31022,"spinlock_wait_cycles":21091200,"thread_start_us":139779,"threads_started":1,"wal-append.queue_time_us":2107477}
I20260709 12:26:38.059463  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:38.065079  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.188s (SENT)
I20260709 12:26:38.051020 11173 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.021523 10528 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:26:38.071638  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2150) took 2087 ms. Trace:
I20260709 12:26:38.076339  7730 rpcz_store.cc:276] 0709 12:26:35.984339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.984822 (+   483us) service_pool.cc:224] Handling call
0709 12:26:35.985343 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:36.016374 (+ 31031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.016427 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.026996 (+ 10569us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.369434 (+342438us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:36.369483 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.369582 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.009406 (+1639824us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:38.060731 (+ 51325us) raft_consensus.cc:1685] finished
0709 12:26:38.060755 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.071522 (+ 10767us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":722,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9872,"spinlock_wait_cycles":10226048,"thread_start_us":276813,"threads_started":1,"wal-append.queue_time_us":720041}
I20260709 12:26:38.080204  7811 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:38.083937  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.085009  7808 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:38.068526 10836 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.005s
I20260709 12:26:38.060356 10987 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:38.060249  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.068864 11186 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.096319  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:38.098243  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.100777 11201 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:38.101320 11201 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.101714  7660 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:26:38.101816  7808 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:38.100850  8405 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:38.104640 11202 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:38.101716 11201 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:38.107767  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:38.112468 11139 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:38.124225  8411 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:38.117996 10987 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:38.129853  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74448.5us
I20260709 12:26:38.123813  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:38.088698 10836 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.125923 11176 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:38.142870  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.132725 11190 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.133121 10987 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:38.134864 11139 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:38.154495  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 529) took 2173 ms (client timeout 1828 ms). Trace:
I20260709 12:26:38.157317  8414 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:38.159287  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:38.158540  7804 rpcz_store.cc:269] 0709 12:26:35.976585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.989151 (+ 12566us) service_pool.cc:224] Handling call
0709 12:26:38.149902 (+2160751us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54003737d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.150040 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177150720}
I20260709 12:26:38.149353 11051 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.165584  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:38.164276 11202 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.168402  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.168280 11202 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:38.168931  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.170339  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.170580 11202 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:38.170796  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:38.170997  7730 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:38.171484  7734 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:38.174089  8405 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:38.174672  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.175832  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.176250  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.176748  8405 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:38.164582 11164 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.157696 10987 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:38.182126  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:38.172608  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:38.190627  7730 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:26:38.192015  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.192669  7730 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.199047  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:38.203907 11231 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:26:38.158047 11190 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.194001 10528 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:38.201483 11057 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.194115 11166 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 12:26:38.210310  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55724.2us
I20260709 12:26:38.207828 10796 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:38.196138 11176 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:26:38.239550  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.244330  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.246922  7734 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:38.250103 11164 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.213569 10528 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.253880  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.256510 11051 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.259316 10528 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:38.259840  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:38.260380 11151 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:38.261581  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.236137 10836 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.279946  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:38.258915 10979 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.248365 10796 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.287938  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.291603  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.292697  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:38.290951  7807 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:38.314404  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.316663  7730 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:38.340621  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.291368 11051 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:38.293812 10796 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:38.346500  7734 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:38.348004  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.364142  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:38.365468  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.368891  8405 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:38.430507  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.350586 11164 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:38.316617 11173 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 12:26:38.435243  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.378190  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.379521 11045 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:38.358248 10836 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:38.387392  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.388487  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:38.406473  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.444384  7660 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:38.444634  7804 raft_consensus.cc:2376] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7. Re-sending same reply.
I20260709 12:26:38.431861  7734 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:26:38.420703  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.446880  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.432058  8405 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:38.433435  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 598) took 2398 ms. Trace:
I20260709 12:26:38.436771  7730 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:38.448681  8407 rpcz_store.cc:276] 0709 12:26:36.030349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.030542 (+   193us) service_pool.cc:224] Handling call
0709 12:26:36.030978 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.411832 (+380854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.427552 (+2015720us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:38.429075 (+  1523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755897088}
W20260709 12:26:38.419555  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126935us
I20260709 12:26:38.431348  7806 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:38.452960  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.454046  7804 raft_consensus.cc:2376] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8. Re-sending same reply.
I20260709 12:26:38.434008 11164 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.455071  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:38.456334  8405 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.459698 11201 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.496018  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:38.473832 11162 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.049s	user 0.004s	sys 0.000s
I20260709 12:26:38.461342 10528 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:38.471256 11138 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:38.504623 11190 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:38.517974 11239 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:26:38.503836 11164 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:38.498643 11219 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.750s	user 0.002s	sys 0.000s
I20260709 12:26:38.443919 10836 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.472067  8405 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:38.516515  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:38.529266 11201 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.472172  8415 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:38.512426 11168 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.471047  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.480659  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.587602  7804 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:38.488216  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 881) took 2668 ms. Trace:
I20260709 12:26:38.482318  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.592942  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:38.530117  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1944) took 1626 ms. Trace:
W20260709 12:26:38.531246  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1946) took 1625 ms (client timeout 1639 ms). Trace:
W20260709 12:26:38.594025  7659 rpcz_store.cc:269] 0709 12:26:36.905942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.008873 (+102931us) service_pool.cc:224] Handling call
0709 12:26:38.531039 (+1522166us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540017b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.531160 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301776640}
I20260709 12:26:38.542248  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.536955  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:38.542865 11051 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.477293  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:38.485833 11228 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.386s	user 0.002s	sys 0.000s
I20260709 12:26:38.543092  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:38.543743 11167 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.004s
I20260709 12:26:38.544286  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.544410  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.545274  7737 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:38.545599  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:38.543648  8413 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:38.497898 10801 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.556252  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:38.617691  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 140496us
I20260709 12:26:38.619691  7810 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:38.621778  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 65673.6us
I20260709 12:26:38.573824  7730 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:38.571311 11138 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.571553  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.621910 11173 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:38.579187  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.619560 11167 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.582933  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:38.630788  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47948.8us
I20260709 12:26:38.582651 10836 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.631088  7729 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:38.633598  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 652) took 1126 ms. Trace:
I20260709 12:26:38.634197  7729 rpcz_store.cc:276] 0709 12:26:37.507811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.018318 (+510507us) service_pool.cc:224] Handling call
0709 12:26:38.018691 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.631759 (+613068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.633260 (+  1501us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54001450d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:38.633513 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407717120}
I20260709 12:26:38.602778  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.637010  7807 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:38.637553 10801 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:38.613538  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1955) took 1283 ms. Trace:
I20260709 12:26:38.639413  7657 rpcz_store.cc:276] 0709 12:26:37.330362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.331517 (+  1155us) service_pool.cc:224] Handling call
0709 12:26:37.331829 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.332047 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:38.422133 (+1090086us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b18002a26e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:38.422295 (+   162us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.612844 (+190549us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:38.612938 (+    94us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:38.613013 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.613234 (+   221us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.613435 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59278,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":13256,"raft.queue_time_us":23697,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":16293,"spinlock_wait_cycles":10473344,"thread_start_us":230321,"threads_started":2}
I20260709 12:26:38.638424 10801 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:38.642560 11201 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.494531 11187 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.628952  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1958) took 1283 ms. Trace:
I20260709 12:26:38.634831  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2411) took 1383 ms. Trace:
I20260709 12:26:38.644835  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:38.593899  7658 rpcz_store.cc:276] 0709 12:26:36.903733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.904985 (+  1252us) service_pool.cc:224] Handling call
0709 12:26:38.529884 (+1624899us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b540021b6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.530014 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222275968}
I20260709 12:26:38.626366  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:38.651875  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.647696  7655 rpcz_store.cc:276] 0709 12:26:37.345902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.564938 (+219036us) service_pool.cc:224] Handling call
0709 12:26:37.565295 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.605430 (+ 40135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.605463 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.606015 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.684053 (+ 78038us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:37.684110 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.684202 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.497998 (+813796us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:38.628480 (+130482us) raft_consensus.cc:1685] finished
0709 12:26:38.628516 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.628855 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41025,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":4420352,"thread_start_us":77585,"threads_started":1,"wal-append.queue_time_us":139451}
I20260709 12:26:38.656814  7731 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:38.647957  7737 rpcz_store.cc:276] 0709 12:26:37.251346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.251504 (+   158us) service_pool.cc:224] Handling call
0709 12:26:38.634584 (+1383080us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54001c57d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.634761 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2937663872}
I20260709 12:26:38.659196  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.646948 11167 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:38.660815  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 658) took 1056 ms. Trace:
I20260709 12:26:38.661424  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2135) took 3585 ms. Trace:
I20260709 12:26:38.662073 10528 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:38.661347  7736 rpcz_store.cc:276] 0709 12:26:37.599925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.696401 (+ 96476us) service_pool.cc:224] Handling call
0709 12:26:38.655953 (+959552us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b540037cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.656113 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107680512}
I20260709 12:26:38.664372  7653 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:38.618498 11051 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.668020  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 649) took 1291 ms. Trace:
I20260709 12:26:38.662137  7732 rpcz_store.cc:276] 0709 12:26:35.075667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.075832 (+   165us) service_pool.cc:224] Handling call
0709 12:26:35.102279 (+ 26447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.504253 (+401974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.504286 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.523123 (+ 18837us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.090908 (+567785us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:36.090946 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.091019 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.473189 (+2382170us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:38.647241 (+174052us) raft_consensus.cc:1685] finished
0709 12:26:38.647269 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.661328 (+ 14059us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62593,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":181,"spinlock_wait_cycles":99648256,"thread_start_us":815059,"threads_started":2,"wal-append.queue_time_us":622225}
W20260709 12:26:38.672840  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 131748us
I20260709 12:26:38.675911  7804 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:38.673717  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:38.579922 10987 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.668495  7728 rpcz_store.cc:276] 0709 12:26:37.376458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.729756 (+353298us) service_pool.cc:224] Handling call
0709 12:26:37.751867 (+ 22111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.820072 (+ 68205us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:37.820416 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.876896 (+ 56480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:37.876955 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:37.877005 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.639557 (+762552us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b1800426940. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:38.667769 (+ 28212us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b540037df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:38.667832 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.667959 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":496,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":3172,"raft.queue_time_us":3119,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":206746880,"thread_start_us":56035,"threads_started":1,"wal-append.queue_time_us":62667}
I20260709 12:26:38.626972 11138 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:38.665985  8405 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:38.627106 11002 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:38.596489 11168 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:38.661896 10781 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:38.706770  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:38.685175  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:38.687729  8413 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEARNER]: Advancing to term 8
I20260709 12:26:38.710886  8409 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.596287  8406 rpcz_store.cc:276] 0709 12:26:35.819450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.885673 (+ 66223us) service_pool.cc:224] Handling call
0709 12:26:35.936651 (+ 50978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.800326 (+863675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.487793 (+1687467us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54006310d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:38.488119 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1669151488}
I20260709 12:26:38.666072 10801 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:38.682633 11045 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.728935  7733 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:38.683351 11242 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.671779 11173 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:26:38.742198  8413 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:26:38.742298  8409 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:38.680081 11190 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:38.744542  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.716027 11002 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:38.731387 10978 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:26:38.749577  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:38.750845  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2169) took 1232 ms. Trace:
I20260709 12:26:38.747139  8413 pending_rounds.cc:85] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 6
I20260709 12:26:38.757406  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.766673  8413 pending_rounds.cc:107] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:26:38.767138  8413 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.732146 10781 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:38.800534  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:38.710093 11166 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 12:26:38.809808  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 529899us
I20260709 12:26:38.839708  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:38.842007 11167 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:38.844250 11167 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:38.845144 11167 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.845718 11167 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.810503 11057 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:38.851639 11057 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.852262 11057 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.811978  7728 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:38.812611  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 127540us
I20260709 12:26:38.813573 11051 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:38.855419  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102517us
I20260709 12:26:38.756547  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:38.818923  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 145286us
I20260709 12:26:38.822961  7730 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:38.856472 10781 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:38.859843  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 694444us
I20260709 12:26:38.860505 11051 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:38.863834 11072 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:38.864073  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:38.834522 11173 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.725857 10528 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:38.867841 11246 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:38.747745 10942 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:38.764261  7735 rpcz_store.cc:276] 0709 12:26:37.518227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.058090 (+539863us) service_pool.cc:224] Handling call
0709 12:26:38.069498 (+ 11408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.069729 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:38.088090 (+ 18361us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.160021 (+ 71931us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.551275 (+391254us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b180035cce0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:38.750313 (+199038us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b180035cce0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:26:38.750482 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:38.750538 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:38.750580 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.750652 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.750786 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":584687,"prepare.queue_time_us":17786,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":124,"raft.queue_time_us":594744,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":2895,"spinlock_wait_cycles":9244416,"thread_start_us":71328,"threads_started":1,"wal-append.queue_time_us":148231}
I20260709 12:26:38.871294  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2159) took 1492 ms. Trace:
I20260709 12:26:38.877148  7731 rpcz_store.cc:276] 0709 12:26:37.379451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.748361 (+368910us) service_pool.cc:224] Handling call
0709 12:26:38.656429 (+908068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.657364 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.871041 (+213677us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540037cb50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:38.871203 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087443200}
I20260709 12:26:38.881410  7660 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:38.883822  7730 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:38.756915 11002 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.868707 11173 raft_consensus.cc:1064] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:26:38.874915 10942 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:38.944749  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88380.4us
I20260709 12:26:38.869668 11058 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:38.965204  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 144491us
I20260709 12:26:38.910351 10925 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.748790 11242 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:38.977303  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:38.981346  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.989391  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 148471us
I20260709 12:26:38.757350 11190 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:38.992534  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 230716us
W20260709 12:26:39.002096  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 144807us
I20260709 12:26:39.014587  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 659) took 1414 ms. Trace:
I20260709 12:26:39.008556  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.015091  7733 rpcz_store.cc:276] 0709 12:26:37.600255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.117617 (+517362us) service_pool.cc:224] Handling call
0709 12:26:38.117953 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.735409 (+617456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.014314 (+278905us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:39.014515 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404619136}
W20260709 12:26:38.965945 11173 raft_consensus.cc:1068] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Unable to promote non-voter 992d6df95ef74f7f8b12534534a45501: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:38.963455 10942 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:39.037261  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:39.002092 10796 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.003s
W20260709 12:26:39.046325  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 291302us
W20260709 12:26:39.044374  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.993252 11246 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:39.067633 11241 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 12:26:39.068403 11245 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 12:26:39.043870 11176 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:39.043296 11166 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:39.032217 10925 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.057252 11248 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:39.035364 11058 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:26:39.056272 10528 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:39.056628 11253 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:39.105899 11166 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:39.127386  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 615) took 1389 ms. Trace:
I20260709 12:26:39.053172 11164 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:39.118402 11244 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.402s	user 0.000s	sys 0.002s
I20260709 12:26:39.050513 10796 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.147754  7659 rpcz_store.cc:276] 0709 12:26:37.733289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.619959 (+886670us) service_pool.cc:224] Handling call
0709 12:26:38.620390 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.655829 (+ 35439us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:38.708398 (+ 52569us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.122824 (+414426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.122882 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:39.122933 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.123007 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.123162 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50717,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4211,"thread_start_us":393820,"threads_started":1}
W20260709 12:26:39.152791  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 139882us
W20260709 12:26:39.151010  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 193467us
I20260709 12:26:39.058727 11246 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.162652 11166 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.123710 11176 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:39.165958 11176 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.124389 10987 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:39.124881 11248 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:39.125262 11266 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:39.193847 10978 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:39.160751 10528 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:39.199162 10528 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.201855 10925 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:39.118702 11257 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 12:26:39.199856 11256 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:26:39.166082 11166 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:39.214536  7810 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.003s
I20260709 12:26:39.215938  7810 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:39.194629 10978 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.255272 11168 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:39.260906 10978 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:39.266126 10836 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:26:39.260911 11242 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:39.212517 10987 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:39.274489 10987 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:39.275095 11187 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:39.285920  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 622) took 1550 ms. Trace:
I20260709 12:26:39.282173  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 614) took 1549 ms. Trace:
I20260709 12:26:39.327028  7656 rpcz_store.cc:276] 0709 12:26:37.733038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.582271 (+849233us) service_pool.cc:224] Handling call
0709 12:26:38.592306 (+ 10035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.617529 (+ 25223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.617562 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.620892 (+  3330us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.794034 (+173142us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:38.815193 (+ 21159us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.815490 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.815601 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.281933 (+466332us) raft_consensus.cc:1685] finished
0709 12:26:39.281954 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.282099 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49199,"prepare.queue_time_us":20377,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":67046,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":10054,"thread_start_us":86924,"threads_started":1,"wal-append.queue_time_us":513423}
I20260709 12:26:39.266638 10836 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.212774 11072 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.149636 11253 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.284915 11246 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:39.285516 11042 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.212630 11266 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:39.291497 11187 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:39.276224 11251 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:26:39.343206 10925 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.280047 11250 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:26:39.275543 10796 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:39.285785 10801 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:26:39.437479  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:39.438690  8408 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:39.278754 11164 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:38.739032  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:39.291951  7655 rpcz_store.cc:276] 0709 12:26:37.735324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.666727 (+931403us) service_pool.cc:224] Handling call
0709 12:26:38.667127 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:38.675509 (+  8382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.675540 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.681731 (+  6191us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.848796 (+167065us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:38.851368 (+  2572us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.856299 (+  4931us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.870307 (+ 14008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.870395 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.285672 (+415277us) raft_consensus.cc:1685] finished
0709 12:26:39.285705 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.285859 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7939,"prepare.queue_time_us":4603,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":79955,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":29354,"thread_start_us":105542,"threads_started":1,"wal-append.queue_time_us":220265}
I20260709 12:26:39.285988 11249 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:39.283838 11190 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:39.394652  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.558s (SENT)
W20260709 12:26:39.445182  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 402868us
W20260709 12:26:39.446453  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 426356us
W20260709 12:26:39.459468 11238 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.221s	user 0.007s	sys 0.000s
W20260709 12:26:39.537585  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 365743us
I20260709 12:26:39.352700 11072 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:39.533931  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.017s	sys 0.054s
W20260709 12:26:39.542761  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.017s	sys 0.054s
W20260709 12:26:39.543273  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 566061us
I20260709 12:26:39.544829  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:39.546614  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:39.512120 11057 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.000s
I20260709 12:26:39.569314 11057 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:39.537932 11002 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.001s
I20260709 12:26:39.345430 11248 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:39.537932 11167 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.692s	user 0.014s	sys 0.004s
I20260709 12:26:39.458182  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:39.509900 11253 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:39.436429 11239 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:39.521749  8411 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEARNER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:39.573995  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45648.7us
I20260709 12:26:39.370031 11042 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.447686  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:39.588399  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 140870us
W20260709 12:26:39.589578  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.021s	sys 0.039s
W20260709 12:26:39.590204  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.022s	sys 0.039s
I20260709 12:26:39.590045  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:39.523656  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:39.556586 11263 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:39.568023 11275 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:39.275650 10987 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.614910  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:39.616847  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 613) took 1884 ms. Trace:
I20260709 12:26:39.552531  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:39.620599  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 961496us
I20260709 12:26:39.445816 10925 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.578116  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:39.618085  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.575238  7700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.011s
W20260709 12:26:39.621958  7700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.011s
I20260709 12:26:39.621551  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:39.623049  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:39.617452  7654 rpcz_store.cc:276] 0709 12:26:37.732735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.538200 (+805465us) service_pool.cc:224] Handling call
0709 12:26:38.569948 (+ 31748us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:38.570501 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.570545 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.577306 (+  6761us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.748914 (+171608us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:38.749070 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.749146 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.616513 (+867367us) raft_consensus.cc:1685] finished
0709 12:26:39.616544 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.616759 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1136,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":2308,"thread_start_us":134133,"threads_started":1,"wal-append.queue_time_us":465795}
W20260709 12:26:39.624094  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64749.7us
I20260709 12:26:39.622503  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:39.629990  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 235402us
W20260709 12:26:39.634689  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49619.2us
W20260709 12:26:39.635165  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.693s (SENT)
I20260709 12:26:39.635787  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:39.636184  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:39.633471  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66972.5us
I20260709 12:26:39.637127  7736 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:39.638870  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:39.640601  7729 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:39.576167 11279 raft_consensus.cc:2764] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:39.571583 11072 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:39.592417 11201 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:39.594697 11253 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:39.642607 11058 raft_consensus.cc:2764] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:26:39.633078 11164 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:39.529796 11187 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.613761 11248 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:39.615607 11263 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:39.661043  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:39.686661  7732 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:39.662588  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.652522  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.615548 11246 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:39.657833  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1959) took 2310 ms. Trace:
I20260709 12:26:39.696971  7728 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:39.697578  7728 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:39.697736  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:39.698392  7734 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:39.678231  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:39.699716  7735 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:39.697237  7661 rpcz_store.cc:276] 0709 12:26:37.347007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.945204 (+598197us) service_pool.cc:224] Handling call
0709 12:26:37.966795 (+ 21591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.967199 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.967252 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.980001 (+ 12749us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.137910 (+157909us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:38.137954 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.138032 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.324628 (+1186596us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:39.657357 (+332729us) raft_consensus.cc:1685] finished
0709 12:26:39.657391 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.657740 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":561,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4670,"spinlock_wait_cycles":5463808,"thread_start_us":127102,"threads_started":1,"wal-append.queue_time_us":201973}
I20260709 12:26:39.662317 11190 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.586906 11167 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.653564  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:39.571676 11002 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:39.702533  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.743955  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91481.8us
I20260709 12:26:39.646939 11057 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:39.744828  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.651502 11051 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:39.649933  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:39.715703  7737 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:39.661948 10781 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:39.756582 10781 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.676447 11173 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:39.759672 11173 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.759186 10781 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:39.764670 11173 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:39.680667 11164 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.629395 11265 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:26:39.699982 11269 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:39.708137  7728 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.679555 11242 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:39.790184  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45457.9us
W20260709 12:26:39.795337  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 155775us
W20260709 12:26:39.749734 11138 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.026s	user 0.012s	sys 0.003s
W20260709 12:26:39.751001 11278 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:26:39.592902 11266 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.812729  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:39.650996 11285 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.125s
I20260709 12:26:39.796663  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:39.697861 11263 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:39.791024  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 141228us
I20260709 12:26:39.829981  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:39.703670 11284 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.748194 10836 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:39.813472  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.782s (SENT)
I20260709 12:26:39.808243  7728 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:39.798836  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 176114us
I20260709 12:26:39.743716 11253 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:39.768513 11139 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:39.830911  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.708083 11246 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.789496 11187 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:39.791896 11164 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:39.776909 11281 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:39.860886  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 234662us
I20260709 12:26:39.860656  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.794694 11269 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:39.797544 10781 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.860806  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47384.3us
I20260709 12:26:39.808353 11072 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:39.878468  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47671.2us
I20260709 12:26:39.828528  7734 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:39.921566  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:39.921164  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91292.6us
I20260709 12:26:39.964336  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:39.930966  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.967921  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46658.7us
I20260709 12:26:39.969504  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:39.953418  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:39.866034 11051 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:39.804349 11058 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:39.834602 11289 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:39.823971 11138 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:39.823562 11242 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:39.831931 11190 raft_consensus.cc:476] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Not starting pre-election -- already a leader
I20260709 12:26:40.018554  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:39.796802 11280 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:39.835737 11271 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:39.924582 11057 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:39.922786 11279 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.030589  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:39.914862 11269 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:39.923326 11265 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:40.019938  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51224.3us
I20260709 12:26:39.870625 11253 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:40.081192  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 111821us
I20260709 12:26:40.083063  8414 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:40.001664 11058 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:39.939914 11273 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 12:26:40.024200 11280 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.100064  7728 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:26:40.097548  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:40.107483  7728 pending_rounds.cc:85] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 4
I20260709 12:26:39.942625 10781 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:40.107928  7728 pending_rounds.cc:107] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 12:26:40.110150  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.862951 11284 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:40.096251 11190 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:40.124195  7728 raft_consensus.cc:2889] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:40.067407 11289 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:40.068783 11302 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.999519 11072 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:40.104175 10978 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:40.100241 11058 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:40.075114 11139 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:40.108297 11280 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:40.126471 11042 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:40.182193  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:40.125716 11265 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:40.190155  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 92716.7us
I20260709 12:26:40.190507  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:40.137432 11190 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:40.197101  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:40.190985  7732 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 3.
I20260709 12:26:40.197862  8413 raft_consensus.cc:606] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:40.200011  8413 raft_consensus.cc:614] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:40.192991  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:40.170902 11313 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.170979 11167 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:40.197146 11282 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.401s	user 0.002s	sys 0.000s
I20260709 12:26:40.178568 11002 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:40.240280 11288 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
W20260709 12:26:40.240972 11296 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 12:26:40.242283 11284 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:40.197835 11190 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:40.246490 11166 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.001s	user 0.012s	sys 0.000s
I20260709 12:26:40.246307 11265 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:40.157603 11289 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:40.279466 11289 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:40.225255 11173 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:26:40.248765 11309 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:40.222456 11057 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:40.088913 11279 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:40.282081 11057 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:40.255460  7735 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.005s
I20260709 12:26:40.264865  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:40.292279  8409 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:40.251075 11295 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.252s	user 0.000s	sys 0.002s
W20260709 12:26:40.280989 11306 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.122s	user 0.001s	sys 0.002s
I20260709 12:26:40.293287  7735 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:40.159514 11302 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:40.302822 11315 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:40.303617 11173 consensus_queue.cc:1243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:40.304522 11138 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:40.265884 11314 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:40.320695  8409 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:40.288169 11279 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:40.341099  8409 raft_consensus.cc:606] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:40.304589 11309 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:40.312772 11317 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.331369 11315 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:40.332405  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.317910 11246 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:40.354662  8407 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.006s	sys 0.022s
W20260709 12:26:40.310070  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.791s (SENT)
W20260709 12:26:40.371588  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.020s	sys 0.069s
I20260709 12:26:40.325472 11242 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:40.258653 11313 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:40.338110 11253 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:40.343849 11314 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:40.393797  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:40.308113 11139 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:40.714483  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.008s	sys 0.031s
I20260709 12:26:40.524168  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:41.078239  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.018s	sys 0.057s
I20260709 12:26:40.398161  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:40.513679  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.666898  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.018s	sys 0.015s
W20260709 12:26:41.080509  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.026s	sys 0.023s
I20260709 12:26:40.802945  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:40.351320 11173 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:40.499749  7809 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:40.851773  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:40.342880  8405 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:26:41.081050  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 994367us
W20260709 12:26:40.403262 10942 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.353s	user 0.017s	sys 0.000s
W20260709 12:26:40.730695 10978 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.011s	sys 0.023s
W20260709 12:26:40.666678  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.031s
W20260709 12:26:41.104622  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.006s	sys 0.042s
W20260709 12:26:41.105036  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 999553us
W20260709 12:26:40.832955 11306 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:40.885869  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:40.833895  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:40.799432 11269 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.002s
W20260709 12:26:40.938259  7558 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 267) took 16346 ms (client timeout 15000 ms). Trace:
I20260709 12:26:40.679432  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.083084  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:41.082444  8407 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.429s	user 0.025s	sys 0.047s
I20260709 12:26:41.114384  7654 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:41.081229  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.036s	sys 0.113s
I20260709 12:26:40.404811 11168 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.814335  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.069574 11271 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:40.605826 10781 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:41.084124  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.091581  7660 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:41.084532 11314 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:40.678431 11229 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.000s	sys 0.032s
I20260709 12:26:41.098544 11242 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:40.408494 11317 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:40.407191 11309 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:40.721174 10836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.003s	sys 0.037s
I20260709 12:26:41.106618  7659 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:40.525323  7737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.010s	sys 0.052s
I20260709 12:26:41.107069  7736 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:40.784981 11322 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:40.765203 11275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.016s	sys 0.048s
I20260709 12:26:40.782349  7555 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 4 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:40.554769 11257 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:41.086227  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:40.352144 11324 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:40.637976 11072 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.118746  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.280853  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:40.394495  8413 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:26:40.719981 11268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.026s	sys 0.006s
I20260709 12:26:40.938859 11320 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.956723 11264 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.292409 11242 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.294394 11337 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.160956 11316 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.124018 11271 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.187054  7660 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:41.295186 11337 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:40.828172  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 741) took 1956 ms. Trace:
I20260709 12:26:41.090085 11173 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:41.135078  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.295352 11316 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.297091 11242 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.298081 11173 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:26:41.298825 11242 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.299716 11173 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.300042 11316 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.300836 11317 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.164906 11341 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.293483 11320 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.301643 11341 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.303427  7661 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 4.
I20260709 12:26:41.126117  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.077510 11315 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:41.012648  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.258853  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:41.275642 11248 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.579s	user 0.013s	sys 0.003s
W20260709 12:26:41.194593 10987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.006s	sys 0.026s
W20260709 12:26:41.283915 11268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.026s	sys 0.018s
W20260709 12:26:41.311308 10987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.013s	sys 0.027s
W20260709 12:26:41.115589  7558 rpcz_store.cc:269] 0709 12:26:24.592140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.592453 (+   313us) service_pool.cc:224] Handling call
0709 12:26:40.938093 (+16345640us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:35.936405 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:35.943463 (+  7058us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:35.943534 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:26:35.943560 (+    26us) tablet.cc:662] Decoding operations
0709 12:26:35.944992 (+  1432us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:35.945078 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:35.945114 (+    36us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:26:35.946353 (+  1239us) tablet.cc:701] Row locks acquired
0709 12:26:35.946416 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:26:35.946614 (+   198us) write_op.cc:270] Start()
0709 12:26:35.946887 (+   273us) write_op.cc:276] Timestamp: P: 1783599995946526 usec, L: 0
0709 12:26:35.946911 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:36.179387 (+232476us) log.cc:844] Serialized 5581 byte log entry
0709 12:26:37.275791 (+1096404us) op_driver.cc:464] REPLICATION: finished
0709 12:26:37.735288 (+459497us) write_op.cc:301] APPLY: starting
0709 12:26:37.735439 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:37.735756 (+   317us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:37.735905 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:37.738035 (+  2130us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:37.738451 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:26:37.738508 (+    57us) write_op.cc:312] APPLY: finished
0709 12:26:37.756382 (+ 17874us) log.cc:844] Serialized 101 byte log entry
0709 12:26:37.775382 (+ 19000us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:37.775567 (+   185us) write_op.cc:454] Released schema lock
0709 12:26:37.775646 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:38.403564 (+627918us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1de28 00007fc005958581 00007fc005957d58 00007fc00595c9f1 00007fc00595c9a9 00007fc00595c939 00007fc00595c901 00007fc00595bb2d 0000000000388634 0000000000388479
0709 12:26:38.403903 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:38.403952 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":521905,"reported_tablets":35,"spinlock_wait_cycles":483072,"child_traces":[["op",{"apply.queue_time_us":441564,"apply.run_cpu_time_us":5153,"apply.run_wall_time_us":51560,"num_ops":14,"prepare.queue_time_us":571886,"prepare.run_cpu_time_us":30627,"prepare.run_wall_time_us":2498870,"raft.queue_time_us":60685,"raft.run_cpu_time_us":64711,"raft.run_wall_time_us":362429,"replication_time_us":1311926,"spinlock_wait_cycles":52728064,"thread_start_us":695124,"threads_started":4,"wal-append.queue_time_us":595898,"wal-append.run_cpu_time_us":20555,"wal-append.run_wall_time_us":2012711}]]}
I20260709 12:26:41.135190 11168 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.103981 11246 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:41.257804  7737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.303s	user 0.027s	sys 0.082s
W20260709 12:26:41.198738  7702 connection.cc:256] RPC call timeout handler was delayed by -2.322159468999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:26:41.316499  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1889594689996557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 12:26:41.316869  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1286594690000129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
W20260709 12:26:41.317214  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1115594689999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:26:41.317507  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0404594689998703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 12:26:41.317826  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9585594689997379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:26:41.318104  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7569594689998667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
W20260709 12:26:41.318413  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7380594689998361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
W20260709 12:26:41.318717  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7236594689998128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 12:26:41.318993  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6993594689997735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 12:26:41.319603  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6597594689997095s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22740000000000002s for a response
W20260709 12:26:41.319921  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5562594689999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2389s for a response
I20260709 12:26:41.258139  7804 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:41.067379 11302 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.123317 11269 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.293911 11264 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:41.329751 11263 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.463s	user 0.014s	sys 0.000s
I20260709 12:26:41.264839  7654 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:41.234993 11321 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.333426 11263 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.093585  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:26:41.322947  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.334898  7803 raft_consensus.cc:606] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:41.335595  7803 raft_consensus.cc:614] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:41.291781 10836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.010s	sys 0.069s
I20260709 12:26:41.115747 11253 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:41.279474 11164 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.356s	user 0.007s	sys 0.010s
W20260709 12:26:41.116353 11058 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.932s	user 0.011s	sys 0.003s
I20260709 12:26:41.322168  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.369170  7735 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:41.323282  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.370236  7728 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
W20260709 12:26:41.339411 11187 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.398s	user 0.017s	sys 0.001s
W20260709 12:26:41.281787 11275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.112s	user 0.031s	sys 0.070s
I20260709 12:26:41.327842 11340 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:41.328269 11337 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.305291 11072 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.329733 11260 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:41.321056  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:41.273975  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.397099  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.397205 11072 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.399623  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.400138 11309 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.124320 11334 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.401419 11334 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.402487  7657 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:41.344754  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.334087 11321 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.403577  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.404210  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.349223  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.380885 11316 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.315829 11248 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.405190 11321 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.377643 11058 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.350203  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.405539 11248 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.352066  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.407091  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.407887  7803 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:41.408146 11248 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.408311  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:41.408667  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.350069  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.360651  7555 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.409600  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:41.410066 11190 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.113s	user 0.012s	sys 0.000s
I20260709 12:26:41.409649  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:41.373126 11164 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.263154 11324 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.412164  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.412313 11324 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.413120  7804 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:26:41.413662 11334 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.413808 11164 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.414363  7804 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:41.350855 11263 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.324628  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.415647  7803 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:41.405218  7734 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:41.416239  7730 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:41.281044 11257 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:41.411921  7555 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:41.279765 11229 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.472s	user 0.009s	sys 0.055s
I20260709 12:26:41.115633 10942 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.372964  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:41.410612 11190 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.389775 10979 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:41.397936  7808 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:41.399794  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:41.402618 11280 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.189s	user 0.010s	sys 0.004s
I20260709 12:26:41.404497  7660 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:41.343266 11246 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.405663 11341 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.448648  7555 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:41.303732 11320 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.457908  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.458345  7657 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:26:41.114135 10978 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.011s	sys 0.038s
I20260709 12:26:41.282164 11322 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:41.416486  7659 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:41.347021 11051 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.344s	user 0.015s	sys 0.000s
I20260709 12:26:41.410998  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.424175  7728 raft_consensus.cc:2376] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
W20260709 12:26:41.388805 11287 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.438s	user 0.005s	sys 0.005s
I20260709 12:26:41.425642  7729 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:41.371984 11253 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.457448 11309 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.405208 11316 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.454420 11337 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.463197 10978 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:41.463747 11341 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.462968 11168 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:41.472640 11330 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:26:41.491488 11320 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.492259 11320 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.380259 11340 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.495251 11340 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.297288  8406 rpcz_store.cc:276] 0709 12:26:38.871574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.875264 (+  3690us) service_pool.cc:224] Handling call
0709 12:26:38.963754 (+ 88490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.147204 (+183450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.241422 (+ 94218us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.372928 (+131506us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.707297 (+334369us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:39.745624 (+ 38327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.775207 (+ 29583us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.445987 (+670780us) raft_consensus.cc:1685] finished
0709 12:26:40.497078 (+ 51091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.828023 (+330945us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147018,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":48251,"thread_start_us":254779,"threads_started":2,"wal-append.queue_time_us":425347}
W20260709 12:26:41.426427  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.11633e+06us
I20260709 12:26:41.496644  7736 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:41.498385  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.446372  8409 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79849a645ae64c3787630b03e7da6307"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:41.499647  8409 raft_consensus.cc:606] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:41.500178  8409 raft_consensus.cc:614] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.501569 11271 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.503786  7555 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.505496  7555 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.507869  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 667) took 1602 ms. Trace:
I20260709 12:26:41.508347  7732 rpcz_store.cc:276] 0709 12:26:39.905473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.905655 (+   182us) service_pool.cc:224] Handling call
0709 12:26:41.507675 (+1602020us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540039b2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:41.507802 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496930048}
I20260709 12:26:41.474036  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:41.511981  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.512809  7661 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:41.513895  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.455703  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 666) took 1550 ms. Trace:
I20260709 12:26:41.334342 11269 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.515568  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 632) took 1497 ms. Trace:
I20260709 12:26:41.516597 11264 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.515424  7737 rpcz_store.cc:276] 0709 12:26:39.905101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.923016 (+ 17915us) service_pool.cc:224] Handling call
0709 12:26:39.953894 (+ 30878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.954181 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:41.343537 (+1389356us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.455340 (+111803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.455396 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:41.455452 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.455522 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.455647 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27004,"prepare.queue_time_us":372507,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":27190,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7262,"spinlock_wait_cycles":384384,"thread_start_us":1446050,"threads_started":2}
W20260709 12:26:41.518419 11304 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.345s	user 0.007s	sys 0.007s
I20260709 12:26:41.518539 11246 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.458595  8412 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:41.474385 11058 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.458645  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:41.520306  7627 connection.cc:256] RPC call timeout handler was delayed by -1.161443662000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 12:26:41.415490 11072 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.471518  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:41.525332  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.995s (SENT)
I20260709 12:26:41.525820  7728 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:26:41.526314  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0042915089998132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2952s for a response
I20260709 12:26:41.509680  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.495440 11164 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.527040  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:41.520584  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.450949 11280 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.422094  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:41.467168 11313 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.521620  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 743) took 2337 ms. Trace:
I20260709 12:26:41.523017 11320 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.523360  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.516217  7656 rpcz_store.cc:276] 0709 12:26:40.017974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.019514 (+  1540us) service_pool.cc:224] Handling call
0709 12:26:40.065055 (+ 45541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.096486 (+ 31431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.096539 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.106611 (+ 10072us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.347898 (+241287us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:40.347944 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.348021 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.515321 (+1167300us) raft_consensus.cc:1685] finished
0709 12:26:41.515359 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.515503 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30832,"raft.queue_time_us":142634,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":14152,"thread_start_us":177854,"threads_started":1,"wal-append.queue_time_us":1259511}
I20260709 12:26:41.461388  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.448691 10979 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:41.499078  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:41.429286  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:41.526890  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:41.516141  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.515159  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.479346  7808 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:26:41.489445  7778 connection.cc:256] RPC call timeout handler was delayed by -1.1308854619996964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 12:26:41.487498  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 737) took 2947 ms (client timeout 2012 ms). Trace:
I20260709 12:26:41.527995  7734 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:41.531266  7655 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:41.484133  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.546823  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:41.534049  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72752.5us
I20260709 12:26:41.528218  7555 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 2 to 4, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 4 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:41.535987  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2535) took 1145 ms. Trace:
W20260709 12:26:41.539284  8414 rpcz_store.cc:269] 0709 12:26:38.539911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.540310 (+   399us) service_pool.cc:224] Handling call
0709 12:26:41.487134 (+2946824us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54004691d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:41.487391 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3361079680}
I20260709 12:26:41.482010 10942 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:41.555756 11339 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:26:41.557837 11190 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.549759  7811 rpcz_store.cc:276] 0709 12:26:40.390681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.391684 (+  1003us) service_pool.cc:224] Handling call
0709 12:26:41.515997 (+1124313us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfdeee 000000000036eb39 00007fc006e711e0 00007fc006f3a0c4 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:41.535876 (+ 19879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5115648}
I20260709 12:26:41.544803  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.535101  7737 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:41.528616  7732 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:41.537396  7661 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.529137  7735 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:41.529289  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.542636  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:41.588136  7808 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:41.586285  7804 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:41.535544  7730 raft_consensus.cc:2376] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6. Re-sending same reply.
I20260709 12:26:41.535022  7728 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:41.563199  7729 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:26:41.537571  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.590919  7736 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:41.530525  8412 rpcz_store.cc:276] 0709 12:26:39.184883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.194208 (+  9325us) service_pool.cc:224] Handling call
0709 12:26:39.194657 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.519453 (+2324796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.521366 (+  1913us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b540046d7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:41.521561 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5205995904}
W20260709 12:26:41.486001 11270 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.979s	user 0.006s	sys 0.003s
I20260709 12:26:41.592139  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2185) took 1968 ms. Trace:
W20260709 12:26:41.596176  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46966.4us
I20260709 12:26:41.593107  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:41.595861  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.599471  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:41.596575  7731 rpcz_store.cc:276] 0709 12:26:39.623667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.623896 (+   229us) service_pool.cc:224] Handling call
0709 12:26:39.647001 (+ 23105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.651730 (+  4729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.651775 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.652273 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.902874 (+250601us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:39.903121 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.903754 (+   633us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.517763 (+1614009us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:41.546103 (+ 28340us) raft_consensus.cc:1685] finished
0709 12:26:41.546130 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.592072 (+ 45942us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47629,"raft.queue_time_us":30842,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":15631,"spinlock_wait_cycles":25063680,"thread_start_us":204929,"threads_started":1,"wal-append.queue_time_us":406576}
I20260709 12:26:41.605098  8414 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:41.605938  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:41.608183  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.605024  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.610248  7656 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:41.610780  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.544278  8405 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:41.558712 11272 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:41.476581 11253 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.679879  7728 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:41.626776  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.550719 11072 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:41.532369 11051 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:41.688410  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.642944  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 619) took 3908 ms. Trace:
I20260709 12:26:41.648370  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:41.591298 11264 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.480911 11316 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.485646 11260 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:41.681152  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70467.3us
I20260709 12:26:41.681766  7729 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:41.684894  7777 connection.cc:256] RPC call timeout handler was delayed by -1.0619898280001507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.334s for a response
I20260709 12:26:41.591066 11190 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.699477  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.701335  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.686839 11281 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:41.687711  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:41.684315  8414 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:41.715999  8414 raft_consensus.cc:606] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:41.716502  8414 raft_consensus.cc:614] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.688395  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:41.667694  7627 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.699s (SENT)
I20260709 12:26:41.717386  7656 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:41.487702 11341 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.732533 11321 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:41.733353 11187 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:41.734472 11341 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.690217  7658 rpcz_store.cc:276] 0709 12:26:37.734398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.649509 (+915111us) service_pool.cc:224] Handling call
0709 12:26:38.678345 (+ 28836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.880435 (+202090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.880693 (+   258us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.887901 (+  7208us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.099121 (+211220us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:39.099172 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.099288 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.456999 (+2357711us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b2400055808. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee7409d 00007fc001aa610e 00007fc001abd88c 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221
0709 12:26:41.642443 (+185444us) raft_consensus.cc:1685] finished
0709 12:26:41.642467 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.642693 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4573,"raft.queue_time_us":155154,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":275,"spinlock_wait_cycles":38883072,"thread_start_us":305522,"threads_started":2,"wal-append.queue_time_us":431079}
I20260709 12:26:41.735708  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2533) took 1346 ms. Trace:
I20260709 12:26:41.604636 11345 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.736744  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.737193 11345 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.738201  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.641009 11058 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:41.640479 11249 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:41.613128 11269 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.680634 11318 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.739573 11058 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.482149 11322 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:41.740016 11349 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:41.740582  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.695869  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2539) took 1264 ms. Trace:
I20260709 12:26:41.741230  7807 rpcz_store.cc:276] 0709 12:26:40.431898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.432075 (+   177us) service_pool.cc:224] Handling call
0709 12:26:41.688243 (+1256168us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:41.695791 (+  7548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664470272}
I20260709 12:26:41.742565  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.743468  7807 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:41.744328  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.745839  7702 leader_election.cc:400] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9.
I20260709 12:26:41.745584 11269 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:41.746201  7702 leader_election.cc:403] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:41.734522 11187 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:41.747371 11187 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:41.747977  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.714107  7808 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:26:41.717346  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:41.749325 11341 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:41.749452  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81869.8us
I20260709 12:26:41.479871 11337 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:41.750969  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.075s (SENT)
I20260709 12:26:41.751466  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:41.751688 11346 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:41.752179 11253 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.752405 11337 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.752960 11337 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:41.752748 11253 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.753263  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 907) took 2172 ms. Trace:
I20260709 12:26:41.605799 11168 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:26:41.736217  7806 rpcz_store.cc:276] 0709 12:26:40.389775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.432251 (+ 42476us) service_pool.cc:224] Handling call
0709 12:26:41.083082 (+650831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.083622 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.083655 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.116350 (+ 32695us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.452265 (+335915us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:41.452304 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.452377 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.735323 (+282946us) raft_consensus.cc:1685] finished
0709 12:26:41.735379 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.735614 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":12672,"spinlock_wait_cycles":9984,"thread_start_us":281438,"threads_started":1,"wal-append.queue_time_us":440047}
I20260709 12:26:41.754878 11253 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.755012 11319 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:41.748304 11187 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.689054 11330 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:41.756899 11187 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.757242 11330 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:41.757432 11072 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9.
I20260709 12:26:41.481407 10978 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.759093  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.760278 11072 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.762079  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.607358 11299 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
I20260709 12:26:41.762960 11072 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:41.764511 11314 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.001s
I20260709 12:26:41.765578  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:41.766707  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.767894 11072 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:41.527635 11164 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.768430 11072 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:41.769091 11072 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.556450 11257 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.772034 11257 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:41.613027 10836 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:41.780170  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:41.780907  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50586.4us
I20260709 12:26:41.782621  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:41.783401 11249 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:41.784801 11249 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.715026 11190 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.730791  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.787544 11269 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:41.731266 11316 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.787880 11321 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:41.788579 11321 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.789052 11321 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.740782 11322 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:41.789069 11190 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:41.756729 11315 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:41.787954 11269 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:41.797515 11269 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:41.798332 11321 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:41.806528  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.679117 11272 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:41.808301 11272 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:41.810835 11321 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.688323 11347 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.816270  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.817080  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:41.816500 11347 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:41.817631  7810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2532) took 1428 ms (client timeout 1840 ms). Trace:
I20260709 12:26:41.788312  7658 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:41.818066  7810 rpcz_store.cc:269] 0709 12:26:40.389398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.389659 (+   261us) service_pool.cc:224] Handling call
0709 12:26:41.816025 (+1426366us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:41.817565 (+  1540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658891008}
I20260709 12:26:41.527189 11324 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.820242  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.821702 11324 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.821981  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:41.755956 11319 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.477021 11309 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:41.731971  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.781205 11330 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:41.730407 11281 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.725692  7660 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:26:41.964834  7660 pending_rounds.cc:85] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 6
I20260709 12:26:41.965291  7660 pending_rounds.cc:107] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:26:41.965634  7660 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:41.822991  7658 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:41.765151 11314 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:41.848358  7656 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:41.886411  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:41.825075  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.831629  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:41.973542  7811 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:41.975828  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:41.956229  7661 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:41.856107  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:41.850195  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.373s (SENT)
I20260709 12:26:41.831462  7653 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:41.822880 11319 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.837706  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:41.949419  7662 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:42.000128  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 113820us
I20260709 12:26:42.000945  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:41.753784  8407 rpcz_store.cc:276] 0709 12:26:39.580830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.631508 (+ 50678us) service_pool.cc:224] Handling call
0709 12:26:39.652666 (+ 21158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.114900 (+1462234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.114945 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.115489 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.421873 (+306384us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:41.421938 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.422074 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.752982 (+330908us) raft_consensus.cc:1685] finished
0709 12:26:41.753013 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.753181 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59270,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":24683,"thread_start_us":1767652,"threads_started":2,"wal-append.queue_time_us":475144}
I20260709 12:26:42.003623  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.003047  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.004787  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.005007  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.005458  7659 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.006153  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.007680  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:41.951090 11255 raft_consensus.cc:670] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } [suppressed 3 similar messages]
W20260709 12:26:41.828830 11305 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.678s	user 0.000s	sys 0.012s
I20260709 12:26:41.768471 11271 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:42.009423  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 171870us
I20260709 12:26:42.011355  7730 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:42.029134  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.011592  7731 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 5: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:42.008461  7728 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:42.014165  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:42.030159  7730 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:42.005722  7737 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:41.755975 11345 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.018236  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.091320  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:41.740069 11058 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.022620  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.023123  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.024717  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:42.094097  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 398250us
I20260709 12:26:42.022554 11299 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:42.013332 11321 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:42.107263 11321 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:42.107931 11321 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.039505 11246 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:42.040304  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.111239  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:42.040930  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:42.041603 11266 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:42.043300 11333 raft_consensus.cc:1064] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:26:41.818370 11269 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.113449 11266 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:42.119771 11266 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:42.121273 11269 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.120908 11266 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.122365  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.122730 11266 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:42.040381  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.129287  7661 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:42.057441 11314 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:41.695395 11331 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.335s
I20260709 12:26:42.047948  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.787016 11072 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.070544 11347 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.071687  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.106979 11299 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.079540  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 549) took 1524 ms. Trace:
I20260709 12:26:42.084894  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2183) took 2462 ms. Trace:
I20260709 12:26:42.085515 11351 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:42.034801  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.058753  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.150918  7736 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:41.824333 11280 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.092501  7654 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.152498  7659 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:42.154458  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.033207  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.094573  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.155328  7659 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.156561  7728 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:41.770977 11164 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:42.096637  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64877us
I20260709 12:26:41.827122 11316 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:42.100795  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 250691us
I20260709 12:26:42.112195  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.113942  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:42.111426  7732 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:42.160847  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:42.160341  7735 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:42.117743  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.163616  7700 connection.cc:256] RPC call timeout handler was delayed by -1.4677036690000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
I20260709 12:26:42.164162  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:41.703729 11260 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.123767  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.015434 11330 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:42.126286 11333 raft_consensus.cc:1068] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Unable to promote non-voter 992d6df95ef74f7f8b12534534a45501: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:42.127518 11187 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:42.128221  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.177251  8408 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 12:26:42.177614  7803 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:42.131937 11139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.012s	sys 0.066s
I20260709 12:26:42.103026  7656 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:42.185310  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.140535  7810 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:41.698189 11264 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.033216  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.115613  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.166639  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.145294  7812 rpcz_store.cc:276] 0709 12:26:40.561263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.601357 (+ 40094us) service_pool.cc:224] Handling call
0709 12:26:42.078428 (+1477071us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:26:42.078737 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.079026 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.079070 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.079131 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:42.079180 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.079293 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.079453 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662500608}
I20260709 12:26:42.188529  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:42.169059  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:42.146720  7733 rpcz_store.cc:276] 0709 12:26:39.622932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.625147 (+  2215us) service_pool.cc:224] Handling call
0709 12:26:39.625488 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.633579 (+  8091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.633610 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.664738 (+ 31128us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.829575 (+164837us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:39.829645 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.829722 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.828013 (+1998291us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:42.084509 (+256496us) raft_consensus.cc:1685] finished
0709 12:26:42.084543 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.084828 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19734,"raft.queue_time_us":14709,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":13484,"spinlock_wait_cycles":6528768,"thread_start_us":84426,"threads_started":1,"wal-append.queue_time_us":276803}
I20260709 12:26:42.170315  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.168231  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:42.190387  7732 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:42.165241  7737 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:42.192348  7734 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.174818  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.193449  7655 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.188045  7805 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:42.145411 11229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.013s	sys 0.066s
I20260709 12:26:42.132364  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.162242  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.196218  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:42.197402  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 164344us
I20260709 12:26:42.197713  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.198994  7656 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:42.194343  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2534) took 1804 ms (client timeout 2091 ms). Trace:
I20260709 12:26:42.196219  7658 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:42.201953  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.182619  7654 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:42.210794  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.147082 11299 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:42.200134  7805 rpcz_store.cc:269] 0709 12:26:40.390147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.431898 (+ 41751us) service_pool.cc:224] Handling call
0709 12:26:42.115396 (+1683498us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:42.194275 (+ 78879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647759232}
I20260709 12:26:42.199170  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.222136  7728 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:42.217209  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.222898  7656 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:42.224272  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.225158  7656 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:42.226166  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.226783  7656 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:42.215837  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.231076  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.231894  7658 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:42.227602 11246 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:42.229226  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.237582  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.242657  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:42.184412 11139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.013s	sys 0.066s
W20260709 12:26:42.244640  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45556us
I20260709 12:26:42.247582  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:42.248381  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.249202  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.249264  7812 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:42.217216  7654 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:42.183545 11164 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:42.250583  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:42.219779 11229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.014s	sys 0.066s
I20260709 12:26:42.278877  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.258952  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:42.260736  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 92689.7us
I20260709 12:26:42.284763  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:42.247468  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.287029  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.277666 11316 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:42.288457  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.160285 11002 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:26:42.171355 11314 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.289990  7805 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:42.290638  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.291316  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.277868  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.306226  7811 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:42.306653  7656 raft_consensus.cc:2376] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7. Re-sending same reply.
I20260709 12:26:42.297260  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.300926  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.307209  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.307934  7811 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:42.221973 11280 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:42.309160  7811 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:26:42.244550 11351 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:42.311698  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.316701  7806 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:42.317340  7803 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:42.252314  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.320773  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.329893  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.329988  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.336498  7654 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:42.336691  7656 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.332813  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.339931  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 610) took 2146 ms. Trace:
I20260709 12:26:42.262027 11330 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:42.340361  8408 rpcz_store.cc:276] 0709 12:26:40.193841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.195528 (+  1687us) service_pool.cc:224] Handling call
0709 12:26:40.195990 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.191617 (+1995627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.339646 (+148029us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:42.339870 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4541715328}
I20260709 12:26:42.191447 11187 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.260792 11246 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:42.343379  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.354296  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.300168 11269 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:42.344290  7807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.039s	sys 0.123s
W20260709 12:26:42.359454  7807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.040s	sys 0.123s
I20260709 12:26:42.260079 11164 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.278339 11303 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.334044  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49418.5us
I20260709 12:26:42.351027  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.328073 11280 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.348660  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:42.290153 11139 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:42.354022  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.293079 11271 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:42.337666  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:42.261127 11268 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 12:26:42.360343  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.372097  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:42.366837  7658 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:42.405622  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.304507 11281 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.326128 11314 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.340987  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:42.385310  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:42.430573  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45421us
I20260709 12:26:42.370783  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.384264  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.338717 11315 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.389492  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:42.391407  7731 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:42.393608  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.385231  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.438444  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:42.395025  8414 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:42.396153  7654 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.439388  7659 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:42.440207  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.440510  7659 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:26:42.440860  7807 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.306361 11167 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:42.355819 11324 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:42.356940 11335 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:42.446431 11335 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:42.446940 11335 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.406783  7658 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:42.452504  7658 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:42.312318 11316 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:42.453614 11355 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:26:42.454334 11316 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.454891 11315 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.455578  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 917) took 1037 ms. Trace:
I20260709 12:26:42.456023  8413 rpcz_store.cc:276] 0709 12:26:41.417982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.418463 (+   481us) service_pool.cc:224] Handling call
0709 12:26:41.446429 (+ 27966us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.446868 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.446898 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.452022 (+  5124us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.952449 (+500427us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:41.952493 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.952568 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.455305 (+502737us) raft_consensus.cc:1685] finished
0709 12:26:42.455327 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.455508 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2264,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8757,"thread_start_us":435941,"threads_started":1,"wal-append.queue_time_us":704832}
I20260709 12:26:42.457417 11315 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.408182  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.443002  7659 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:42.356614 11057 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.369869 10845 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:42.466636  7658 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:42.385622 11164 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.382117  7656 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:42.356024 11345 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:26:42.346050  7548 maintenance_manager.cc:419] P 004cbdb090d14f8ca89ed535ed34a625: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033396
I20260709 12:26:42.399469 11280 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.316773 10979 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.408910 11139 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:42.445374 11324 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.445626 11167 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.463618  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:42.429090 11257 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.656s	user 0.013s	sys 0.000s
I20260709 12:26:42.465858  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.371343 11289 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:42.410729 11347 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:42.476056  7654 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:42.455618 11201 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:42.427696 10925 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.483436  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:42.430498 11309 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.397877 11303 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.528471  8413 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:42.517915  8408 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:42.532274  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.533068  7811 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:42.521819  7654 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:42.524695  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.539319  7805 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:42.548064  7624 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.016s	sys 0.027s
W20260709 12:26:42.548533  7624 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.016s	sys 0.027s
I20260709 12:26:42.548069  7812 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.9-6.9]
W20260709 12:26:42.548934  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 1.08843e+06us
W20260709 12:26:42.549507  7624 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.596s (SENT) [suppressed 33 similar messages]
I20260709 12:26:42.494001 11167 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.493891 11324 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.515618 11289 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.496217  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.557204  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:42.522895 11260 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.477685 11057 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.516798 11072 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.518805 11347 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.575829  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.578344  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.579422  8408 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:42.580785  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.591331  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:42.593307  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.548341 11201 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:42.598121  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.569583 11289 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.600518  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:42.606328  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1983) took 1207 ms. Trace:
I20260709 12:26:42.611879  7660 rpcz_store.cc:276] 0709 12:26:41.418711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.467478 (+ 48767us) service_pool.cc:224] Handling call
0709 12:26:41.467813 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.966264 (+498451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.605938 (+639674us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54000f4e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:42.606232 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591773056}
W20260709 12:26:42.603041  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 248815us
I20260709 12:26:42.614003  7660 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:42.586825 11347 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:42.621560  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 914) took 1581 ms. Trace:
I20260709 12:26:42.554566 11309 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:42.624308  8411 rpcz_store.cc:276] 0709 12:26:41.040228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.040567 (+   339us) service_pool.cc:224] Handling call
0709 12:26:41.066277 (+ 25710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.334840 (+268563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.334872 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.334919 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:41.334965 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.621183 (+1286218us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54007b3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:42.621264 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.621473 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617017088}
W20260709 12:26:42.635407  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 445092us
W20260709 12:26:42.624387 11357 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:26:42.636147  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1984) took 1182 ms. Trace:
I20260709 12:26:42.646473  7657 rpcz_store.cc:276] 0709 12:26:41.454209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.470406 (+ 16197us) service_pool.cc:224] Handling call
0709 12:26:41.473092 (+  2686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.473389 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:41.970603 (+497214us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.635593 (+664990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.635682 (+    89us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:42.635774 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.635883 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.636047 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":635213,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":4734,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":20321,"spinlock_wait_cycles":21289728,"thread_start_us":1119070,"threads_started":3,"wal-append.queue_time_us":501188}
I20260709 12:26:42.646943  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.635402  7655 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:42.613533 11281 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:42.656766  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 315849us
I20260709 12:26:42.651150  7655 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:42.636941 11272 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.828s	user 0.008s	sys 0.006s
I20260709 12:26:42.627413 11264 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.670639  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.611832  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.649551  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.670643  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.636606 11187 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:42.678012  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 291502us
I20260709 12:26:42.680869  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.613807 11299 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.685230  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.653755 11263 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.694519  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 529537us
I20260709 12:26:42.677044 11272 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.715729  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:42.668223 11331 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:26:42.605180 11139 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.667987 11260 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 12:26:42.717847  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.694839 11299 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:42.693076 11360 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:42.695644 11345 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:42.719844  7807 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:42.722500  7659 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:42.686700 11168 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:42.727556  7737 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:42.727990  7733 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:26:42.688752 11072 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:42.739405  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.744658  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 640) took 1147 ms. Trace:
I20260709 12:26:42.745357  7659 rpcz_store.cc:276] 0709 12:26:41.597277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.721786 (+1124509us) service_pool.cc:224] Handling call
0709 12:26:42.722248 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.737727 (+ 15479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.744539 (+  6812us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:42.707773 11361 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 12:26:42.689898 11330 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:42.755306  8414 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 4: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:42.759248  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.759681  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.761317  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.765152  7659 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:42.725752 11257 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:42.768731  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:42.769098  7730 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:42.769699  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:42.741003 11356 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:42.771529  7737 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:42.780426  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.753961 11314 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.781039  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:42.786582  7735 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:42.744011 11345 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:42.793578  7730 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:26:42.794216  7730 pending_rounds.cc:85] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:26:42.794525  7730 pending_rounds.cc:107] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:26:42.795177  7730 raft_consensus.cc:2889] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:42.761902 11002 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:42.739348 11315 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:42.760967 11303 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:42.721005 11263 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:42.810173 11058 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.001s
I20260709 12:26:42.767021 11072 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:42.796718 11345 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.815392 11315 consensus_queue.cc:1243] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:42.815652 11330 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.725481 11187 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:26:42.723022 11201 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:42.869268  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 251644us
I20260709 12:26:42.816792 11173 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:42.862653  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1990) took 1037 ms. Trace:
I20260709 12:26:42.872359  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.874238  7656 rpcz_store.cc:276] 0709 12:26:41.825602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.491785 (+666183us) service_pool.cc:224] Handling call
0709 12:26:42.492137 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.492391 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:42.552390 (+ 59999us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.861928 (+309538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.862037 (+   109us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:42.862115 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.862231 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.862534 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32097,"prepare.queue_time_us":61567,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":276828,"threads_started":1}
I20260709 12:26:42.878347  7656 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:42.868048 11319 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.007s
W20260709 12:26:42.864310 11321 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.003s
I20260709 12:26:42.892627  7807 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:26:42.873668 11365 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:26:42.865751 11057 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.807018 11280 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.905658  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.856053 11164 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:42.891100 11315 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:42.856336 11058 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.912024  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.923331  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.924304  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.924218 11072 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:42.913236  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 228090us
I20260709 12:26:42.916917 11246 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:42.917893  7811 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:42.887498 11314 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:42.918121  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:42.912529  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:42.923453  7806 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:42.904974 11173 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:42.919946 11315 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:42.895026 11257 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:42.933668 11272 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:42.926918  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.929979 11314 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.897650 11319 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.976421 11315 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.978116 11315 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:42.978308 11319 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.979698 10978 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:42.980898 10978 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:42.981272 10978 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:42.982213 10978 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.985180 10978 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.948864  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:42.949225  7729 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:42.927817 11246 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:42.946741 11164 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:42.968684  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:42.997756  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70890.5us
I20260709 12:26:42.897701 11321 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:42.976132 11314 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.986478  7733 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:42.956616 11367 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 12:26:42.939188  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:43.002748  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63679.1us
I20260709 12:26:43.003878  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:43.004908  7733 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.965716  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:42.959014 11264 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.961081  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 290536us
I20260709 12:26:43.006692  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.955992 11057 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:26:43.036042  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.995359 11246 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:42.930224 11373 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
W20260709 12:26:43.009570 11266 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.873s	user 0.013s	sys 0.002s
I20260709 12:26:43.012521 11257 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:43.042657  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 361887us
I20260709 12:26:43.066197 11257 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.066690  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:43.043742  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.042203  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:42.985389 11324 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.009440 11272 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:43.043747  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 164661us
I20260709 12:26:43.066864 11257 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.070246  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:42.989979 11380 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:26:43.067947  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:43.071574  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.073287  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.073339  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.051409 11187 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:43.057475 11164 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:43.077502  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.081488  7737 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:43.078615 11371 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:43.088977 11377 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:43.089278 11371 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:43.089942 11371 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:43.015625 11264 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:43.100212  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1966) took 2216 ms (client timeout 1928 ms). Trace:
W20260709 12:26:43.102782  7662 rpcz_store.cc:269] 0709 12:26:40.884971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.885463 (+   492us) service_pool.cc:224] Handling call
0709 12:26:43.099933 (+2214470us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:43.100110 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404694912}
W20260709 12:26:43.104418  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.082847 11335 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 12:26:43.113569  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:43.117213  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:43.115464  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.118350 11280 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:43.123864  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52381.6us
I20260709 12:26:43.135316  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.156711  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:43.124423 11321 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:43.125049 11366 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:43.126698 11324 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:43.126945 11382 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:43.127302 11372 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:43.127873 11379 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:43.131944 10925 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:43.173589 10925 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:43.174045 10925 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:43.175066 10925 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.176507 10925 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:43.169921 11381 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.243s	user 0.000s	sys 0.003s
I20260709 12:26:43.132886 11168 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:43.161355 11366 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.183806 11366 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.132781 11388 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:43.050365 11167 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:43.137297  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.084003 11164 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:43.133616 11246 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:43.194366  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120892us
I20260709 12:26:43.098838 11187 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:43.169584  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.170831  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:43.116000 11263 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.171659 11260 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:43.172201 11303 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.158131 11385 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:43.196316  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.198478  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 677) took 1499 ms. Trace:
I20260709 12:26:43.203440  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.237360  7729 rpcz_store.cc:276] 0709 12:26:41.698651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.700671 (+  2020us) service_pool.cc:224] Handling call
0709 12:26:41.700993 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.986294 (+1285301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.198273 (+211979us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400283054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:43.198415 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2870226304}
I20260709 12:26:43.238579  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:43.227250  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56468.6us
I20260709 12:26:43.156277 11335 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.232025  7736 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:43.238859  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:43.230194 11316 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.194391 11164 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:43.241766  7729 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:43.251974  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:43.240178 11281 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:43.302181  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:43.236603 11249 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:43.304831  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 1968) took 2206 ms (client timeout 1670 ms). Trace:
W20260709 12:26:43.308728  7653 rpcz_store.cc:269] 0709 12:26:41.098744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.098916 (+   172us) service_pool.cc:224] Handling call
0709 12:26:43.304539 (+2205623us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54004371d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:43.304743 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263552256}
I20260709 12:26:43.241271 11331 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:43.311921  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60061us
I20260709 12:26:43.337721  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:43.302910  7734 raft_consensus.cc:2376] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7. Re-sending same reply.
I20260709 12:26:43.329128  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.331297  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 554) took 1790 ms. Trace:
W20260709 12:26:43.278465 11363 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.725s	user 0.000s	sys 0.004s
I20260709 12:26:43.197834 11371 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.288275 11255 raft_consensus.cc:993] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:26:43.342952  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:43.330017 11355 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:26:43.250810 11319 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:43.248281 11391 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:43.362545  7733 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:43.308064 11249 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.273150 11248 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:43.391793  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:43.391999  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:43.358706 11168 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:43.368038  7728 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:43.368531  7658 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:43.360294 11387 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:43.241882 11385 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:43.341336 11347 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 12:26:43.339298  7809 rpcz_store.cc:276] 0709 12:26:41.541229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.541877 (+   648us) service_pool.cc:224] Handling call
0709 12:26:41.545675 (+  3798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.215031 (+669356us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:42.261543 (+ 46512us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.551687 (+290144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.551744 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:42.551792 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.330895 (+779103us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:43.330970 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.331193 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7821,"prepare.queue_time_us":77470,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4855,"raft.queue_time_us":7949,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12828,"spinlock_wait_cycles":1538665216,"thread_start_us":259704,"threads_started":1,"wal-append.queue_time_us":364232}
I20260709 12:26:43.388998  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.372442  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.332775 11331 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.276224 11264 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.394523  7729 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
I20260709 12:26:43.403455  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1987) took 1764 ms. Trace:
I20260709 12:26:43.420607  7658 rpcz_store.cc:276] 0709 12:26:41.639319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.361564 (+1722245us) service_pool.cc:224] Handling call
0709 12:26:43.361955 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.403021 (+ 41066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.403364 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:43.388056 10796 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:43.267364 11316 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.394053 11249 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:43.396324 11248 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.279765 11187 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.372408 11373 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.413532 11345 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.000s
W20260709 12:26:43.429188  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57021.8us
I20260709 12:26:43.436355 11167 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:43.408699 11260 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:43.400745 11002 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 12:26:43.422706 11289 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.793s	user 0.008s	sys 0.006s
W20260709 12:26:43.443858 11396 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 12:26:43.444749 11404 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:26:43.448895  7728 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:43.449167 11257 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:43.428879 11351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.018s	sys 0.010s
I20260709 12:26:43.460108 11335 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.462930 11246 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.464926  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 614) took 1002 ms. Trace:
I20260709 12:26:43.465404 11411 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:43.465751  7731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2451) took 1455 ms (client timeout 1686 ms). Trace:
W20260709 12:26:43.466517 11402 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:43.420434 11331 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.416056 11385 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.426672 11264 raft_consensus.cc:1081] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:26:43.503862  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:43.517768 11351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.018s	sys 0.010s
I20260709 12:26:43.477365 11333 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:43.482456 10796 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:43.518656  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 121052us
W20260709 12:26:43.520824  7731 rpcz_store.cc:269] 0709 12:26:42.010111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.047643 (+ 37532us) service_pool.cc:224] Handling call
0709 12:26:43.465533 (+1417890us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540037c154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:43.465691 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588158208}
I20260709 12:26:43.520998  8414 rpcz_store.cc:276] 0709 12:26:42.462892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.463270 (+   378us) service_pool.cc:224] Handling call
0709 12:26:43.464671 (+1001401us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:43.464847 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578219776}
I20260709 12:26:43.525210  8409 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:43.421645 11347 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.478247 11345 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.545167  7731 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:43.534531  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:43.536118 11266 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.468549 11248 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:43.535444 11391 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.514580 11168 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.548871  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:43.562299  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.554312  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 177228us
I20260709 12:26:43.557346  7731 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:43.422246 11387 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.567396 11168 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:43.538628 11394 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.335s	user 0.001s	sys 0.001s
I20260709 12:26:43.568153 11347 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.562443 11333 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:43.569653 11410 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:43.570017 11333 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:43.570676 11314 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.002s
I20260709 12:26:43.503741 11289 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.560329 11411 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:43.565680  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:43.478520 11373 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.538295 11351 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:43.560853 11268 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:43.585984 11072 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:43.584182 11397 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:26:43.560218 11057 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:43.578639  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.573199  7659 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:43.571193 11314 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.598306 11167 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:43.663410  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2075) took 1795 ms. Trace:
I20260709 12:26:43.668896  7812 rpcz_store.cc:276] 0709 12:26:41.868520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.307881 (+439361us) service_pool.cc:224] Handling call
0709 12:26:42.308230 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.549122 (+240892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.663006 (+1113884us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:43.663315 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523711360}
I20260709 12:26:43.599900 11330 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:26:43.567082 11263 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:26:43.602958 11268 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:43.673175  7735 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:43.674664  7735 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:43.660441 10978 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:43.675807  7735 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.677389 11268 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:43.683331  7735 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.671840 11371 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.688297  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 562) took 1849 ms. Trace:
I20260709 12:26:43.677083 11167 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:43.688805  7808 rpcz_store.cc:276] 0709 12:26:41.838829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.838965 (+   136us) service_pool.cc:224] Handling call
0709 12:26:41.977026 (+138061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.977386 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.977414 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.018042 (+ 40628us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.226371 (+1208329us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b18002f5180. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:43.535435 (+309064us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:43.535484 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.535584 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.684188 (+148604us) raft_consensus.cc:1685] finished
0709 12:26:43.684215 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.688213 (+  3998us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":263,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":26375,"spinlock_wait_cycles":107003264,"thread_start_us":305082,"threads_started":1,"wal-append.queue_time_us":1496177}
I20260709 12:26:43.664952 11057 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.703015  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:43.704380  7810 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:43.706629  7812 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:26:43.707518  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.722025  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:43.722378  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:43.722800  7810 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.663939 10796 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.688529 10978 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:43.726433  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:43.729282  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:43.729880  7735 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 12:26:43.730548  7735 pending_rounds.cc:85] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:26:43.730826  7735 pending_rounds.cc:107] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 12:26:43.731142  7735 raft_consensus.cc:2889] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:43.731832 11422 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:26:43.733915 11398 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:26:43.736124  7731 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:43.734668  7735 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:43.730000  7809 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:43.723951 11415 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:43.744455  7731 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:43.684746 11411 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.747658 11345 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:43.749881 11420 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:26:43.751190 11420 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:43.705549 11351 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.751698 11322 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:43.754338 11340 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:43.754663 11289 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.754855 11340 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:43.755105 11369 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 12:26:43.747722 11400 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:26:43.760327 11415 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.774199  7810 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
W20260709 12:26:43.774798  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.754338 11057 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:43.740460  7735 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:53564
I20260709 12:26:43.778195  7735 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:43.778785  7735 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:43.756014 11373 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:43.788524  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58786.9us
I20260709 12:26:43.745942 11072 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:43.797433  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.799759  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.800951  7735 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.776350  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 925) took 1052 ms. Trace:
I20260709 12:26:43.737668 11167 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.809031  8412 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:43.805330  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:43.808543  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.810904  8411 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:43.810590  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.812664  7735 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:43.815722 11408 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.226s	user 0.001s	sys 0.001s
I20260709 12:26:43.820860 11072 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:43.821961  8407 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.9-6.9]
W20260709 12:26:43.824829 11164 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.002s
I20260709 12:26:43.825333 11164 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.755908 10978 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.829851  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:43.827940  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.832867  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:43.837731  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:43.839509  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:43.842941  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:43.852240  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.853269  8411 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:43.853744  8411 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:43.837801 11315 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.841584 11373 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:43.852291 11319 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:43.848893 11057 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:43.872047 11319 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.812343 11340 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.870329  8411 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.887657  8411 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:43.885020  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:43.889143  8413 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:43.865559 11272 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:43.889925 11272 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:43.892129 11272 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:43.892527 11430 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:26:43.893491  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:43.893910 11430 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:43.894106 10979 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:43.894613 11430 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:43.899685 11415 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:43.894186  7734 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:43.905807  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:43.906919  7735 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:43.908730  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 174692us
I20260709 12:26:43.914932  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:43.954006 11427 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:43.962702  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 680) took 1014 ms. Trace:
I20260709 12:26:43.963322  7733 rpcz_store.cc:276] 0709 12:26:42.948150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.393496 (+445346us) service_pool.cc:224] Handling call
0709 12:26:43.468064 (+ 74568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.468227 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.468256 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.468296 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:43.468328 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.468388 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.962445 (+494057us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b540037c150. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:43.962608 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170753408}
I20260709 12:26:43.942926  8411 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:43.967695  7804 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:43.967906  7700 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:43.900914 11164 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:43.806145  8405 rpcz_store.cc:276] 0709 12:26:42.724248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.728466 (+  4218us) service_pool.cc:224] Handling call
0709 12:26:42.819475 (+ 91009us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:42.887927 (+ 68452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.896572 (+  8645us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:42.991960 (+ 95388us) log.cc:844] Serialized 52 byte log entry
0709 12:26:43.227802 (+235842us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:43.230877 (+  3075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.241005 (+ 10128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.770580 (+529575us) raft_consensus.cc:1685] finished
0709 12:26:43.770605 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.776272 (+  5667us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12383,"thread_start_us":113535,"threads_started":1,"wal-append.queue_time_us":488322}
I20260709 12:26:43.904778 11315 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:43.908460 11340 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:43.962358 11366 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.772s	user 0.014s	sys 0.000s
I20260709 12:26:43.890981 11373 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
W20260709 12:26:43.989436 11432 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:43.991348 11058 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.000582  7731 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:44.002696  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:43.951645 11335 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:43.992563  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:43.981352 11381 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:43.951900 11201 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:43.976145 11057 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:44.002769  7806 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:44.004319  7811 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:43.992041 11425 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:43.985865 11369 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.015486 11369 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.010504  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:43.996762 11315 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.016866  7803 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.018563 11315 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.018615 11369 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.013562  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:44.029284  7808 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:44.016010  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:43.961309 11428 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:44.019812  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:44.032159 11057 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:44.047609 11057 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.050969  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.052380 11387 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.055603  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 679) took 1107 ms. Trace:
I20260709 12:26:44.056115  7737 rpcz_store.cc:276] 0709 12:26:42.947830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.949521 (+  1691us) service_pool.cc:224] Handling call
0709 12:26:42.949938 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.084932 (+134994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.055317 (+970385us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:44.055518 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301942656}
I20260709 12:26:44.005961 11411 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.012775  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:44.065665  7804 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:44.033278  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:44.067531  7808 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:44.026183 11331 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:44.060173  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.062386  7803 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:44.040346  7811 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:44.081954  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:44.068131 11249 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.002s
I20260709 12:26:43.998781 11366 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.063580 11431 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.042618 11385 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.101601  7804 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:44.103845  7805 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:44.110785 11314 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.119469  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:44.121407  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:44.123378  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:44.124681  7803 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:44.102886 11431 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.130514 11431 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:44.131680 11395 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.961s	user 0.000s	sys 0.010s
I20260709 12:26:44.133042  8415 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:44.134117 10978 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.134984 10978 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.135813  8409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 616) took 1670 ms (client timeout 1803 ms). Trace:
W20260709 12:26:44.136283  8409 rpcz_store.cc:269] 0709 12:26:42.465781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.475608 (+  9827us) service_pool.cc:224] Handling call
0709 12:26:43.328883 (+853275us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:44.135556 (+806673us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:44.135720 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276003584}
I20260709 12:26:44.139777  8407 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:44.140579  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:44.143807  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:44.075569 11411 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.142639 11314 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:44.148212 11315 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:44.134120 11369 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:26:44.146790 11373 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:44.149366 11248 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.002s
I20260709 12:26:44.160421 11248 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.163525  7810 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:44.165725 11331 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:44.149681 11351 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:44.097193 11249 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.185108 11255 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:44.185928 11255 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:44.186321 11249 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:44.187441  7812 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:44.172140 11168 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.174504 11366 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:44.176275 11316 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.177556 11257 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.179615  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:44.186556  8412 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:26:44.197338  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:44.198559  7808 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:44.202904  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:44.203773  7804 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:44.172513 11268 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.215834  8413 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:44.207722  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:44.217657  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:44.233142  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:44.234112  8409 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:44.163156 11248 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:44.237298  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:44.238371  7730 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:44.240935  7811 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:44.205981 11351 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:44.212232 11314 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:44.214733 11315 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:44.231434 11428 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:44.246788 11255 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.262372 11411 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.212889 11420 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:26:44.231783 11373 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:44.241240 11385 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.216598 11340 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:44.283293  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:44.279801  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:44.285179  7730 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:44.280570  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:44.277592  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:44.289031  7737 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:44.291553 11319 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.291839 11373 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.294209 11319 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.295678 10979 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:44.296125 11441 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:44.295745  8405 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:44.296999 11441 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:44.299952 11441 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.307950 11414 raft_consensus.cc:1064] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
W20260709 12:26:44.308830 11414 raft_consensus.cc:1068] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Unable to promote non-voter 992d6df95ef74f7f8b12534534a45501: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:44.310932 11268 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.265913 11421 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:26:44.312786 11421 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:44.313342 11421 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:44.319949 11314 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.266956 11428 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.266050 11351 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.336697  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.329398 11255 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:44.331404 11428 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.338223  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:44.338763  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54368.3us
I20260709 12:26:44.351593  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:44.347623 11168 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:26:44.314579 11421 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.365777 11421 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.337709 11315 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:44.369553  8415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 754) took 2050 ms (client timeout 1992 ms). Trace:
W20260709 12:26:44.370040  8415 rpcz_store.cc:269] 0709 12:26:42.319618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.320359 (+   741us) service_pool.cc:224] Handling call
0709 12:26:44.369344 (+2048985us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:44.369475 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4132579072}
I20260709 12:26:44.348102 11268 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:44.374014 11272 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:44.376160 11268 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:44.416442  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 183543us
I20260709 12:26:44.420186  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:44.431871  7803 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 12:26:44.432260  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:44.440528  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:44.447911 10925 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:44.452827 10925 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:44.455190  7733 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:44.457258 10796 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:44.457890 10796 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:44.461829 11057 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:44.464085 10796 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.467761 11268 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.472046 10978 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:44.465083  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:44.473148 10796 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.478474 10978 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:44.479118 10978 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.434470  7810 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:44.484956 10978 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.519826  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:44.521945  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:44.540622 11229 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:44.505865 11057 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:44.683010  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 129363us
I20260709 12:26:44.566386  7728 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:26:44.698799  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.624970  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53464us
W20260709 12:26:44.564484  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.717358  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72267.4us
I20260709 12:26:44.697363  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2204) took 1935 ms. Trace:
I20260709 12:26:44.568910  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 1994) took 2598 ms. Trace:
I20260709 12:26:44.566741 11289 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:44.649250 11369 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.757404 11289 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.758069 11369 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:44.761577 11448 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.217s
W20260709 12:26:44.692693 11433 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.801s	user 0.003s	sys 0.000s
I20260709 12:26:44.785219 11289 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.643424 11257 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.788530 11268 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.807690  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43651.6us
W20260709 12:26:44.811430  7730 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.020s
I20260709 12:26:44.812126  7730 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
W20260709 12:26:44.812711  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 1.69898e+06us
I20260709 12:26:44.814133  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:44.813922 11257 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.816264  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.817389 11257 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.804734 11057 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.820019 11201 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:44.820827 11201 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:44.668131 11441 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 12:26:44.727990  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:44.746140  7736 rpcz_store.cc:276] 0709 12:26:42.761313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.761912 (+   599us) service_pool.cc:224] Handling call
0709 12:26:44.696776 (+1934864us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:44.697041 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036373888}
I20260709 12:26:44.822475 11201 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:44.897634  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80276.5us
I20260709 12:26:44.905891 11164 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:44.906539 11164 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:44.906941 11173 raft_consensus.cc:2764] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:44.908265 11441 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:44.908578 11173 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.908918 11441 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:44.909582 11173 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
W20260709 12:26:44.911607  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.911749 11280 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:44.912920 11280 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:44.913348 11280 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.913770 11280 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:44.914408 11268 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:44.905586 11415 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:44.907051 11164 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.879873 11369 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:44.928146  7730 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:44.910465 11441 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:44.976317 11246 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:44.985152  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85429.1us
I20260709 12:26:44.968515 11201 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:44.747198  7654 rpcz_store.cc:276] 0709 12:26:41.977269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.538171 (+560902us) service_pool.cc:224] Handling call
0709 12:26:42.538430 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.538631 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:42.935503 (+396872us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.111527 (+176024us) log.cc:844] Serialized 14 byte log entry
0709 12:26:44.131015 (+1019488us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:44.304263 (+173248us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18003ab640. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:26:44.567528 (+263265us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b18003ab640. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:26:44.567932 (+   404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:44.568060 (+   128us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:44.568147 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.568320 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.568575 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84997,"prepare.queue_time_us":109451,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":4744,"raft.queue_time_us":13164,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6620,"spinlock_wait_cycles":71245184,"thread_start_us":477979,"threads_started":2,"wal-append.queue_time_us":176052}
W20260709 12:26:45.003958  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 78838.8us
W20260709 12:26:45.026259 11272 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.006s
I20260709 12:26:45.028175 11309 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:45.029414 11430 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.032603  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 309189us
W20260709 12:26:45.075086  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52554.2us
I20260709 12:26:45.064062 11441 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:45.065886 11430 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.068882  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 645) took 2617 ms. Trace:
W20260709 12:26:45.072981 11406 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.652s	user 0.001s	sys 0.005s
W20260709 12:26:45.074481  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.95881e+06us
I20260709 12:26:45.030826 11335 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:45.055727 11428 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.705s	user 0.015s	sys 0.008s
I20260709 12:26:45.076421  7659 rpcz_store.cc:276] 0709 12:26:42.451572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.770749 (+319177us) service_pool.cc:224] Handling call
0709 12:26:42.771223 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.614675 (+843452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.036008 (+1421333us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54002198d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:45.068625 (+ 32617us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843859328}
I20260709 12:26:45.103351  8408 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:45.100746  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.123569  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:45.090837 11428 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.156210  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 631) took 1164 ms. Trace:
I20260709 12:26:45.156772  8408 rpcz_store.cc:276] 0709 12:26:43.992143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.992301 (+   158us) service_pool.cc:224] Handling call
0709 12:26:45.155885 (+1163584us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:45.156063 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346045184}
W20260709 12:26:45.159777 11467 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:26:45.171799  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.179687 11173 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:45.180440  7777 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.776s (SENT)
I20260709 12:26:45.181454  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:45.181857  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 367966us
I20260709 12:26:45.183079  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.190516  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.191097  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.191689  7655 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:45.191892  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.192466  7659 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:45.192747  7654 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:45.194342  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.195567  7658 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:45.196025 11255 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.776s	user 0.016s	sys 0.009s
I20260709 12:26:45.196859 11399 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:45.197906 11399 raft_consensus.cc:2764] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:45.198122  8409 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:45.198799 11399 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.203938 11399 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.199476 11309 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:45.209475  7662 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:26:45.211117  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.89621e+06us
I20260709 12:26:45.210685 11309 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.200582 11347 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:45.200644  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.196993  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:45.209506  7658 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:44.977859  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:45.202772  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:45.203029 11393 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:45.199451 11246 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.207830  7655 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:26:45.208190  7659 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:45.202282  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 116598us
I20260709 12:26:45.226048 11316 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.226887  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:45.227025 11246 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.229411  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:45.230203  7654 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:45.230252 11246 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.226207  7653 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
W20260709 12:26:45.233598  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 64120.6us
W20260709 12:26:45.217859 10978 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.006s
I20260709 12:26:45.236887  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:45.196957 11255 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:26:45.239444 11399 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.227020 11316 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.232762 11433 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 12:26:45.217001 11347 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:45.253564 11167 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.253679 11139 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:45.254647 11369 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:45.259101  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:45.259438 11393 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:45.267475  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:45.269520 11428 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.272190  8415 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.278839 11249 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:45.280802  8407 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:26:45.280633  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 100323us
I20260709 12:26:45.281709 11414 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:45.285169  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.287622 10979 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:45.291316  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.298727  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:45.306377  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:45.329373  7653 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:45.307263  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.307632 11461 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.308214  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.309011  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:45.309245  7660 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:26:45.310359  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:45.324923  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 171595us
W20260709 12:26:45.332509  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.871s (SENT)
I20260709 12:26:45.333508  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:45.334089 10796 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:45.336237  7654 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
W20260709 12:26:45.339717  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.345240  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 108479us
I20260709 12:26:45.347640 11347 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.348366  7657 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:45.349001 11347 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:45.351401 11167 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.352438  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:45.353075 11316 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:45.353746 11393 consensus_queue.cc:1243] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:45.354013  7655 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:45.354185 11167 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.354732 11461 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.356678 10979 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.342245 10796 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:45.359966 11393 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:45.359745 10796 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.360570 11393 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:45.361294 11393 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:45.361564  7659 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.334592 11399 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.364187  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:45.314980 10978 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.370297  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71854us
I20260709 12:26:45.383746  7653 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:45.386441  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:45.388149  8407 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:45.390228  7811 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:45.392069  7812 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:45.383064 10796 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:45.383507 11461 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.424808  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:45.391569 11167 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.425861  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.428217  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:45.438753  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.457355  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 124953us
I20260709 12:26:45.456454  8413 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:45.465272 11167 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:45.466132  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.408267 11316 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.454547  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:45.472417  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 579) took 1029 ms. Trace:
I20260709 12:26:45.454123 11369 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.475502  8409 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:45.454625 11366 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:45.481338  8415 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:45.477057  8407 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:26:45.469504 11280 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.555s	user 0.016s	sys 0.000s
I20260709 12:26:45.484931  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:45.489518  7653 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:45.490553  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:45.493773  8411 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:45.472123 11249 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:26:45.503038  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100136us
I20260709 12:26:45.504626  7812 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:45.481303 11428 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.503412  7654 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
I20260709 12:26:45.509424  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:45.501996 11164 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.001s
I20260709 12:26:45.472899  7809 rpcz_store.cc:276] 0709 12:26:44.443198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.444760 (+  1562us) service_pool.cc:224] Handling call
0709 12:26:44.445446 (+   686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.446182 (+   736us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:44.708764 (+262582us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.034331 (+325567us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b18000c82c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:26:45.467247 (+432916us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:45.467329 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.470948 (+  3619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.471046 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.472352 (+  1306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62,"prepare.queue_time_us":204034,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2798,"raft.queue_time_us":4062,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":152,"spinlock_wait_cycles":45846272,"thread_start_us":611119,"threads_started":2}
I20260709 12:26:45.481045 10979 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:45.499194 11461 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.519336  7811 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:45.522114  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.522590  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:45.501447 11366 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:45.479681 11472 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.541011  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:45.541754  7811 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:45.500144 11316 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.529332  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:45.549331  7736 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.551945  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.552901  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:45.526147 11414 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:45.506480 11280 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:45.561780  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.017s (SENT)
I20260709 12:26:45.561357  7654 raft_consensus.cc:2410] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:45.563280  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.508493 11249 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.565384  8415 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:45.547983 11057 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:26:45.520201 11164 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.557760 11167 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.569496  7812 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:45.523741 11369 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.534245 11428 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.577203  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.582008  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.558116 11246 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:45.564848 10978 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.566234 11472 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.548635 11139 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 12:26:45.568728  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 329493us
I20260709 12:26:45.565186 11414 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.561363 11461 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.594674 11201 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.008s
I20260709 12:26:45.604378  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:45.610473  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.620004  7737 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:45.591581 11260 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:45.626215  7661 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:45.626631  7729 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:45.630124 11421 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.630587 11351 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:26:45.631137 11321 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:26:45.631520 11366 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:45.608501 11441 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.002s
I20260709 12:26:45.593873 11268 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:45.647446 11168 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:45.652727  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90983.8us
I20260709 12:26:45.649832 11421 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.660121 11414 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:45.660754 11260 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.664028 11472 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:45.672043  7653 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:45.691515  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88468.2us
I20260709 12:26:45.669293  7811 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
W20260709 12:26:45.672415  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89284.6us
I20260709 12:26:45.672847  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:45.665517 11139 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:45.711827  7777 leader_election.cc:336] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.038s (SENT)
I20260709 12:26:45.672544 11168 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:45.721231  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:45.718400  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.735404 11002 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:45.737389 11474 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:45.737231  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:45.741272  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.729763  7660 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:45.745448  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.753489  7653 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:45.756857  7657 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:26:45.757009  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 192138us
I20260709 12:26:45.767992 11268 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:45.782491  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.793905 11072 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:26:45.805490 11314 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 12:26:45.806038  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116230us
I20260709 12:26:45.806051  7657 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:26:45.809391  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 158182us
I20260709 12:26:45.809985 11168 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.759177 11321 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:26:45.813069  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.784478 11173 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:45.828377 11272 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:45.832141  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:45.838469  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 132802us
W20260709 12:26:45.839692  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.844551 11002 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.803936 11280 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.807469 11268 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:45.813083 11164 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.768559 11139 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:45.825038 11316 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.837266 11272 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:45.895987  7736 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:45.899127  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:45.900112  7812 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.902412 11345 consensus_queue.cc:765] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 8 is ahead of the local log (next sequential op: 8). Destination peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 9, Last known committed idx: 8, Time since last communication: 0.644s
W20260709 12:26:45.903033 11479 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:26:45.879030 11246 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:45.865141 11002 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:45.909571  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.914601  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 166431us
I20260709 12:26:45.918128  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:45.919114  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:45.848156 11441 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:45.874034 11401 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:45.894134 11335 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:45.899807 11057 raft_consensus.cc:2764] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:45.933279  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:45.933995  7732 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.908563 11316 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:45.914559 11280 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.913798 11369 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:45.923460 11428 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:45.923702 11249 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:45.923095 11002 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:45.927340 11401 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:45.952006  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:45.953384  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:45.964071  7659 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
I20260709 12:26:45.964955  7659 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:45.983527  7661 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:45.984416 11484 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:45.988157  7657 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:45.991276  7662 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:45.991472 11280 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:45.994701 11280 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:45.997576 11249 raft_consensus.cc:1064] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:26:45.943921 11421 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:45.967195 11272 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:46.004061  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47539.6us
I20260709 12:26:46.021811  8405 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 12:26:46.022060  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.024044  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.024610  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 702) took 1471 ms. Trace:
W20260709 12:26:46.012136 11481 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:46.024991  7734 rpcz_store.cc:276] 0709 12:26:44.552891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.553044 (+   153us) service_pool.cc:224] Handling call
0709 12:26:44.664221 (+111177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.700244 (+ 36023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.700300 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.160732 (+460432us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:26:45.161284 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.523825 (+362541us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:45.523871 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.523947 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.024157 (+500210us) raft_consensus.cc:1685] finished
0709 12:26:46.024181 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.024412 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3530,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"spinlock_wait_cycles":12516096,"thread_start_us":361585,"threads_started":1,"wal-append.queue_time_us":758008}
I20260709 12:26:46.012590 11490 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:46.028169  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 682) took 3056 ms. Trace:
I20260709 12:26:46.028715  7729 rpcz_store.cc:276] 0709 12:26:42.971652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.420375 (+448723us) service_pool.cc:224] Handling call
0709 12:26:44.698232 (+1277857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.657182 (+958950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.027002 (+369820us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:46.027910 (+   908us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001c48d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:46.028066 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5015291520}
I20260709 12:26:46.030212 11139 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:46.030383  7729 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:46.007962 11347 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.658s	user 0.015s	sys 0.000s
I20260709 12:26:46.022285  8405 pending_rounds.cc:85] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 7
I20260709 12:26:46.032184  8405 pending_rounds.cc:107] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:26:46.032591  8405 raft_consensus.cc:2889] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:46.027868 11490 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:46.013934 11483 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:46.031034 11272 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:46.014381  7812 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:45.936980 11414 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:26:46.076063  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 338908us
W20260709 12:26:46.085813  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 245781us
I20260709 12:26:46.095026  7657 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:45.966101 11316 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:46.075145 11167 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:46.121869  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:46.142146  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.117988  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 183145us
I20260709 12:26:46.119431  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.064875 11201 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:46.080070 11428 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:46.121445 11167 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:46.208382  7653 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:46.209265 11167 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.209918 11401 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.203861  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.215798  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:46.121137 11430 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.030s	user 0.010s	sys 0.005s
I20260709 12:26:46.082100 11490 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:46.274566  7659 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:46.117126 10796 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.005s
I20260709 12:26:46.221671  7657 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:46.222426  7658 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:46.226347 11441 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.221076 11428 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.243420 11255 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.204452  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.211769 11167 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:46.257190 11316 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:46.190508  7661 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:26:46.358657  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 439596us
I20260709 12:26:46.211788  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.211809 11401 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:46.278175  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:46.280098 11497 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:46.286419 11461 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.308682 11057 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:46.313298 11260 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.316608 11351 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.317026  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.319124  8408 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:46.374926  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.320616  8413 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:46.376626  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:46.344936  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 496851us
I20260709 12:26:46.349992  7653 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:46.356638  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:46.356570  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:46.362999  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.365165  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.391762  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.394004  7736 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:26:46.362294 11314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.025s	sys 0.041s
W20260709 12:26:46.372442  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.395430  8412 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:46.395849  7656 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:46.395877  7661 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:26:46.396481  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.024s
W20260709 12:26:46.400847  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.011s	sys 0.024s
W20260709 12:26:46.398299  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.369021 10979 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.003s	sys 0.008s
I20260709 12:26:46.402755  7660 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:46.376513 11072 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.015s	sys 0.015s
W20260709 12:26:46.396747  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.014s	sys 0.021s
W20260709 12:26:46.376688 11472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.009s	sys 0.022s
W20260709 12:26:46.377640 11393 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.004s
I20260709 12:26:46.430207  7812 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:26:46.431869  7656 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:46.432241  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.433955  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.014s	sys 0.021s
W20260709 12:26:46.447284  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.352536 11441 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.440554  7811 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:26:46.426823 10979 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.004s	sys 0.009s
I20260709 12:26:46.357934 11255 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.456557  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.368736 11316 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:26:46.460192  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.461413  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:46.433655 11072 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.016s
I20260709 12:26:46.469709  7803 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:46.470484 11268 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:46.471107  7805 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:46.444955 11472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.023s
I20260709 12:26:46.473726  7656 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:46.474928  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.431679 11168 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:46.476504  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.461438  7809 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:46.466302  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:46.476631 11168 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.455662  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.400104 11351 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.363579 11497 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:46.481051  8414 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:46.490509  7659 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:46.481761  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 206226us
I20260709 12:26:46.506654  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:46.508958  7654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.015s	sys 0.029s
W20260709 12:26:46.510556  7654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.015s	sys 0.029s
I20260709 12:26:46.470571 11487 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.520084  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.526242  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:46.528142  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.524474  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:46.457581 11393 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:46.482868 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.046s
I20260709 12:26:46.492249 11466 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.536859  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.585117  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:46.687008  7658 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:46.545019  7660 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:46.557345  7656 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:26:46.628670  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 346444us
W20260709 12:26:46.565897  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.566674 11168 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.692464  7656 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:46.520828 11164 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:46.660516  7555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 180) took 21104 ms (client timeout 15000 ms). Trace:
W20260709 12:26:46.688931 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.014s	sys 0.052s
I20260709 12:26:46.713699  7655 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:46.717059  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 260233us
I20260709 12:26:46.720970  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.721165  7654 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:26:46.721079  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 173028us
I20260709 12:26:46.718293 11335 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:46.596463 11487 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.729482  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.692770 11466 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.694609 11347 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.728874  7658 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:46.745141  7660 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:46.728760  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.726099  7656 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.745141  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:46.744481  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46425.3us
W20260709 12:26:46.750439  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 226104us
I20260709 12:26:46.621398 11428 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:46.754807  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.722064 11280 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:46.760370  7659 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:46.760986  7656 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:26:46.763113  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.729403 10979 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.755744 11520 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:46.771027 11507 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:46.772662  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2242) took 1268 ms. Trace:
W20260709 12:26:46.772775 11515 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:46.773115  7730 rpcz_store.cc:276] 0709 12:26:45.504429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.504598 (+   169us) service_pool.cc:224] Handling call
0709 12:26:45.526404 (+ 21806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.871020 (+344616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.871060 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.922116 (+ 51056us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.400864 (+478748us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:46.400909 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.400992 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.772392 (+371400us) raft_consensus.cc:1685] finished
0709 12:26:46.772414 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.772576 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31280,"raft.queue_time_us":18310,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":6866,"thread_start_us":741616,"threads_started":2,"wal-append.queue_time_us":656884}
I20260709 12:26:46.751976 11347 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.769017 11512 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:46.776825 11512 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.777645  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:46.779398  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:46.779904  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.780153  7737 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:46.780895  7658 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:46.768396 11514 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:26:46.782181 11280 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.782764 11280 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.755956 11335 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.785749  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:46.768736 11506 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:26:46.776330 10978 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:26:46.788367 11506 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:46.788836 11506 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:46.789886 11506 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:46.783183  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.796415  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:46.794306  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:46.801848  7732 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.850s	user 0.013s	sys 0.000s
I20260709 12:26:46.802385  7732 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:46.802938  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2250) took 1143 ms. Trace:
I20260709 12:26:46.760739 11441 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:46.803494  7732 rpcz_store.cc:276] 0709 12:26:45.660021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.921765 (+261744us) service_pool.cc:224] Handling call
0709 12:26:46.802869 (+881104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77312}
I20260709 12:26:46.809424  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.769294 11520 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.796579 11351 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.822067  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:46.822602  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.817308  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.823289 11268 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:46.816289 11421 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.824002 11268 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:46.824139 11347 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.824215 11421 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:46.830399 11421 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:46.796936 11280 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.824594 11268 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:46.807782 10796 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.818718  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.836802  7659 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:46.842015  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:46.817668 11441 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:46.798432 10979 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:46.845245  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 88128.4us
I20260709 12:26:46.851054 10979 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:46.817425 11520 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:46.860097  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:46.866366  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:46.841369 10796 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:46.873804 11529 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:46.843341 11506 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:46.916514  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.917480  7812 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:46.711822  7555 rpcz_store.cc:269] 0709 12:26:25.278526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.279695 (+  1169us) service_pool.cc:224] Handling call
0709 12:26:46.360217 (+21080522us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:42.447535 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:42.448183 (+   648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:42.448220 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:26:42.448241 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:42.449154 (+   913us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:42.449224 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:42.449255 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:26:42.449888 (+   633us) tablet.cc:701] Row locks acquired
0709 12:26:42.449908 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:42.450082 (+   174us) write_op.cc:270] Start()
0709 12:26:42.450202 (+   120us) write_op.cc:276] Timestamp: P: 1783600002450025 usec, L: 0
0709 12:26:42.450227 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:42.451045 (+   818us) log.cc:844] Serialized 2814 byte log entry
0709 12:26:42.618888 (+167843us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:26:43.534547 (+915659us) op_driver.cc:464] REPLICATION: finished
0709 12:26:43.789680 (+255133us) write_op.cc:301] APPLY: starting
0709 12:26:43.789800 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:43.790579 (+   779us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:43.790602 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:43.800247 (+  9645us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:43.839323 (+ 39076us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:26:43.843992 (+  4669us) write_op.cc:312] APPLY: finished
0709 12:26:43.877940 (+ 33948us) log.cc:844] Serialized 59 byte log entry
0709 12:26:43.908147 (+ 30207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:43.924086 (+ 15939us) write_op.cc:454] Released schema lock
0709 12:26:43.959300 (+ 35214us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:43.999665 (+ 40365us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:44.001699 (+  2034us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1645406,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":247973,"apply.run_cpu_time_us":4757,"apply.run_wall_time_us":191497,"num_ops":7,"prepare.queue_time_us":680876,"prepare.run_cpu_time_us":26471,"prepare.run_wall_time_us":417560,"raft.queue_time_us":298251,"raft.run_cpu_time_us":15354,"raft.run_wall_time_us":304070,"replication_time_us":1067073,"spinlock_wait_cycles":7935744,"thread_start_us":1270726,"threads_started":4,"wal-append.queue_time_us":466249,"wal-append.run_cpu_time_us":132758,"wal-append.run_wall_time_us":2153985}]]}
I20260709 12:26:46.840085 11512 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.847411 11441 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.920324 10796 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.920521 11441 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:46.922425  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.868512 11497 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:46.923867 11526 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.924499 11526 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.925804 11393 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:46.927466 11167 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.003s
I20260709 12:26:46.928906 11430 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:46.929389 11430 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:46.929848  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:46.930312 11430 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.952412 11430 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:46.952800  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.953572  7660 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:46.954586  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.960487 11466 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:46.963364 11139 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:46.963845 11139 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:46.965451 11139 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:46.966598 11139 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:46.909988  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:46.872424 11510 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:46.857944 11280 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.911756  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:46.914345  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:46.931445  7657 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:46.931761  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:46.999089  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84857.6us
I20260709 12:26:46.932660  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:46.905877  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83885.7us
I20260709 12:26:46.963752 11369 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:46.857033 11428 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:46.972705  7659 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:47.007094  7653 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:46.979671  7660 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
W20260709 12:26:46.981617  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.996493  7656 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:46.996912  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.005965 11428 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:47.026347  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.023133  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2284) took 1434 ms (client timeout 1650 ms). Trace:
I20260709 12:26:46.962425 11249 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:46.997396 11510 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:47.041271  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.053784  8414 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:47.056646  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:47.059090 11504 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.059289 11255 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.059849  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.056264  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.003212 11369 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:47.060256  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:47.059696 11504 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.064998  8409 raft_consensus.cc:2410] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:47.060379  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.049399 11164 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.061902  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:47.070976  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 3.340s (SENT)
I20260709 12:26:47.020712 11516 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.040697 11428 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:47.064540 11528 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:47.022451 11502 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:47.042220 11487 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:47.069968  8411 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:47.056531 11510 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.008601 11280 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:47.071249  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70847.6us
I20260709 12:26:47.060664 11466 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:47.106460 11369 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:47.108414 11369 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.108741  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.109844 11335 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:47.110304  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:47.074399  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 208697us
W20260709 12:26:47.081418  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 164703us
W20260709 12:26:47.070231  7655 rpcz_store.cc:269] 0709 12:26:45.572132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.572308 (+   176us) service_pool.cc:224] Handling call
0709 12:26:47.006467 (+1434159us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54004353d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:47.006687 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548269312}
I20260709 12:26:47.087834  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.070775 11504 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.099498  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:47.099668  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.118636  7654 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:47.101421  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.104990 11526 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.113905  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:47.113965  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:47.122491  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.084069 11528 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.123843  7627 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.616s (SENT)
I20260709 12:26:47.127650  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:47.128589  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.057405 11393 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 12:26:47.124311  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.125484  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.131332 10979 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:47.133948 10979 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:47.134385 10979 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:47.134593  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.111733  8405 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:26:47.130594  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.019156 11268 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.142835  8411 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:26:47.082777 11164 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.153332  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:47.154002  7557 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 4, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.155493  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2247) took 1519 ms. Trace:
I20260709 12:26:47.155643 11513 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:47.153201  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 621380us
W20260709 12:26:47.157150 11519 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:47.136035 11421 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:47.163725  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.095654 11520 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.176563  7737 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:47.131016 11528 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:47.128324 11335 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.171187  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.156013 11255 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:47.175526 11272 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:47.181104  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.202670  7732 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 5: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:47.088604 11516 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:47.182497  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69199.9us
I20260709 12:26:47.184787  7730 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:26:47.094384 11502 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.163148  7735 rpcz_store.cc:276] 0709 12:26:45.635693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.635911 (+   218us) service_pool.cc:224] Handling call
0709 12:26:45.647846 (+ 11935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.691064 (+ 43218us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:46.288620 (+597556us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b180004b700. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:46.288794 (+   174us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.155094 (+866300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.155150 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:47.155198 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.155296 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.155436 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":566570,"prepare.queue_time_us":209893,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":533202,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":7431424,"thread_start_us":536111,"threads_started":2,"wal-append.queue_time_us":432981}
I20260709 12:26:47.172573 11487 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:47.172392 11421 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.168490 11526 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:47.185557 11335 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:47.191404 11255 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:47.201944 11466 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:47.265030  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.265651  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.168926 11366 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 12:26:47.266376  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88442us
I20260709 12:26:47.237991 11421 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:47.202797 11072 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 12:26:47.278059  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 144898us
I20260709 12:26:47.242846 11272 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:47.257711 11335 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.266078 11255 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:47.238202 11487 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:47.283560  7557 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.295028 11164 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:47.297116  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.264027 11321 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:26:47.304291  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 643) took 1269 ms. Trace:
I20260709 12:26:47.275740 11472 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 12:26:47.311300 11531 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:26:47.312189 11534 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:26:47.281713 11414 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:26:47.315111 11461 raft_consensus.cc:2764] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:47.317214  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:47.323136  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:47.324931  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:47.326543  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:47.308056  8415 rpcz_store.cc:276] 0709 12:26:46.020351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.056915 (+ 36564us) service_pool.cc:224] Handling call
0709 12:26:46.079509 (+ 22594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.098095 (+ 18586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.098150 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.220285 (+122135us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.532329 (+312044us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:46.532397 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.532496 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.289578 (+757082us) raft_consensus.cc:1685] finished
0709 12:26:47.289601 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.289780 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5882,"raft.queue_time_us":35247,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":95918,"thread_start_us":249256,"threads_started":1,"wal-append.queue_time_us":759423}
I20260709 12:26:47.328233 11072 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:47.328890 11072 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:47.329918 11072 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.332623 11520 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:47.330480  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.334514 11516 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.336445  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:47.337453 11461 raft_consensus.cc:2764] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:26:47.340102  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.340301  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:47.350711  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.355428  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 173302us
I20260709 12:26:47.288515 11280 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:47.360577  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123482us
I20260709 12:26:47.368954 11441 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.370447  8412 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:47.303303 11272 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.385869  8405 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:47.361179 11461 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:47.387730  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48146.6us
I20260709 12:26:47.388401  7557 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.431877  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.432688  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.433317  7659 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:47.433400  7656 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:47.436995  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.445873  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.449357  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.450233  7659 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:47.451365  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 166793us
I20260709 12:26:47.453693  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:47.456840  8411 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:47.395205 11441 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.459410  7557 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 6, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.463153  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.464249  7657 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:47.386159 11541 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:26:47.467557  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.467909  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:47.467845  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.470839  7660 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.470281  7732 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:47.474206  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.387553 10979 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:47.479329  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.503350  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.508742  7557 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.471676 11057 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:26:47.517664  7557 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:47.521492  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 92102.6us
I20260709 12:26:47.469390 11002 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:26:47.523209  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 128026us
I20260709 12:26:47.515611 11520 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:47.517100 11314 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:47.540406  7732 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:47.552872  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:47.533494 11401 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.156s	user 0.017s	sys 0.000s
I20260709 12:26:47.553784  7735 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9. Re-sending same reply.
I20260709 12:26:47.555243 11550 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:47.555717 11539 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:26:47.556303 10979 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:47.554451 11351 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:47.558203 11546 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:26:47.558889 11428 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.567881  7557 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 4, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.570886  7655 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 4.
W20260709 12:26:47.576700  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.537616 11487 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:47.609853 11428 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.630350  8413 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:47.573055 11057 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:26:47.670753  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 142156us
I20260709 12:26:47.672643  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.673875 11497 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:47.674387 11497 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:47.674832 11497 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:47.695680  7557 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:47.696712  7777 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 4.217s (SENT)
I20260709 12:26:47.697345  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:47.697922  7557 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.579427 11550 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.701118 11556 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:47.702962 11347 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:47.730501 11347 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:47.730152  7557 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 3, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 3 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.731590 11347 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.706115 11268 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:47.732744 11537 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:26:47.733434 11504 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.001s
W20260709 12:26:47.736068  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.598349 11351 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.736913 11255 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.737808 11550 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:47.730937 11139 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.764s	user 0.013s	sys 0.004s
I20260709 12:26:47.738626 11351 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:47.733888 11504 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:47.750476 11524 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.723s	user 0.004s	sys 0.001s
I20260709 12:26:47.752660 11504 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:47.754426 11528 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 12:26:47.726835  7662 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:47.742435 11347 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:47.739454  7557 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 4, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.726177  7660 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:47.763001 11510 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.648s	user 0.004s	sys 0.008s
I20260709 12:26:47.768347 11472 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:47.674399  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:47.747134  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.026s (SENT) [suppressed 27 similar messages]
W20260709 12:26:47.768836  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.835s (SENT)
I20260709 12:26:47.772584  7557 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 6, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:47.779686  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 276494us
W20260709 12:26:47.778512  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104207us
W20260709 12:26:47.782351  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.788861  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.789774  7660 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:47.711120 11421 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.795584  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.783737 11351 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:47.801115  7557 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 4 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.816632  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.823092  7660 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:47.770185 11510 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:47.820345  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55483.2us
I20260709 12:26:47.799508  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.815995  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:47.817524  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.755165 11528 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:47.792515 11255 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:47.826174  7657 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:47.832952 11497 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:47.837174  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48966.9us
I20260709 12:26:47.840569  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:47.839515 11421 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:47.843314  7557 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 6, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 6 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:47.845181 11539 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:47.860064  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:47.862694  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:47.867831  7655 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:26:47.871292  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.877135  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:47.877830  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.878103  7657 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:47.886004  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:47.862154 11255 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:47.888404 11497 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:47.907980  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 649) took 1797 ms (client timeout 1533 ms). Trace:
W20260709 12:26:47.908440  7662 rpcz_store.cc:269] 0709 12:26:46.110789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.110910 (+   121us) service_pool.cc:224] Handling call
0709 12:26:47.907732 (+1796822us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000f4954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:47.907891 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3126032640}
I20260709 12:26:47.911440  7661 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:47.888069 11255 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:47.912379 11255 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:47.891882  8411 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:47.915757  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.918105  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:47.891278  8405 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:47.893821  7660 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:47.903097  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 955) took 1064 ms. Trace:
I20260709 12:26:47.908478  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:47.924173  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:47.924947  8412 rpcz_store.cc:276] 0709 12:26:46.838981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.839164 (+   183us) service_pool.cc:224] Handling call
0709 12:26:47.902870 (+1063706us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:47.903022 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213298432}
I20260709 12:26:47.927830  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:47.866528 11072 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:47.989132  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 661) took 1621 ms. Trace:
W20260709 12:26:47.947955  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 159461us
I20260709 12:26:47.948912 11543 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.949447 11571 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 12:26:47.949851 11502 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:47.950870 11557 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.160s	user 0.001s	sys 0.003s
I20260709 12:26:47.951454 11539 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:47.951862  7735 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:26:48.029318  7735 pending_rounds.cc:85] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:26:48.029579  7735 pending_rounds.cc:107] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:48.029836  7735 raft_consensus.cc:2889] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:47.905465 11558 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:47.886479 11559 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.955690 11472 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:47.948948 11516 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:47.957335  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.891558  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:47.893388 11560 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:47.966063  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:48.073665  8415 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:47.967494 11561 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:26:47.905988 11167 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:47.977149  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:47.978314  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:48.090452  7662 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:47.984150 11510 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:47.985512  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.989777  7661 rpcz_store.cc:276] 0709 12:26:46.367442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.409693 (+ 42251us) service_pool.cc:224] Handling call
0709 12:26:47.910843 (+1501150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.912182 (+  1339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.915050 (+  2868us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54000f4950. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:47.989027 (+ 73977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3369247616}
I20260709 12:26:48.002526  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:48.009943  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.025484  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:48.027133  7728 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:48.027844  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:48.030710  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.041054  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:48.041954  7653 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:26:48.069057  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61696.2us
W20260709 12:26:48.020243 11335 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.007s
W20260709 12:26:48.084493  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.008823 11072 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:47.979815 11512 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.089893  7660 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:48.098341  8413 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:48.099915  8411 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:48.101280  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:48.067607 11497 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.042754 11559 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.061842 11472 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:48.120414  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:48.145042  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 43884.2us
I20260709 12:26:48.147491  7654 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:26:48.060818 11461 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:26:48.095206 11516 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:48.088934 11167 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:48.013278 11543 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.094951 11539 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:48.217542  7661 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:48.155031  7653 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:48.176599  7662 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:48.143436 11559 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:48.148072 11558 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:48.193028  7660 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:48.212347  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:48.143760 11497 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.155372 11249 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:26:48.183045 11523 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.122524 11512 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.108377 11335 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.151392 11510 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:48.529953  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.700049  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.248067  7654 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:26:48.168093 11201 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.260980 11500 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:48.610244 11461 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.195138 11519 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:26:48.628162 11547 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:48.717730  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 505628us
W20260709 12:26:48.467777  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.523569  7657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.015s
I20260709 12:26:48.720319  8412 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:48.184859 11272 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.271880  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 148979us
W20260709 12:26:48.781574  7657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.022s	sys 0.021s
I20260709 12:26:48.737212  7654 pending_rounds.cc:85] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 3
I20260709 12:26:48.686973 10978 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:48.643813 11528 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:48.753211  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.005s	sys 0.018s
W20260709 12:26:48.823632  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.006s	sys 0.018s
W20260709 12:26:48.244168 11567 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:26:48.254077 11540 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.062s	user 0.000s	sys 0.004s
I20260709 12:26:48.772538  8407 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.778179  7653 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.261080 11511 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.779121  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.779879  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.008s	sys 0.030s
W20260709 12:26:48.837112  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.008s	sys 0.030s
I20260709 12:26:48.782284  7654 pending_rounds.cc:107] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:26:48.838831  7654 raft_consensus.cc:2889] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:48.602181 11550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.017s	sys 0.039s
I20260709 12:26:48.771337 11500 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:48.748796 11523 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:48.183684 11556 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.582s
I20260709 12:26:48.246685 11544 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:48.741101 11510 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:48.742875  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.010s
I20260709 12:26:48.843611  7775 leader_election.cc:400] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:48.235255 11255 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.705405  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:48.853474  7775 leader_election.cc:403] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:48.854007  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.008s	sys 0.010s
I20260709 12:26:48.856125  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:48.253867 11564 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.745s
W20260709 12:26:48.842060 11550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.019s	sys 0.039s
W20260709 12:26:48.866014  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.573s (SENT)
I20260709 12:26:48.879458  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:48.884446 11573 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.639s	user 0.000s	sys 0.010s
W20260709 12:26:48.886461 11575 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.620s	user 0.002s	sys 0.000s
I20260709 12:26:48.887284 11528 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:48.770509 11201 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.664319 11516 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.614225 11167 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.859761 11500 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:48.784713 11461 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:48.900308  7654 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:48.893478  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:48.261363 11072 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:48.864521 11510 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:48.897161  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:48.239799 11539 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.908200  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 963) took 1021 ms. Trace:
I20260709 12:26:48.820515 11547 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.832157 10978 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.911160  7662 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:48.843333 11272 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.912674  8414 rpcz_store.cc:276] 0709 12:26:47.886342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.886685 (+   343us) service_pool.cc:224] Handling call
0709 12:26:47.887075 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.903495 (+ 16420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.903533 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.904134 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.099063 (+194929us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:48.099199 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.099487 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.885837 (+786350us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:48.901590 (+ 15753us) raft_consensus.cc:1685] finished
0709 12:26:48.901821 (+   231us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.908090 (+  6269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11427,"raft.queue_time_us":18188,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":1092881152,"thread_start_us":194356,"threads_started":1,"wal-append.queue_time_us":361427}
I20260709 12:26:48.913527 11002 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.927173  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:48.915375 11351 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.091s	user 0.009s	sys 0.004s
I20260709 12:26:48.927987  7662 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:48.872859 11544 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:48.932260  7659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.024s	sys 0.026s
W20260709 12:26:48.934841  7659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.025s	sys 0.027s
I20260709 12:26:48.851594 11511 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.918128 11167 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:48.923502 11414 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:26:48.922816 11366 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:48.914712 11580 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:48.923509 11543 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:48.953642  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:48.963685  7734 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:48.964125  7734 pending_rounds.cc:85] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:26:48.936565 11351 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:48.964396  7734 pending_rounds.cc:107] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:26:48.968627  7662 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:48.926739 11002 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:48.995744  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:48.996794 11512 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:48.998479 11335 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:49.023550  7653 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:48.999727 11201 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:49.000110 11577 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:26:49.001214 11578 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:26:49.004966  7660 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:49.013119 11255 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:49.013441 11523 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:48.975324  7734 raft_consensus.cc:2889] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.016876  7654 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.019603  7737 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:49.042083 11002 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:49.043529 11201 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.045194  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.046654  7653 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:49.047919 11576 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:26:49.016062 11585 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:26:49.061717  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.952s (SENT)
W20260709 12:26:49.062338  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 272855us
I20260709 12:26:49.092864  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.066392  7662 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
W20260709 12:26:49.072670  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 290454us
I20260709 12:26:49.077556  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:49.017340  8413 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.868s	user 0.015s	sys 0.003s
I20260709 12:26:49.083483  7655 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.051870 11547 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:48.966596 11580 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.135162  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 685) took 1148 ms. Trace:
I20260709 12:26:49.149809  8413 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:26:49.150239  7659 rpcz_store.cc:276] 0709 12:26:47.986921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.987110 (+   189us) service_pool.cc:224] Handling call
0709 12:26:47.987628 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.987953 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:48.935599 (+947646us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.134773 (+199174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.134838 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:49.134885 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.134954 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.135088 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13319,"prepare.queue_time_us":979667,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":13431,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":144,"thread_start_us":1132447,"threads_started":2}
W20260709 12:26:49.144611  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 296859us
I20260709 12:26:49.168510  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 652) took 1738 ms. Trace:
W20260709 12:26:49.114094  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 248104us
I20260709 12:26:49.150627  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 967) took 1153 ms. Trace:
W20260709 12:26:49.165774  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.120463  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96489us
I20260709 12:26:49.121146  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.263166  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:49.162719 11584 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:26:49.121462  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.255697  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.256369  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.086135 11268 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.273038  8415 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:49.244500  7660 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:49.264665  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.274928  7655 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:49.153313  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 259816us
I20260709 12:26:49.245863  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.091751 10978 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:49.268623  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.016s (SENT)
W20260709 12:26:49.268745  7777 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.717s (SENT)
I20260709 12:26:49.270107  7653 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.270771  7659 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.113756 11564 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.151548 11506 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:49.232242 11603 raft_consensus.cc:476] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 12:26:49.154698 11592 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.126253 11490 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.176474 11502 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:49.300813  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.260272  8413 rpcz_store.cc:276] 0709 12:26:47.997590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.997912 (+   322us) service_pool.cc:224] Handling call
0709 12:26:49.150559 (+1152647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:49.162886  8406 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:49.257203 11489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.010s	sys 0.009s
I20260709 12:26:49.282029  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:49.284395  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.285950  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:49.151463 11586 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:49.301504  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.219846 11461 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.324678 11586 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.292225  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.325398 11461 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:49.325944 11461 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:49.282984  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:49.285535 11511 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:49.295506  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49779.5us
I20260709 12:26:49.090837 11201 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.287309 11506 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:49.312871  7655 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:49.325163  7662 raft_consensus.cc:2376] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7. Re-sending same reply.
W20260709 12:26:49.294276  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.326826  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.249697 11272 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.327601  7659 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.325965  7660 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:26:49.245546  8407 rpcz_store.cc:276] 0709 12:26:47.430298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.430496 (+   198us) service_pool.cc:224] Handling call
0709 12:26:47.430965 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.835343 (+1404378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.168101 (+332758us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:49.168417 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3084894592}
W20260709 12:26:49.328068  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.266417 11556 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:26:49.344132  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61313.6us
I20260709 12:26:49.094671 11585 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.326445 11461 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:49.333912  7653 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
W20260709 12:26:49.328899  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91633us
W20260709 12:26:49.333529  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45573.8us
W20260709 12:26:49.328110 11489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.010s	sys 0.009s
I20260709 12:26:49.284530 11268 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.298276 11502 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:49.358215  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.359134  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:49.359843  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.356837  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.359351  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.357496  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:49.349121 11559 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.000s
I20260709 12:26:49.370566  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.371315  7809 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:49.294142 11544 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:49.371434  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109861us
W20260709 12:26:49.373262  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.374660  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 105993us
W20260709 12:26:49.378739  7777 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.106s (SENT)
I20260709 12:26:49.379585  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.347692 11497 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:49.382474  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:49.398483  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.400530  7660 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:49.400632 11539 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:49.401873  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.402119  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.402369  7653 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:49.297863 11592 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.402755  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.404124  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:49.392151  7655 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.410913  7662 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.341822 11608 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:49.404893  7660 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.314567 11595 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:26:49.407604  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:49.305015 11490 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.420182  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.297515 11603 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:26:49.404439  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:49.354676 11606 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:49.354449 11522 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:49.415422  7777 leader_election.cc:336] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.542s (SENT)
I20260709 12:26:49.431597  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:49.435302 11602 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:26:49.436523 11268 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:49.437387  7658 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:26:49.401574 11539 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:49.402711 11272 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:49.423434  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:49.423966  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.430902  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.438663  7658 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:49.474855  7662 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.416942  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.476146  7659 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 6: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:49.435209  7660 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:26:49.440565  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.479622  7655 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
W20260709 12:26:49.480659  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49835.2us
I20260709 12:26:49.478374  7659 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:49.481069  7660 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:49.450297  7811 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:49.370479 11559 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.378578 11594 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:26:49.486233  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2283) took 1525 ms. Trace:
W20260709 12:26:49.484409  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 128967us
I20260709 12:26:49.484598  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:49.487591  7658 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:49.487883  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.488075  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.438331 11547 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:49.438978 11580 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:49.498464  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.488600  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:49.500378 11472 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.344s	user 0.009s	sys 0.006s
I20260709 12:26:49.479051 11260 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:49.486654  7731 rpcz_store.cc:276] 0709 12:26:47.960550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.993674 (+ 33124us) service_pool.cc:224] Handling call
0709 12:26:47.994041 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:48.006607 (+ 12566us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:48.094637 (+ 88030us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.824167 (+729530us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:48.860664 (+ 36497us) log.cc:844] Serialized 52 byte log entry
0709 12:26:48.860867 (+   203us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:48.868934 (+  8067us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.869247 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.869360 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.485983 (+616623us) raft_consensus.cc:1685] finished
0709 12:26:49.486004 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.486160 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44802,"prepare.queue_time_us":91556,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":21127,"raft.queue_time_us":183282,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":82030,"thread_start_us":656448,"threads_started":1,"wal-append.queue_time_us":791217}
I20260709 12:26:49.522342  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.525218  8408 raft_consensus.cc:2368] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:26:49.516386  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.516855  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.514705  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:49.518267  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.512748  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.533310  7658 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:26:49.473807 11586 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:49.501538 11574 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.249s	user 0.002s	sys 0.006s
I20260709 12:26:49.523195  7731 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:49.541841  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.538446  7730 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:49.542514  7653 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:49.522030  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.533144  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.543929  7660 raft_consensus.cc:2376] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10. Re-sending same reply.
I20260709 12:26:49.550093  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.422022 11506 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.554836  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:49.517812 11596 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:49.559597  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.560456  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:49.563128  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.564162  7812 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:49.427578 11502 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.518193 11489 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:26:49.555733 11497 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:49.656920  7728 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:49.527731 11472 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.558235 11596 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:49.667852  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 244495us
I20260709 12:26:49.668758  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:49.616127 11586 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:49.674126  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.675730  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:49.625377 11618 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:49.676800  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.657258 11590 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.680246  7732 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:49.659044 11497 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:49.681377  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 128736us
W20260709 12:26:49.687484  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.666047 11502 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:49.686179  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.693509  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.697156  7812 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:49.682097  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.664383 11621 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:49.675988 11510 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.012s
I20260709 12:26:49.705909  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.706624  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:49.734428  7653 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:49.710402  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:49.711936 11600 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:26:49.712260  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.739494  7730 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:49.713346 10978 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:49.713531  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2284) took 1752 ms. Trace:
I20260709 12:26:49.713855 11621 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:49.715755 11592 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:49.745935  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.716528 11490 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.742456  7737 rpcz_store.cc:276] 0709 12:26:47.960861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.994564 (+ 33703us) service_pool.cc:224] Handling call
0709 12:26:47.994834 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.068305 (+1073471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.712961 (+644656us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002490d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:49.713317 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356084096}
I20260709 12:26:49.717289 11472 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.718472  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 964) took 1829 ms. Trace:
I20260709 12:26:49.753528  8408 rpcz_store.cc:276] 0709 12:26:47.889254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.889580 (+   326us) service_pool.cc:224] Handling call
0709 12:26:49.718233 (+1828653us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54007a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:49.718395 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3362287872}
I20260709 12:26:49.719424 11550 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:49.719763 11604 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:49.720887 11572 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.207s	user 0.005s	sys 0.001s
W20260709 12:26:49.722100 11615 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:26:49.722728 11620 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:49.724257  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:49.715687  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:49.727702 11167 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.008s
I20260709 12:26:49.727906  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:49.729110 11598 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 12:26:49.777155  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71455.4us
I20260709 12:26:49.729982  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:49.762085 10978 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:49.780236 11556 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:49.780882 11556 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.781344 11556 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.781159 10978 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:49.731171  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.512s (SENT)
I20260709 12:26:49.783675  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:49.731447  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51503.2us
I20260709 12:26:49.744822 11622 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.670455 11596 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:49.672987 11201 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.737797  7809 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:26:49.743733  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50313.4us
I20260709 12:26:49.748669  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.700975 11607 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.682569 11594 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:49.700306 11497 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.728278 11522 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:49.638226 11539 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.682341 11618 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:49.803593 11618 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:49.768345  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.759569 11604 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.806106 11604 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.807752 11619 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.808358 11619 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.743039 11510 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.813356  7731 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:49.813810  7731 pending_rounds.cc:85] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:26:49.814071  7731 pending_rounds.cc:107] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:26:49.756834 11550 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:49.755352 11316 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:49.762310 11500 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.851s	user 0.015s	sys 0.000s
I20260709 12:26:49.790071  7812 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:49.837158  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:49.791538  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60405us
I20260709 12:26:49.679920 11586 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.746083 11621 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:49.795840  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:49.853772  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.814342  7731 raft_consensus.cc:2889] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.704846 11590 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.810923 11607 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:49.815716  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47486.4us
I20260709 12:26:49.845122  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.876981  7809 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:49.739542 11585 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:49.849506  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.624s (SENT)
W20260709 12:26:49.890519  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.639s (SENT)
I20260709 12:26:49.891115  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:49.860496  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.799147 11559 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.745965 11603 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 12:26:49.887043  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.810326 11522 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:49.889686  7654 raft_consensus.cc:2368] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8.
I20260709 12:26:49.839949 11624 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:49.815589 11594 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:49.895522  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:49.787719 11620 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:49.898715  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 350960us
I20260709 12:26:49.797557 11472 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:49.888362 11607 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:49.869846 11414 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.005s
I20260709 12:26:49.923923  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.914009  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2027) took 1664 ms. Trace:
I20260709 12:26:49.892814 11502 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:49.917225 11585 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:49.918145  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 106434us
I20260709 12:26:49.867306 11500 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.812234 11622 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.840515 11316 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:49.849087 11580 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:26:49.850713 11168 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:26:49.906392 11522 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:49.924690  7661 rpcz_store.cc:276] 0709 12:26:48.249865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.504951 (+255086us) service_pool.cc:224] Handling call
0709 12:26:48.505426 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.827083 (+321657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.827135 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.845805 (+ 18670us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.249945 (+404140us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:49.249997 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.250074 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.913678 (+663604us) raft_consensus.cc:1685] finished
0709 12:26:49.913713 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.913935 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5165,"raft.queue_time_us":455761,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":6162,"thread_start_us":641672,"threads_started":2,"wal-append.queue_time_us":465328}
I20260709 12:26:49.907476 11624 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.906967 11614 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:49.905691 11260 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:49.910974  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.939123  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:49.912956 11591 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:49.928014 11414 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6.
I20260709 12:26:49.934216 11603 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:49.934473 11625 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:49.958628  7735 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:49.901284 11272 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:49.941383 11316 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:49.939191 11516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.008s	sys 0.012s
I20260709 12:26:49.963759  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:49.940083 11559 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:49.939149 11072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.015s	sys 0.003s
I20260709 12:26:49.975243  7809 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:49.976606  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.954878 11624 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.954519 11592 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:49.984730 11626 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:49.985270 11586 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:49.985919  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:49.964471  8406 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:49.935935 11502 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:49.965704 11591 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:49.969251 11472 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:49.996837  7810 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:49.945930 11585 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:49.971359 11603 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:50.082136  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:49.988150 11516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.008s	sys 0.013s
I20260709 12:26:50.007045  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:50.007376  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69930.5us
W20260709 12:26:49.991503 11072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.016s	sys 0.003s
W20260709 12:26:50.014168  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123685us
I20260709 12:26:50.014992  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:49.990783 11559 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.099100 11559 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.101071 11632 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:50.101845 11632 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.075877  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:50.035876 11623 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:26:50.017148 11586 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:50.083406  7803 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:26:50.085686 11500 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.087517  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:50.114589  8413 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:50.088465 11590 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:49.994076 11594 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.091614 11640 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.094895  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.661s (SENT)
I20260709 12:26:50.081003 11510 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:50.119025  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.687s (SENT)
I20260709 12:26:50.076443 11472 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:50.094061 11631 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.120571  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:50.110559 11564 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.002s
I20260709 12:26:50.120931  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:50.109438 11547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.009s	sys 0.025s
I20260709 12:26:50.123657 11640 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.122712 11272 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:50.126626  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.940s (SENT)
I20260709 12:26:50.134649 11497 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.130929 11472 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:50.163692  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.123757 11625 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:50.164616  7811 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:50.165102 11625 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.169371 11544 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:50.169927 11544 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:50.170059 11605 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.836s	user 0.007s	sys 0.013s
I20260709 12:26:50.172060 11624 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.173681  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.178668 11351 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:50.179143 11351 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:50.180056 11351 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.180810  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.181092 11351 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:50.181349  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.136287 11631 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.146697  7803 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:50.175045  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.190932  8413 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:50.153616  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:50.153590 11547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.010s	sys 0.026s
I20260709 12:26:50.135875  7737 raft_consensus.cc:2376] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8. Re-sending same reply.
W20260709 12:26:50.197496  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70903.7us
W20260709 12:26:50.175041 11461 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.002s
I20260709 12:26:50.145821 11564 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:50.191816  7735 raft_consensus.cc:2368] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8.
I20260709 12:26:50.193936  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:50.197585  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.204792  7737 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.200588  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.139410  7810 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:50.177501 11472 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.207109 11510 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:50.207813  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.208048 11510 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.193440 11625 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.218809  7733 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:50.216948 11641 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:50.208810  8406 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:50.220481  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.195811 11547 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:50.226075  7803 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:50.228196  7735 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:50.170897 11544 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.231812  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 736) took 1483 ms. Trace:
I20260709 12:26:50.208657 11510 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.194625 11581 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:26:50.235176  7734 rpcz_store.cc:276] 0709 12:26:48.748144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.748279 (+   135us) service_pool.cc:224] Handling call
0709 12:26:48.782931 (+ 34652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.082057 (+299126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.231554 (+1149497us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:50.231752 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414715648}
I20260709 12:26:50.238636  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 683) took 2282 ms. Trace:
I20260709 12:26:50.199071 11260 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:50.239287  7656 rpcz_store.cc:276] 0709 12:26:47.956484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.958058 (+  1574us) service_pool.cc:224] Handling call
0709 12:26:47.958575 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.959129 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.959173 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.960020 (+   847us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.270676 (+310656us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:48.940239 (+669563us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.231713 (+1291474us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b540035faa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e9d733 00007fc001f22dbf 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c
0709 12:26:50.231764 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.231853 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.231905 (+    52us) raft_consensus.cc:1685] finished
0709 12:26:50.238269 (+  6364us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.238533 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":697289,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":125099,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":10902,"spinlock_wait_cycles":21276544,"thread_start_us":827608,"threads_started":3,"wal-append.queue_time_us":932351}
I20260709 12:26:50.245036  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:50.246495 11635 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:26:50.141839 11591 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.232396 11641 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:50.248020 11645 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:50.248786 11641 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.250267  7728 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:50.248306 11591 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.251817  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:50.252784  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:50.253985 11620 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:50.254518 11639 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.104s	user 0.001s	sys 0.001s
I20260709 12:26:50.256035 11544 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:50.256163  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:50.257594 11592 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.257673 11260 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:50.261075  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2040) took 1216 ms (client timeout 1542 ms). Trace:
W20260709 12:26:50.261610  7654 rpcz_store.cc:269] 0709 12:26:49.044964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.066276 (+ 21312us) service_pool.cc:224] Handling call
0709 12:26:50.260811 (+1194535us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54003980d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:50.260986 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796021504}
I20260709 12:26:50.263700  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2299) took 2202 ms. Trace:
I20260709 12:26:50.264170  7732 rpcz_store.cc:276] 0709 12:26:48.061333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.086661 (+ 25328us) service_pool.cc:224] Handling call
0709 12:26:48.087557 (+   896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.692971 (+1605414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.263383 (+570412us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54001c4dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:50.263629 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3662404864}
I20260709 12:26:50.265820  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.267684  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.268713  7732 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:50.266625  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
W20260709 12:26:50.239668 11636 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:26:50.210260 11461 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.232182 11608 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:50.233392 11585 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.283612  7803 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:50.258297 11260 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:50.230852 11624 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.145850 11603 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.288659  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.289403  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.289505  7736 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:50.292331  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:50.295954 11603 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:50.300832  7811 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:50.258153 11592 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.285689 11502 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:50.275434 11414 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:26:50.276908 11606 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 12:26:50.308254 11579 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.355s	user 0.004s	sys 0.003s
I20260709 12:26:50.310693  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2020) took 2426 ms. Trace:
W20260709 12:26:50.312480  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.311149  7657 rpcz_store.cc:276] 0709 12:26:47.883917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.884058 (+   141us) service_pool.cc:224] Handling call
0709 12:26:47.969838 (+ 85780us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:47.970462 (+   624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.970509 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:47.981181 (+ 10672us) log.cc:844] Serialized 52 byte log entry
0709 12:26:48.782264 (+801083us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:49.079578 (+297314us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.079862 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.079981 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.237153 (+1157172us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:26:50.310338 (+ 73185us) raft_consensus.cc:1685] finished
0709 12:26:50.310451 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.310618 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11222,"prepare.queue_time_us":185871,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":10339,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":16789,"spinlock_wait_cycles":17334144,"thread_start_us":1013432,"threads_started":2,"wal-append.queue_time_us":907246}
W20260709 12:26:50.319088  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.987s (SENT)
W20260709 12:26:50.320734  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.999s (SENT)
I20260709 12:26:50.321731  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.323529  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:50.325373  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.035s (SENT)
I20260709 12:26:50.325834  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:50.324954  7729 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:50.328557  7736 raft_consensus.cc:1217] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-6.7]   Dedup: 6.7->[]
I20260709 12:26:50.329416  7736 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:26:50.332860 11632 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.314366 11316 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.340384  7731 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:50.349051 11316 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:50.349375 11644 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:26:50.351450  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2056) took 1061 ms. Trace:
I20260709 12:26:50.351912  7662 rpcz_store.cc:276] 0709 12:26:49.290688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.475625 (+184937us) service_pool.cc:224] Handling call
0709 12:26:49.479167 (+  3542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.933974 (+454807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.934020 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.951370 (+ 17350us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.073157 (+121787us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:50.094898 (+ 21741us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.104838 (+  9940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.104947 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.351136 (+246189us) raft_consensus.cc:1685] finished
0709 12:26:50.351160 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.351372 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22547,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":80237,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":13383,"thread_start_us":535182,"threads_started":2,"wal-append.queue_time_us":225340}
I20260709 12:26:50.353426 11506 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:50.325513 11260 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.354830 11260 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:50.355357 11585 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:50.359381 11522 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.360054 11472 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.361197  7806 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:50.322049 11510 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.362530 11522 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.355479 11497 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.365002  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.371102  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.372020  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:50.373008  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.373270  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.374079  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:50.374315  8412 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:50.375694  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.375828  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.377739  8411 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:50.379104  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.381783  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.366065 11614 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:50.367133 11556 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.003s
W20260709 12:26:50.367350 11628 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:26:50.368393  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:50.372539  7808 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:50.374773  7655 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.397957  7737 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:50.377771  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.384590  7809 raft_consensus.cc:2376] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6. Re-sending same reply.
I20260709 12:26:50.384156  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:50.390328  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.403607  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.289054 11608 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.385902  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.404520  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:50.364840 11618 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.001s
I20260709 12:26:50.405257  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:50.421427  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.424181  7656 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
W20260709 12:26:50.405957  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.385807  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:50.425339  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.420289  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.414369  7655 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:50.433478  8408 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:50.435348  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.436666  7737 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:50.432399  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.441781  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.444450  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:50.404263 11487 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:50.385334 11461 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.391804 11631 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.383831 11550 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.471421  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.383141 11592 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.462867  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.483881  7737 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:50.305953 11547 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.430104 11510 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:50.472551 11607 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.540s	user 0.002s	sys 0.011s
I20260709 12:26:50.384035 11556 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.462256  7736 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:50.484287  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.482509  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:50.488446  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 44101.9us
I20260709 12:26:50.493978  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.494722  7775 leader_election.cc:400] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:26:50.495189  7775 leader_election.cc:403] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.432242 11614 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:50.496721  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.506243  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.506690  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.507114  7736 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:50.431879 11618 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.498234 11622 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.499855 11606 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:50.499794 11649 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:50.507808  7737 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:50.493999  7729 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8. Re-sending same reply.
I20260709 12:26:50.514621  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.514966  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.516093  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:50.518431  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.520031  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.520555  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.507963 11604 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.005s
I20260709 12:26:50.524597  7811 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:50.531401  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.534026  7732 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:50.515985  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97101.2us
I20260709 12:26:50.535266  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.507859 11640 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.540827  7733 raft_consensus.cc:2368] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10.
I20260709 12:26:50.516959 11550 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:50.517822 11614 consensus_queue.cc:1243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:50.492547 11510 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.547643  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.549110  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:50.553107  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:50.555922  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.596035  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.597182  7734 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:50.504793 11607 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.611357  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:50.612751  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:50.615836 11624 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:50.605919 11528 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:26:50.616552 11624 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.624855 11510 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.578442 11502 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.641153  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:50.644131  8412 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:50.534727 11604 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.632766 11624 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:50.637871 11606 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:50.671370 11621 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.767s	user 0.014s	sys 0.000s
W20260709 12:26:50.683014  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63927.1us
I20260709 12:26:50.683940  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:50.684885  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:50.685003 11581 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:50.685720 11581 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:50.685895 11652 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:26:50.686162 11581 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:50.675372 11002 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:50.700407 11618 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.702607 11604 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.685834 11547 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:50.708127 11559 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.004s
I20260709 12:26:50.710984 11607 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:50.690925 11621 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.713028 11547 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:50.717022 11606 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.721768  7731 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:50.725127 11596 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.913s	user 0.005s	sys 0.008s
I20260709 12:26:50.726078 11596 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.715006 11621 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.735180  8407 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:50.744433  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 654) took 2691 ms. Trace:
I20260709 12:26:50.712083 11559 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.745160  8415 rpcz_store.cc:276] 0709 12:26:48.053089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.143273 (+ 90184us) service_pool.cc:224] Handling call
0709 12:26:48.199530 (+ 56257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.273539 (+1074009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.744023 (+1470484us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:50.744333 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468178816}
I20260709 12:26:50.725158 11547 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:50.747985  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.749233  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.750900  8409 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:50.752081  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.705561 11002 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:50.754093  7733 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:50.754858  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:50.757795  7733 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:50.763118 11002 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:50.768069 11002 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:50.785291 11502 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:26:50.987346 11559 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:50.850317 11624 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.094971 11581 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.094069 11596 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:51.194198  8379 leader_election.cc:336] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.514s (SENT)
W20260709 12:26:51.336632  8379 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.589s (SENT)
I20260709 12:26:51.337271  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:51.139518 11581 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:26:51.369081  8379 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.618s (SENT)
W20260709 12:26:51.362136 11670 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
W20260709 12:26:51.498144 11669 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
I20260709 12:26:51.500187 11624 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:51.397717 11665 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:51.678273 11002 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.909s	user 0.005s	sys 0.008s
W20260709 12:26:51.717129  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.718309  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.725303 11461 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:51.726079 11461 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:51.727113 11461 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.728886 11461 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:51.743683 11619 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:51.749644 11487 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:51.750289 11487 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:51.750442  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.751106  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.753536  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:51.754272 11461 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:51.757073 11461 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:51.758936 11461 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.761857 11487 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:51.765002 11487 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:51.766139  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:51.767146  7731 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:51.769775  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.771000 11547 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.771749 11547 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.771908  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:51.775148 11547 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:26:51.785126  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:51.796689  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.787732  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:51.789023 11580 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:51.798002 11580 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:51.799523 11580 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:51.790625  7731 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:26:51.801173  7731 pending_rounds.cc:85] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:26:51.803364  7731 pending_rounds.cc:107] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:26:51.803926  7731 raft_consensus.cc:2889] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.806089  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:51.793906  7728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.303s	user 0.019s	sys 0.026s
W20260709 12:26:51.809218  7728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.021s	sys 0.027s
I20260709 12:26:51.806844 11547 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:51.810537 11547 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:51.802351 11580 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:51.809974 11648 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.477s	user 0.000s	sys 0.003s
I20260709 12:26:51.812985  7731 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:51.817879  7807 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:51.808372  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:51.811847 11547 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.820369 11632 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:51.821087 11632 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:51.821681 11632 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:51.835943  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.836570  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.837960 11272 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:51.848176  7655 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:51.850875  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:51.852000  7660 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:51.859890  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:51.861061  7733 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:51.862277  7731 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:26:51.868008  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:51.873153  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:51.876029  7660 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:51.878197 11624 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:51.880437 11624 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:51.882035 11624 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.883744 11581 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:51.894068 11632 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.899961 11632 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:51.896986 11581 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.903771 11581 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:51.949653  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2482) took 1568 ms. Trace:
I20260709 12:26:51.951001  7728 rpcz_store.cc:276] 0709 12:26:50.381646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.381788 (+   142us) service_pool.cc:224] Handling call
0709 12:26:50.382108 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.382348 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:50.445690 (+ 63342us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.857777 (+1412087us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.949217 (+ 91440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:51.949314 (+    97us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:51.949362 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.949434 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.949579 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":225916,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":125,"raft.queue_time_us":692,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":1206,"thread_start_us":1519804,"threads_started":4,"wal-append.queue_time_us":222436,"wal-append.run_cpu_time_us":1513,"wal-append.run_wall_time_us":1001567}
I20260709 12:26:51.954088  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:51.956003  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:51.957031  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:51.958119  7660 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7. Re-sending same reply.
I20260709 12:26:51.959973  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:51.960963  7728 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:26:51.961810  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:51.962620  7728 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:51.964010  7728 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:51.964713  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:51.965754 11624 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:51.968490  7654 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:51.969507 11581 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:51.974671  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:51.976060  7654 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:51.990417  7808 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:51.993916  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 604) took 1635 ms. Trace:
I20260709 12:26:51.994346  7808 rpcz_store.cc:276] 0709 12:26:50.358166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.385316 (+ 27150us) service_pool.cc:224] Handling call
0709 12:26:51.993603 (+1608287us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:51.993790 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3596379520}
I20260709 12:26:52.005019  7654 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:26:52.007578  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.008103  7655 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 12:26:52.008234  7653 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:52.012475  7655 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:52.013558  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.014086  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.014665  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.986997  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:52.016119  7731 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:52.024592  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.025422  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.027376  7808 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:26:52.033057  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.034404  7653 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:52.050469  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.052306 11692 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.057492  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:52.062094  7812 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:52.062260  7655 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:52.071966  7731 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:52.074084  7653 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:52.083896  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:52.027706 11619 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.086791  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.089453  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:52.090294  7812 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
W20260709 12:26:52.101604  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.103093  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:52.103358  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.105484  7655 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:52.103621 11688 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:52.058995 11268 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.090482  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:52.113742  7808 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.119838  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.124356  7655 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:52.102751 11619 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:52.101619  7653 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:52.126091  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:52.112524 11461 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:52.133898  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.088335 11692 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:52.124661 11688 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:52.176137  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:52.179970  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.117823 11268 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:52.248862  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.248682  7653 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:52.257812  7655 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.258126  7734 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:52.260848  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:52.261811  7655 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.263353  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.264056  7655 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:52.259642  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2512) took 1665 ms. Trace:
I20260709 12:26:52.259581  7803 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.265415  7808 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:52.266872  7734 rpcz_store.cc:276] 0709 12:26:50.594419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.598106 (+  3687us) service_pool.cc:224] Handling call
0709 12:26:52.259520 (+1661414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.158179 11461 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:52.275005  7655 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:52.276768  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.277598  7655 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:52.280470 11702 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.281482 11702 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:52.284497  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.284130 11702 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:52.293215  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.295548  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.288105 11695 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:52.298352  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.299710  7655 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:26:52.294992  7702 connection.cc:256] RPC call timeout handler was delayed by -4.4685254400001213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 12:26:52.291006 11581 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:26:52.300962  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.300976  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:52.301980  7655 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:52.302023  7702 connection.cc:256] RPC call timeout handler was delayed by -4.363225439999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
I20260709 12:26:52.302385  7653 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 4.
W20260709 12:26:52.302631  7702 connection.cc:256] RPC call timeout handler was delayed by -4.1652254400000857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 12:26:52.302979  7702 connection.cc:256] RPC call timeout handler was delayed by -4.1391254400000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:26:52.303331  7702 connection.cc:256] RPC call timeout handler was delayed by -4.12472544000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 12:26:52.303601  7702 connection.cc:256] RPC call timeout handler was delayed by -4.0428254399998877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20179999999999998s for a response
W20260709 12:26:52.303853  7702 connection.cc:256] RPC call timeout handler was delayed by -3.9996254400002726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 12:26:52.304088  7702 connection.cc:256] RPC call timeout handler was delayed by -3.9708254400002261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 12:26:52.304314  7702 connection.cc:256] RPC call timeout handler was delayed by -3.9285254400001577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 12:26:52.304564  7702 connection.cc:256] RPC call timeout handler was delayed by -3.8826254400000835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 12:26:52.304780  7702 connection.cc:256] RPC call timeout handler was delayed by -3.5298254399999678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25880000000000003s for a response
I20260709 12:26:52.307797  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.309646  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:52.311368  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.312155 11268 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:52.311030  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.312997  7655 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:52.316509  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:52.323113 11461 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:52.302067 11499 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:52.328986 11499 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:52.293752 11688 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:52.327031  7734 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:52.301322 11695 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:52.342728  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:52.343489 11695 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:52.319499 11689 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:26:52.324215 11619 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:52.343638  7653 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:52.347800 11581 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
W20260709 12:26:52.371066  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.356209 11556 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:52.341426 11461 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:26:52.372825 11695 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:52.373880  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68355.3us
I20260709 12:26:52.383510 11619 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:52.384196  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:52.385274  7653 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:52.387691  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:52.388629  7660 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.387691  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:52.392407  8415 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:52.394165  7653 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:52.370698 11692 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:52.379746  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:52.344899 11700 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:26:52.388338 11695 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:52.630713  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.631482  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.631906  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.632921  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.633306  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.830103  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 445920us
I20260709 12:26:52.840700  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:52.646049  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.845275  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:52.853005  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:52.866683  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:52.456786 11695 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:52.899003  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.865779 11700 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:52.855038 11706 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:52.907701  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:52.908398  7658 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:52.914600 11700 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:52.915082  7809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.040s
W20260709 12:26:52.915733  7809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.012s	sys 0.040s
W20260709 12:26:52.916288  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49669.9us
I20260709 12:26:52.912592 11703 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:52.917805  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:52.919613  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:52.920796 11700 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:52.921383 11700 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.921947 11700 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:52.923578 11708 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:52.926779  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:52.935560 11703 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:52.954121  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 601) took 2633 ms (client timeout 2172 ms). Trace:
I20260709 12:26:52.942412 11711 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:52.960300  7811 rpcz_store.cc:269] 0709 12:26:50.321016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.321418 (+   402us) service_pool.cc:224] Handling call
0709 12:26:52.953745 (+2632327us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:52.953971 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464318336}
W20260709 12:26:52.967192  7805 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.575s	user 0.020s	sys 0.028s
W20260709 12:26:52.972821  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.468s	user 0.016s	sys 0.018s
W20260709 12:26:52.981623  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.986172  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.989228  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.992908  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64001.9us
W20260709 12:26:52.995258  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.490s	user 0.018s	sys 0.018s
W20260709 12:26:52.995924  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.996879  8409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 974) took 2623 ms (client timeout 1704 ms). Trace:
W20260709 12:26:52.997089  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.997318  8409 rpcz_store.cc:269] 0709 12:26:50.373544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.378803 (+  5259us) service_pool.cc:224] Handling call
0709 12:26:52.996599 (+2617796us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:52.996770 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852229376}
W20260709 12:26:53.008188  7805 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.616s	user 0.024s	sys 0.029s
I20260709 12:26:53.015692  7660 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:26:53.025911 11703 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:53.026461  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2310) took 1020 ms. Trace:
I20260709 12:26:53.029130 11706 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:53.033531 11711 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:53.037941  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2318) took 1022 ms. Trace:
I20260709 12:26:53.038468  7653 rpcz_store.cc:276] 0709 12:26:52.015139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.842134 (+826995us) service_pool.cc:224] Handling call
0709 12:26:52.893640 (+ 51506us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:52.904559 (+ 10919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.904601 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:52.913418 (+  8817us) log.cc:844] Serialized 52 byte log entry
0709 12:26:52.986247 (+ 72829us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:52.986298 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.986385 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.037611 (+ 51226us) raft_consensus.cc:1685] finished
0709 12:26:53.037646 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.037860 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1440,"raft.queue_time_us":13015,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":1748,"thread_start_us":56053,"threads_started":1,"wal-append.queue_time_us":89147}
I20260709 12:26:53.031086 11706 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.042819 11706 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:53.044385  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:53.045666  7661 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:53.047628  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2311) took 1040 ms. Trace:
I20260709 12:26:53.048090  7661 rpcz_store.cc:276] 0709 12:26:52.006825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.043940 (+1037115us) service_pool.cc:224] Handling call
0709 12:26:53.047478 (+  3538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.050110  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:53.051643  7729 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:53.060076  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.027114  7654 rpcz_store.cc:276] 0709 12:26:52.006428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.016235 (+  9807us) service_pool.cc:224] Handling call
0709 12:26:52.060934 (+ 44699us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:52.080532 (+ 19598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.080586 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:52.081863 (+  1277us) log.cc:844] Serialized 78 byte log entry
0709 12:26:52.127132 (+ 45269us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:52.263708 (+136576us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.025539 (+761831us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800389080. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:26:53.025795 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.025896 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.025952 (+    56us) raft_consensus.cc:1685] finished
0709 12:26:53.026104 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.026358 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5163,"prepare.queue_time_us":124773,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":218889,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":12299,"spinlock_wait_cycles":19041792,"thread_start_us":162248,"threads_started":3,"wal-append.queue_time_us":22646}
W20260709 12:26:53.067664  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2506) took 2555 ms (client timeout 2161 ms). Trace:
W20260709 12:26:53.068472  7729 rpcz_store.cc:269] 0709 12:26:50.505651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.579920 (+ 74269us) service_pool.cc:224] Handling call
0709 12:26:53.060375 (+2480455us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:53.074810  7777 connection.cc:256] RPC call timeout handler was delayed by -3.6590779040002417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33440000000000003s for a response
W20260709 12:26:53.127514  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 133635us
W20260709 12:26:53.077592  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.093s (SENT) [suppressed 43 similar messages]
I20260709 12:26:53.075845  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2129) took 1044 ms. Trace:
W20260709 12:26:53.077814  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.163498  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.164106  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.166059  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 593) took 2917 ms. Trace:
I20260709 12:26:53.168481  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 595) took 2884 ms. Trace:
W20260709 12:26:53.165758  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.173246  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:53.136065  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.162060  7809 rpcz_store.cc:276] 0709 12:26:52.031908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.032083 (+   175us) service_pool.cc:224] Handling call
0709 12:26:52.073450 (+ 41367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.073803 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:52.296842 (+223039us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.059328 (+762486us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.075290 (+ 15962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.075375 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:53.075458 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.075568 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.075750 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243518,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":221076,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":228,"thread_start_us":973168,"threads_started":4,"wal-append.queue_time_us":638221}
W20260709 12:26:53.174678  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.175246  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:53.169395  7805 rpcz_store.cc:276] 0709 12:26:50.249851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.251328 (+  1477us) service_pool.cc:224] Handling call
0709 12:26:50.251670 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.251908 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:53.010113 (+2758205us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.133326 (+123213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.133430 (+   104us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:53.133819 (+   389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.133932 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.134138 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139731,"prepare.queue_time_us":232086,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":56759,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":94,"thread_start_us":2739510,"threads_started":3,"wal-append.queue_time_us":59711}
I20260709 12:26:53.123965 11706 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:53.093379 11700 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:53.175011 11168 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.183811 11721 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:53.174652 11725 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:53.181061 11718 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:26:53.184345 11168 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.182307 11724 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.189627 11724 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.191023 11168 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:53.177243 11717 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:26:53.191779  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 599) took 2940 ms. Trace:
I20260709 12:26:53.191834 11168 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.200136 11168 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.199811  7807 rpcz_store.cc:276] 0709 12:26:50.266357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.295564 (+ 29207us) service_pool.cc:224] Handling call
0709 12:26:50.296002 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.818614 (+1522612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.191514 (+1372900us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:53.191715 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3498961152}
I20260709 12:26:53.203764 11168 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:53.193769 11499 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:53.195336 11725 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.197139  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2322) took 1177 ms. Trace:
I20260709 12:26:53.213300 11700 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:53.209157  7658 rpcz_store.cc:276] 0709 12:26:52.019398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.966670 (+947272us) service_pool.cc:224] Handling call
0709 12:26:52.967488 (+   818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.968163 (+   675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.968195 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.968980 (+   785us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.128571 (+159591us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:53.174071 (+ 45500us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.174352 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.177647 (+  3295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.196820 (+ 19173us) raft_consensus.cc:1685] finished
0709 12:26:53.196856 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.197051 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37752,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":578,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":4395,"thread_start_us":196999,"threads_started":2,"wal-append.queue_time_us":159574}
I20260709 12:26:53.192039 11724 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:53.228905  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.232638  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:53.233237  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:53.233572  7730 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:53.235176  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:53.235663  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:53.241024  7731 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:53.249481  7728 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:53.238263  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 160734us
W20260709 12:26:53.256069  7777 connection.cc:256] RPC call timeout handler was delayed by -3.3203794190003464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
W20260709 12:26:53.256436  7777 connection.cc:256] RPC call timeout handler was delayed by -3.3069554799999423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 12:26:53.256752  7777 connection.cc:256] RPC call timeout handler was delayed by -3.2159864750001361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26280000000000003s for a response
W20260709 12:26:53.257040  7777 connection.cc:256] RPC call timeout handler was delayed by -3.1952794190001441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23260000000000003s for a response
W20260709 12:26:53.257337  7777 connection.cc:256] RPC call timeout handler was delayed by -2.8350043999998888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 12:26:53.257652  7777 connection.cc:256] RPC call timeout handler was delayed by -2.7270673180000813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33690000000000003s for a response
W20260709 12:26:53.257925  7777 connection.cc:256] RPC call timeout handler was delayed by -2.4415572140001132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:26:53.262008  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:53.275846  7731 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:53.279294  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:53.279690  7734 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:53.283679  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:53.289273 11724 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:53.291867 11516 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.292939 11516 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.208611 11725 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:53.296751  7730 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:53.261088 11168 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.262279 11716 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:53.302220  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 99429.9us
I20260709 12:26:53.301441  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:53.306442  7734 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:53.306883  7626 connection.cc:256] RPC call timeout handler was delayed by -1.6792094669999642s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
W20260709 12:26:53.312559  7626 connection.cc:256] RPC call timeout handler was delayed by -1.6685798960002103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:26:53.296667 11624 raft_consensus.cc:670] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } [suppressed 4 similar messages]
W20260709 12:26:53.313416  7737 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 2.735s	user 0.032s	sys 0.009s
I20260709 12:26:53.314656  7737 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:26:53.313390  7626 connection.cc:256] RPC call timeout handler was delayed by -1.5604239790000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.48019999999999996s for a response
W20260709 12:26:53.315569  7626 connection.cc:256] RPC call timeout handler was delayed by -1.5490059509997991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 12:26:53.315135  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139990us
W20260709 12:26:53.316381  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2499) took 2911 ms (client timeout 2239 ms). Trace:
W20260709 12:26:53.316864  7737 rpcz_store.cc:269] 0709 12:26:50.405400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.517200 (+111800us) service_pool.cc:224] Handling call
0709 12:26:53.316302 (+2799102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.303799 11716 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:53.318107  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:53.306320 11704 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.302449 11724 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:26:53.327312  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:53.326921 11724 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:53.327931 11516 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:53.329684 11724 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:53.330469 11733 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:26:53.315990  7626 connection.cc:256] RPC call timeout handler was delayed by -1.5074559019999469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:26:53.331794  7626 connection.cc:256] RPC call timeout handler was delayed by -1.4561798959998669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 12:26:53.332361  7626 connection.cc:256] RPC call timeout handler was delayed by -1.4292881110000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:26:53.333325  7626 connection.cc:256] RPC call timeout handler was delayed by -1.286846365999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
I20260709 12:26:53.307140 11692 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.333792 11704 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.334342 11692 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:53.306106 11695 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.315773 11721 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:53.331204 11733 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:53.338152 11695 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:53.306190 11168 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:53.302867 11725 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:26:53.323882 11732 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.328096  7656 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:53.333844  7626 connection.cc:256] RPC call timeout handler was delayed by -1.0861463659998662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 12:26:53.366721  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.387419  7626 connection.cc:256] RPC call timeout handler was delayed by -1.0042094670002371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2989s for a response
I20260709 12:26:53.389709  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:53.392290  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:53.393249  7658 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9.
I20260709 12:26:53.397146  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:53.420457  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104213us
I20260709 12:26:54.002437  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:54.004295  7660 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:54.004539  7656 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.621s	user 0.017s	sys 0.014s
I20260709 12:26:54.005142  7656 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:53.465253 11168 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:54.002732  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 685990us
I20260709 12:26:53.420953 11700 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.006377 11168 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:53.625742  7734 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:53.451838  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:54.010285  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:53.786137  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.011685  7654 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:54.012072  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.322214 11716 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:54.002166  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 684446us
W20260709 12:26:53.957078  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.025992  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:53.475493  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:54.027768  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:53.421705  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:53.583057  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.063426  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.064673  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.555395  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.005733  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 698) took 2165 ms (client timeout 1583 ms). Trace:
W20260709 12:26:54.070636  7656 rpcz_store.cc:269] 0709 12:26:51.840802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.326924 (+1486122us) service_pool.cc:224] Handling call
0709 12:26:54.005657 (+678733us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:53.473340  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080026cec0 after lost signal to thread 6454
I20260709 12:26:54.105895  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:54.106752  7659 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:54.133399  7737 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:54.268159  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 183330us
I20260709 12:26:54.356755  7734 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:54.357823  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2342) took 1106 ms. Trace:
I20260709 12:26:54.358273  7734 rpcz_store.cc:276] 0709 12:26:53.251039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.355900 (+1104861us) service_pool.cc:224] Handling call
0709 12:26:54.356438 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.357439 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.357738 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.368427 11704 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.413168  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2343) took 1156 ms. Trace:
I20260709 12:26:54.415046  7737 rpcz_store.cc:276] 0709 12:26:53.256481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.411919 (+1155438us) service_pool.cc:224] Handling call
0709 12:26:54.412392 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.412672 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.412710 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.412773 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:54.412818 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.412892 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.413078 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.420619  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 699) took 2579 ms. Trace:
I20260709 12:26:54.421144  7662 rpcz_store.cc:276] 0709 12:26:51.841131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.409123 (+2567992us) service_pool.cc:224] Handling call
0709 12:26:54.420002 (+ 10879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.420219 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.420251 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.420305 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:54.420340 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.420402 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.420538 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.694451 11725 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:54.007146 11700 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.432981 11719 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:54.436673 11736 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:54.067764 11168 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:26:54.105265 11624 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.001s	sys 0.006s
I20260709 12:26:54.105509 11703 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:53.481074 11733 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.446877  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 774) took 1132 ms. Trace:
I20260709 12:26:54.469609  7737 rpcz_store.cc:276] 0709 12:26:53.314464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.445937 (+1131473us) service_pool.cc:224] Handling call
0709 12:26:54.446295 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.446489 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.446518 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.446566 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:54.446614 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.446674 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.446805 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.471836  7737 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:54.472306  7737 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:53.465432 11735 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:54.449627  7627 connection.cc:256] RPC call timeout handler was delayed by -3.7736449200001516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 12:26:54.454777 11704 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:54.005774 11695 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.460602 11168 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:54.493338  7700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.032s	sys 0.059s
W20260709 12:26:54.493804  7700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.032s	sys 0.059s
W20260709 12:26:54.494141  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 531091us
I20260709 12:26:54.466684 11736 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:54.466255 11703 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:54.468851  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac320 after lost signal to thread 7555
I20260709 12:26:53.421104 11732 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.474665  7659 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:54.079305 11724 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:54.477613  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.05603e+06us
W20260709 12:26:54.464316 11624 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.002s	sys 0.007s
I20260709 12:26:54.456262 11719 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:54.502298  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.463661 11692 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.463959 11738 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.489323  7737 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:54.507645  8379 connection.cc:256] RPC call timeout handler was delayed by -2.9230879020001339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 12:26:54.507952  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 700) took 2666 ms (client timeout 3333 ms). Trace:
W20260709 12:26:54.522356  7626 leader_election.cc:336] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.573s (SENT)
W20260709 12:26:54.522141  7659 rpcz_store.cc:269] 0709 12:26:51.841530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.105550 (+2264020us) service_pool.cc:224] Handling call
0709 12:26:54.507769 (+402219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.522965  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:54.509084  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 481411us
W20260709 12:26:54.525679  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 256247us
I20260709 12:26:54.527899  7737 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:54.509526 11700 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.511258 11725 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:54.459424  7734 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:54.511830 11703 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:54.515468 11168 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.535892 11168 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:54.536765 11732 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.537243  7737 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:54.537933  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 775) took 1221 ms. Trace:
I20260709 12:26:54.537798 11736 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.538347  7737 rpcz_store.cc:276] 0709 12:26:53.316343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.471395 (+1155052us) service_pool.cc:224] Handling call
0709 12:26:54.471709 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.537661 (+ 65952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.537870 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.516458 11747 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:54.539538 11747 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:54.517994 11748 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:54.512223 11724 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:54.519426 11704 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:54.497529 11695 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:54.530469  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:54.531966  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:54.540933  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2346) took 1225 ms. Trace:
W20260709 12:26:54.542073  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.549661  7809 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:54.549423  7807 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:54.553114  7626 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.624s (SENT)
I20260709 12:26:54.553673  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.553730  7734 rpcz_store.cc:276] 0709 12:26:53.315840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.458868 (+1143028us) service_pool.cc:224] Handling call
0709 12:26:54.459199 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.530910 (+ 71711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.540870 (+  9960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.557845  7734 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:54.527336  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:54.564337  7627 connection.cc:256] RPC call timeout handler was delayed by -3.3384646810000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 12:26:54.564795  7627 connection.cc:256] RPC call timeout handler was delayed by -3.1341646810001293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
W20260709 12:26:54.565129  7627 connection.cc:256] RPC call timeout handler was delayed by -3.1184496850000869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:26:54.565434  7627 connection.cc:256] RPC call timeout handler was delayed by -2.9600496849998308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 12:26:54.566527  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2211653790000128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:26:54.566975  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2094653789999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:26:54.567485  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1248653789998571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 12:26:54.567811  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1149653789998411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 12:26:54.530660 11751 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.565740  7627 connection.cc:256] RPC call timeout handler was delayed by -2.9060496849997435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 12:26:54.568692 11139 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.568221  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0618653789997552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 12:26:54.569085  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:54.570981  7627 connection.cc:256] RPC call timeout handler was delayed by -2.805664681000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25120000000000003s for a response
W20260709 12:26:54.571612  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7683496849999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
I20260709 12:26:54.571447  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:54.572067  7627 connection.cc:256] RPC call timeout handler was delayed by -2.4789562740002111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 12:26:54.571506  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49634.7us
W20260709 12:26:54.572458  7627 connection.cc:256] RPC call timeout handler was delayed by -2.4439994899998965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
W20260709 12:26:54.572736  8379 connection.cc:256] RPC call timeout handler was delayed by -2.8969879020005465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 12:26:54.572947  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0662309170002118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
W20260709 12:26:54.573313  8379 leader_election.cc:336] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.861s (SENT)
W20260709 12:26:54.573374  7627 connection.cc:256] RPC call timeout handler was delayed by -2.031130917000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
W20260709 12:26:54.573876  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0302309170001536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
W20260709 12:26:54.574236  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0231496850001349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2989s for a response
I20260709 12:26:54.570609  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:54.522154 11745 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:26:54.574615  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8333385569999336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26330000000000003s for a response
W20260709 12:26:54.576455  7627 connection.cc:256] RPC call timeout handler was delayed by -1.6360309170004257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2677s for a response
W20260709 12:26:54.577931  7736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2511) took 4027 ms (client timeout 1854 ms). Trace:
I20260709 12:26:54.578284  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:54.578560  7736 rpcz_store.cc:269] 0709 12:26:50.550355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.579053 (+ 28698us) service_pool.cc:224] Handling call
0709 12:26:54.577847 (+3998794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:54.587508  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:54.587838  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.589788  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.563655 11746 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:54.602818  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 969) took 4226 ms. Trace:
I20260709 12:26:54.476313 11721 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.603505  8407 rpcz_store.cc:276] 0709 12:26:50.371810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.375384 (+  3574us) service_pool.cc:224] Handling call
0709 12:26:50.375817 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.738814 (+362997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.597567 (+3858753us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:54.597866 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825901312}
I20260709 12:26:54.563397 11688 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:26:54.611545  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:54.612308  7807 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:54.612859 11688 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.613262  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:54.613468 11688 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.614602  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:54.575598 11735 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:54.616305 11735 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.579974 11741 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.574923 11692 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:54.620265 11692 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:54.621052 11692 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:54.606194  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.569394 11139 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.622984  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:54.619572 11741 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.526273 11738 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.628822  7804 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:54.618911 11735 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.629484  7807 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:54.563791 11695 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.653393 11695 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.592700 11743 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:54.654651 11743 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.657564 11743 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.658262 11743 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:54.658793 11743 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:54.599524 11744 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:54.591781 11742 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:54.663722 11742 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.666244 11742 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.622454 11750 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.667017 11750 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.668681 11760 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:54.669268 11760 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:54.669662  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46812.9us
I20260709 12:26:54.671591  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:54.672307  7734 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:54.672155  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:54.670182 11760 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:54.674448  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84755.7us
I20260709 12:26:54.677471  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.626273 11753 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.684324  7809 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:54.673430 11750 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:54.733170 11741 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:54.584838 11751 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.735602  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:54.735913  8379 connection.cc:256] RPC call timeout handler was delayed by -2.3404700450000746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 12:26:54.674293 11760 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:54.677599 11748 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:54.738710  7807 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
W20260709 12:26:54.742581  8379 connection.cc:256] RPC call timeout handler was delayed by -2.3286240229999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3231s for a response
I20260709 12:26:54.605870 11724 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.567291 11747 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.584199 11719 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.627810 11725 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.724059 11753 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.741832  7734 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:26:54.744287  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156839us
W20260709 12:26:54.868558  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 191054us
I20260709 12:26:54.868973  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:26:54.864293  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 299412us
W20260709 12:26:54.751497  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79427.2us
W20260709 12:26:54.872697  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.873456  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.874275  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.870824  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.881062  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:54.797374 11747 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:54.803733 11724 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:54.814776 11753 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.908532 11746 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:54.909040 11753 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:54.910300 11746 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.782955 11688 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:54.815879 11765 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:54.914157 11753 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:54.914775 11753 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:54.744802 11745 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.174s
I20260709 12:26:54.780669 11139 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:54.917438  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.753453 11741 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:54.916755 11753 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.920353 11741 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:26:54.782303 11751 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:54.921059 11741 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:54.780702 11754 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:26:54.935649 11139 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.937294 11704 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:54.937983 11763 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:54.938800 11763 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:54.768540 11738 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.935873 11759 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:54.936985 11139 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.950691 11763 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.953830 11139 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.795919 11762 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:26:54.745282 11735 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.960632  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90200.3us
I20260709 12:26:54.779903 11703 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.796896 11716 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:54.749785 11748 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:54.788483 11743 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.968549  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:54.808030 11744 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:54.968854 11748 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:54.751792  8379 connection.cc:256] RPC call timeout handler was delayed by -2.264380500000243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 12:26:54.972409  8379 connection.cc:256] RPC call timeout handler was delayed by -2.1742879020002874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28350000000000003s for a response
W20260709 12:26:54.972967  8379 connection.cc:256] RPC call timeout handler was delayed by -2.16767004500025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 12:26:54.973122 11762 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:54.750643 11750 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:54.974627  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53666.1us
W20260709 12:26:54.973331  8379 connection.cc:256] RPC call timeout handler was delayed by -1.981200741000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 12:26:54.973783 11762 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:54.970899  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:54.958534  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:54.977306  8379 connection.cc:256] RPC call timeout handler was delayed by -1.953100206000272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:26:54.978442 11735 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:54.980564 11716 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:54.981479 11735 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.982067  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:54.983148 11745 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:54.983836 11745 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.984278 11745 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:54.985448  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:54.987104  8379 connection.cc:256] RPC call timeout handler was delayed by -1.8130700450001314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 12:26:54.978633  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98079.2us
W20260709 12:26:54.987797  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5265583089999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
W20260709 12:26:54.988233  8379 connection.cc:256] RPC call timeout handler was delayed by -1.1940552969999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25470000000000004s for a response
I20260709 12:26:54.992219 11759 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:54.994306 11746 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:54.995282 11704 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:54.995870 11704 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.996335 11704 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.001387 11703 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.007128  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:55.026523  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144689us
I20260709 12:26:55.031975 11688 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:55.035593  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:55.039513  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:55.038233  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:55.050758 11746 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.051721 11746 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:55.052515  7626 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.661s (SENT)
I20260709 12:26:55.053660  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.059870 11742 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:55.060458 11757 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:55.061136 11757 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:55.049542 11750 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:55.066699  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81489.1us
I20260709 12:26:55.069521  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:55.070502  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2127) took 3043 ms (client timeout 1501 ms). Trace:
W20260709 12:26:55.071143  7806 rpcz_store.cc:269] 0709 12:26:52.026939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.069877 (+3042938us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:55.070026 (+   149us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:55.072754  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.072980 11695 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:55.073889  7655 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:55.062922 11757 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.077958 11749 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:55.078289  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.079478  7659 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:55.080464  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73570.7us
I20260709 12:26:55.054702 11746 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:55.008278 11762 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.092257 11762 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.049877 11741 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.105192 11761 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:55.106968 11741 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:55.083738  7777 leader_election.cc:400] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9.
I20260709 12:26:55.110124  7777 leader_election.cc:403] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.112094  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.115602 11741 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:55.115995 11704 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.117126 11414 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:55.118911 11704 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:55.124825  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48980us
W20260709 12:26:55.122870 11594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.664s	user 0.012s	sys 0.024s
I20260709 12:26:55.120787 11688 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:55.131300 11594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.675s	user 0.014s	sys 0.024s
I20260709 12:26:55.134864  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:55.135656 11763 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:55.139533 11763 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:55.141039 11748 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.141460 11763 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.143399 11688 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:55.155774 11760 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:55.156473  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2136) took 3048 ms. Trace:
I20260709 12:26:55.148034 11688 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.156987  7812 rpcz_store.cc:276] 0709 12:26:52.107900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.111946 (+  4046us) service_pool.cc:224] Handling call
0709 12:26:52.112439 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.122622 (+ 10183us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:54.621863 (+2499241us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b180041c6e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:54.622034 (+   171us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.085352 (+463318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.125122 (+ 39770us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:55.138876 (+ 13754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.152790 (+ 13914us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.156254 (+  3464us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134501,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":4544,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":14643,"spinlock_wait_cycles":16229248,"thread_start_us":502975,"threads_started":2,"wal-append.queue_time_us":435623}
I20260709 12:26:55.160743 11745 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.161784  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:55.163347  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49035.6us
I20260709 12:26:55.168560  8413 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:55.168450 11741 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9.
I20260709 12:26:55.166304 11745 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:55.173118  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.174504 11745 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:55.178511 11748 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:55.184032  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.187239  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.190642 11762 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:55.194983  8379 leader_election.cc:336] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.544s (SENT)
I20260709 12:26:55.186177  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.186823 11757 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:55.196012 11757 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:55.196347  8407 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:55.197502 11757 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:55.210837  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.604s (SENT)
I20260709 12:26:55.215562  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:55.216871 11665 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:53.166925  7810 rpcz_store.cc:276] 0709 12:26:50.249080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.250183 (+  1103us) service_pool.cc:224] Handling call
0709 12:26:50.250518 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.250729 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:53.034645 (+2783916us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b18000bb3c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:26:53.034888 (+   243us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.165322 (+130434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.165515 (+   193us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:53.165582 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.165680 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.165896 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140453,"prepare.queue_time_us":181634,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1661,"raft.queue_time_us":2113,"spinlock_wait_cycles":17066368,"thread_start_us":215101,"threads_started":2}
W20260709 12:26:55.217658  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.796s (SENT)
W20260709 12:26:55.212550  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.220216  8379 leader_election.cc:336] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.190s (SENT)
I20260709 12:26:55.220757  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.222080 11665 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:55.231545  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 182661us
W20260709 12:26:55.232724  7626 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.687s (SENT)
I20260709 12:26:55.233237  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.236449 11704 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:55.238502  7626 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.920s (SENT)
I20260709 12:26:55.239092  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.240087 11704 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:55.240670  7626 leader_election.cc:336] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.948s (SENT)
W20260709 12:26:55.242667  7626 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.987s (SENT)
I20260709 12:26:55.243170  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.244205 11704 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:55.244839  7626 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.999s (SENT)
I20260709 12:26:55.245397  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.246994 11746 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.247804 11746 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.249675  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.250469 11746 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.250553  7656 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:26:55.246696 11704 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:55.251933  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.252735  7656 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:26:55.253872  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.254860  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.255275  7656 raft_consensus.cc:2368] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
I20260709 12:26:55.255784  7654 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 5.
I20260709 12:26:55.256319  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.256865  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.257128  7656 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:26:55.257722  7654 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:26:55.258744  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.259955  7654 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:55.261102  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.258253  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.262001  7656 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.262019  7654 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:55.271878  7654 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:55.271867  7656 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
W20260709 12:26:55.272694  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.273593  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.273892  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.274407  7656 raft_consensus.cc:2376] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9. Re-sending same reply.
I20260709 12:26:55.275520  8379 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.274577  7654 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:55.277184  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.278280  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.275802  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.279735  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.280629 11547 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:55.281312 11547 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:55.283393 11601 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:55.283885  7656 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.282497 11547 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.284126 11601 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:55.286823 11665 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:55.288568 11665 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.289026 11665 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:55.288302 11601 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:52.419764 11619 raft_consensus.cc:2764] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:26:55.291265  7547 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.278s	user 0.010s	sys 0.011s
I20260709 12:26:55.291635 11601 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.293263  7656 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:55.294709  7656 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:55.304997 11665 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.308954 11665 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:55.302258  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 272744us
I20260709 12:26:55.316390  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:55.320204 11706 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:55.321136 11706 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:55.322935 11706 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.327461  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:55.330535 11461 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:55.331125 11461 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:55.331796 11461 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:55.341020  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:55.355614 11763 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:55.356312 11763 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:55.357581 11763 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.358155  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:55.362586  7775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.854s	user 0.021s	sys 0.033s
W20260709 12:26:55.330397  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2481) took 4948 ms (client timeout 2133 ms). Trace:
W20260709 12:26:55.363448  7775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.855s	user 0.023s	sys 0.033s
W20260709 12:26:55.363632  7732 rpcz_store.cc:269] 0709 12:26:50.381298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.381466 (+   168us) service_pool.cc:224] Handling call
0709 12:26:55.329746 (+4948280us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001c5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:55.330086 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347741440}
W20260709 12:26:55.365078  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 4.87017e+06us
I20260709 12:26:55.372061  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:55.372239  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:55.374231  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.375029 11601 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:55.377009  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.377487  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.377794  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2132) took 3342 ms (client timeout 2211 ms). Trace:
W20260709 12:26:55.377996  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.378234  7803 rpcz_store.cc:269] 0709 12:26:52.035613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.125800 (+ 90187us) service_pool.cc:224] Handling call
0709 12:26:55.377433 (+3251633us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400331654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:55.377604 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290537728}
I20260709 12:26:55.382007  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:55.382750 11743 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:55.382869  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2473) took 5162 ms (client timeout 1987 ms). Trace:
I20260709 12:26:55.383514 11743 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:55.385682 11743 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.388203  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:55.389474 11461 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:55.383514  7735 rpcz_store.cc:269] 0709 12:26:50.225792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.229125 (+  3333us) service_pool.cc:224] Handling call
0709 12:26:55.357821 (+5128696us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:26:55.382792 (+ 24971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4781440}
I20260709 12:26:55.398377  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.399554  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.400482  7656 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:55.407948  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:55.409106 11706 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:55.409720 11706 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:55.410185 11706 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.410876 11461 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:55.404735  7661 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:55.417198 11601 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:55.418370  7661 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:55.438719 11608 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:55.439076 11793 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 12:26:55.439515  7656 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:55.442723  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.444916  7661 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:55.445320  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.448989  7654 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
W20260709 12:26:55.449491  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080022a5a0 after lost signal to thread 7593
W20260709 12:26:55.450268  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08002eec60 after lost signal to thread 7613
I20260709 12:26:55.446810  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:26:55.451579  7626 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.133s (SENT)
I20260709 12:26:55.452348  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:55.452406 11547 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:55.452610  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.446224  7805 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.453740  7810 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:55.453892  7803 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:26:55.456210  7654 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:26:55.459558  7626 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.161s (SENT)
I20260709 12:26:55.462613  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.455061 11645 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:26:55.464725  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:55.466662 11744 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:55.467271 11744 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.464740 11272 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:26:55.467549 11139 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.467759 11744 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:55.468441 11139 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.469342  7810 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:26:55.470785  7656 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:55.471429 11716 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.476445 11716 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.480291 11716 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:55.455088 11547 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:55.461220 11608 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:55.471020 11139 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.488492  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:55.474184 11743 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:55.495200 11743 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.495673 11743 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.498725  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:55.499101  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 597) took 5248 ms (client timeout 1796 ms). Trace:
W20260709 12:26:55.499629  7804 rpcz_store.cc:269] 0709 12:26:50.250606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.252171 (+  1565us) service_pool.cc:224] Handling call
0709 12:26:55.498679 (+5246508us) spinlock_profiling.cc:243] Waited 4.26 s on lock 0x7b54003f3754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:55.498895 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9806244224}
I20260709 12:26:55.500079  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:55.503044 11461 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:55.503574 11139 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:55.503600 11461 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.504238 11139 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.504691 11461 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.504793 11139 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:55.497038  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.508184  7805 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:55.496248 11706 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.513496  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.514703  7803 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:55.486801  7547 maintenance_manager.cc:643] P 004cbdb090d14f8ca89ed535ed34a625: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 12.855s	user 0.246s	sys 0.119s Metrics: {"bytes_written":30201,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":256504,"dirs.run_cpu_time_us":719,"dirs.run_wall_time_us":13651,"drs_written":1,"lbm_read_time_us":464,"lbm_reads_lt_1ms":4,"lbm_write_time_us":623662,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":8,"lbm_writes_gt_100_ms":3,"lbm_writes_lt_1ms":20,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":417314,"threads_started":2,"update_count":21,"wal-append.queue_time_us":118287}
I20260709 12:26:55.515532 11706 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:55.516564 11706 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:55.517194  7810 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:55.520538  7548 maintenance_manager.cc:419] P 004cbdb090d14f8ca89ed535ed34a625: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8926 bytes on disk
I20260709 12:26:55.523650 11272 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:55.524221  7547 maintenance_manager.cc:643] P 004cbdb090d14f8ca89ed535ed34a625: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":846,"lbm_reads_lt_1ms":4}
I20260709 12:26:55.525586  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.523650 11547 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:55.526665  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.527590  7810 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:55.529670  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.530261  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.531028  7809 raft_consensus.cc:2376] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8. Re-sending same reply.
I20260709 12:26:55.533895  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.534927  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.535447 11716 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:55.535926 11716 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:55.536365 11716 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.536460  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.536158  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.541066  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.542244  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.545799  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.547983  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:55.549257  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.550118  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.558066  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.558276  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.559067  7808 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:55.559545  7807 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:55.560662  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.562134  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.563019  7808 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:55.564793  7662 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:26:55.588541 11702 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.589193 11702 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.589219  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.592272  7662 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:55.593246  7729 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:55.593504  7656 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.593442  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.594295 11702 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.594530  7808 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:55.593631  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.595672  7662 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:55.595594  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.596446  7808 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:55.596720  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:55.597406  7662 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:55.597359  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.598160  7808 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:55.599116  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.612412 11139 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:55.613940  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 544611us
I20260709 12:26:55.610699  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.575604  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.805884  7803 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:55.572561 11744 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.810755 11744 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.577592  7660 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:55.577195 11743 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.816201 11743 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.582221 11716 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.571590  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.819494  7806 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.579043  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:55.831866  7809 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.585258  7812 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:55.584972  7810 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
W20260709 12:26:55.569631  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.101758  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.102687  7810 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:55.615167  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:56.103946  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 489037us
I20260709 12:26:56.104928  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.105803 11619 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.106528 11619 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:55.646124  7656 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:56.108956 11619 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:56.111572 11692 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:56.112493 11692 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:55.676614  7662 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:56.116007  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.116787  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:26:56.118217  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.668853 11139 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.580467 11461 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:56.125670 11461 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.711154  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.618719 11702 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.141985 11702 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:55.712170  7626 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.512s (SENT)
I20260709 12:26:55.713006  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:56.143487  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.143707  7658 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:26:56.144541  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 432392us
I20260709 12:26:56.145238  7658 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:55.714409  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.716746  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.726617  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.731438  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.776676 11719 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.818821 11716 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:55.815120  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:55.830994 11753 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.837075  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.845732  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2328) took 2614 ms. Trace:
W20260709 12:26:55.858944  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.860476  7624 connection.cc:256] RPC call timeout handler was delayed by -1.2572607630004313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23740000000000003s for a response
I20260709 12:26:55.880537  7806 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:55.885601  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.887635  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:55.888831  7809 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:56.153635  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:55.894073 11700 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:55.895133  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 767) took 3927 ms. Trace:
I20260709 12:26:55.927130  7734 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:55.928676  7735 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:55.930087  7732 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:55.931370  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:26:55.934952  7775 connection.cc:256] RPC call timeout handler was delayed by -3.7077020719998472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
I20260709 12:26:55.895017  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:55.969588  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9f20 after lost signal to thread 7686
W20260709 12:26:56.160156  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800196220 after lost signal to thread 7687
W20260709 12:26:55.969982  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.976217 11624 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:26:55.980342  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 993414us
I20260709 12:26:55.616564  7808 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:55.997000  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:55.999696  7661 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:55.999607  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:56.001623  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.004132  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.005585  7805 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:56.010353  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:56.013096  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:56.020638 11255 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:56.022773  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:56.040544  7731 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:56.064508  8413 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:26:56.066701  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.079716  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 776) took 2607 ms. Trace:
I20260709 12:26:56.084323  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2146) took 1534 ms. Trace:
I20260709 12:26:56.165766 11759 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:56.166003  7811 rpcz_store.cc:276] 0709 12:26:54.550309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.552159 (+  1850us) service_pool.cc:224] Handling call
0709 12:26:54.568953 (+ 16794us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:54.879748 (+310795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.879813 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:54.881626 (+  1813us) log.cc:844] Serialized 52 byte log entry
0709 12:26:54.930701 (+ 49075us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:55.167029 (+236328us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.171318 (+  4289us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.172583 (+  1265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.083394 (+910811us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003f3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:56.083453 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.083595 (+   142us) raft_consensus.cc:1685] finished
0709 12:26:56.083610 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.084044 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173,"prepare.queue_time_us":92447,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":93,"raft.queue_time_us":136515,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":385,"spinlock_wait_cycles":276657536,"thread_start_us":428082,"threads_started":3,"wal-append.queue_time_us":89762}
I20260709 12:26:56.168691  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.169348  7811 raft_consensus.cc:2376] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6. Re-sending same reply.
I20260709 12:26:56.170356  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.170890  7659 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:56.171300 11272 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.171859 11272 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:55.680907  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70696.6us
I20260709 12:26:56.172468  7655 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:56.119199  7810 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:56.163120  7803 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:56.106860  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:56.128031  7653 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:26:56.162686  7660 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:56.135494 11608 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.151137  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.152072  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:56.149755  7729 rpcz_store.cc:276] 0709 12:26:53.231675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.234261 (+  2586us) service_pool.cc:224] Handling call
0709 12:26:55.592799 (+2358538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.641371 (+ 48572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.845424 (+204053us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b54001c5cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:55.845629 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5422971136}
I20260709 12:26:56.172116  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:56.088972  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.178866  7806 raft_consensus.cc:2376] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8. Re-sending same reply.
I20260709 12:26:56.162457  7807 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:56.184283  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.178449  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.180140  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.186192  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.187568  7809 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:56.189237  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.194109  7808 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:56.198573  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.198945  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.199649  7811 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:56.196981  8409 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:56.206701  7807 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.156682  7733 rpcz_store.cc:276] 0709 12:26:51.968094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.028483 (+ 60389us) service_pool.cc:224] Handling call
0709 12:26:52.075294 (+ 46811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.075795 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.075850 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.076483 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:26:55.853538 (+3777055us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b1800300a00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:26:55.893693 (+ 40155us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:55.893740 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.894626 (+   886us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.894673 (+    47us) raft_consensus.cc:1685] finished
0709 12:26:55.894684 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.894944 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":598,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6345,"spinlock_wait_cycles":19638656,"thread_start_us":39888,"threads_started":1,"wal-append.queue_time_us":3814518}
I20260709 12:26:56.207422  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.207638  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.212769  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.199093  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.215322  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.215157  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:56.215780  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.216341  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.216228  7812 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:26:56.216881  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.221302  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.221959  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.165149  7737 rpcz_store.cc:276] 0709 12:26:53.472465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.540787 (+1068322us) service_pool.cc:224] Handling call
0709 12:26:54.549401 (+  8614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.792340 (+1242939us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:55.792750 (+   410us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.794533 (+  1783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.794583 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:55.794644 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.079271 (+284627us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:56.079333 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.079524 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":216,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":122,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":2857605248,"thread_start_us":831,"threads_started":1,"wal-append.queue_time_us":1303}
I20260709 12:26:56.227380  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.232092  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.232363  7810 raft_consensus.cc:2376] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8. Re-sending same reply.
I20260709 12:26:56.236052  7808 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.233237  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.204021 11704 raft_consensus.cc:2764] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:26:56.245026  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.204442 11650 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:56.246127  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.256204  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.246891  7655 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:56.247926  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.248459 11725 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.214969 11700 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:56.240168  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 133422us
I20260709 12:26:56.259518  8408 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:56.268217  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:56.270045  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.271324  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68918.5us
I20260709 12:26:56.258245  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.272207  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.284004  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.278395  7803 raft_consensus.cc:2376] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6. Re-sending same reply.
W20260709 12:26:56.284989  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6720 after lost signal to thread 7761
I20260709 12:26:56.287673  7805 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:56.291109  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:56.291504  8379 leader_election.cc:336] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.693s (SENT)
I20260709 12:26:56.295464  7805 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.298058  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.301880  7805 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:56.193783 11608 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.304693  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.316955  7806 raft_consensus.cc:2376] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10. Re-sending same reply.
I20260709 12:26:56.306569  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.308713  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.309922  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.309994  7659 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:56.315205  8407 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:26:56.307576  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.315932  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.312444  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:56.315126  7812 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.315582  7655 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:56.321800  7810 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.333689  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.336584  7731 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 12:26:56.332432  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.342680  7729 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:56.341622  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800358300 after lost signal to thread 7836
W20260709 12:26:56.345464  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08003d49a0 after lost signal to thread 7837
I20260709 12:26:56.355978  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.344683  7803 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:56.356413  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.357378  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.358129  7806 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:56.356354  7657 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:56.312912 11704 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.361936  7660 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:56.358407  7807 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:26:56.367177  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.375643  7803 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:56.376911  7808 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:26:56.377071  7809 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:56.378326 11624 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:56.378943 11624 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:56.379889 11255 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:56.380383 11624 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.382740 11624 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:56.383545 11811 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:56.278020 11650 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:56.388247  7730 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:56.387145  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.390417  7805 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:56.391111  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:56.374595  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88876.7us
W20260709 12:26:56.396620  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800351fa0 after lost signal to thread 8439
I20260709 12:26:56.396679  7809 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:56.397187  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800225100 after lost signal to thread 8440
I20260709 12:26:56.385362  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.398995  7810 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:56.311978  8408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:56.404944 11608 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:56.405972 11608 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:56.389312  8406 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:56.409279  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.409850 10979 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:56.410344 10979 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:56.411796 11704 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.405299  7806 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:26:56.412460  8408 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.413761  8408 raft_consensus.cc:614] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:56.413594 11760 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:56.417202  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:56.419814  7806 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:26:56.420060  7803 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:56.421784  7806 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:56.422286  7806 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:56.404929 11650 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:56.425547 11650 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.426034 11650 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:56.425603 11268 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.155s
I20260709 12:26:56.426632 11650 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.427752 11268 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:56.428239 11268 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:56.444989  7809 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:56.445973 11272 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:56.448899 11700 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:56.315894 11692 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:26:56.455441  7806 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.464779  7808 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:26:56.483363  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.284271 11725 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:56.367009 11759 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:56.489405  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800390be0 after lost signal to thread 11335
I20260709 12:26:56.489933  7807 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:56.490052  7803 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:56.490800  7811 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:26:56.487665  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 233917us
I20260709 12:26:56.502346  7806 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:56.462016 11753 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:56.520274  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 222601us
I20260709 12:26:56.415028 11704 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:56.525679  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.527858  7734 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:56.544909  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 314124us
I20260709 12:26:56.545892  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:56.520570 11753 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:56.570685  7833 debug-util.cc:398] Leaking SignalData structure 0x7b0800360e00 after lost signal to thread 11590
I20260709 12:26:56.571025  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:56.578718  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 262847us
I20260709 12:26:56.539790 11268 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:56.599924 11268 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:56.600081 11547 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:56.600693 11547 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:56.602707  7558 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:56.604792  7558 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:56.606230 11760 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:56.607045 11719 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.607345  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 219715us
I20260709 12:26:56.500627 10979 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.572201 11702 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:56.609500  7657 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:56.601769 11547 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.606772  7558 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:56.607681 11719 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.606772 11760 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.640045  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:56.638567  7806 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:26:56.652712  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.653622  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2b40 after lost signal to thread 11641
I20260709 12:26:56.641803 11760 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:56.626855 11255 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:26:56.653452 11702 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:56.676009 11255 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:56.676560 11255 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:56.677585 11255 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.667586 10979 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:56.662320  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91419.3us
W20260709 12:26:56.686340  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 161791us
I20260709 12:26:56.623713 11818 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:56.679049 11255 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:56.691870  7804 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:26:56.693711  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 147936us
W20260709 12:26:56.697892  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.698014  7777 leader_election.cc:400] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
I20260709 12:26:56.698643  7777 leader_election.cc:403] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:26:56.698724  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.699240  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.701362  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.702651  7803 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:56.702370  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.732635  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.712706  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080036c020 after lost signal to thread 6451
W20260709 12:26:56.733549  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080036be20 after lost signal to thread 6454
I20260709 12:26:56.733676  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.734040  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080016cb60 after lost signal to thread 7555
I20260709 12:26:56.734334  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
W20260709 12:26:56.734817  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080016c840 after lost signal to thread 7593
W20260709 12:26:56.735332  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080016cb40 after lost signal to thread 7613
W20260709 12:26:56.731570  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48042.1us
W20260709 12:26:56.736552  7758 debug-util.cc:398] Leaking SignalData structure 0x7b0800335b00 after lost signal to thread 7686
W20260709 12:26:56.736861  7758 debug-util.cc:398] Leaking SignalData structure 0x7b0800335be0 after lost signal to thread 7687
W20260709 12:26:56.738446  7758 debug-util.cc:398] Leaking SignalData structure 0x7b08003939e0 after lost signal to thread 7761
I20260709 12:26:56.706794  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.742345  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:56.743108  7812 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:26:56.705139  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:26:56.745757  7758 debug-util.cc:398] Leaking SignalData structure 0x7b0800319de0 after lost signal to thread 7836
I20260709 12:26:56.746197  7807 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:56.746272  7758 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1440 after lost signal to thread 7837
W20260709 12:26:56.747977  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080022e080 after lost signal to thread 8439
W20260709 12:26:56.748978  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080035fb60 after lost signal to thread 8440
I20260709 12:26:56.748848  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.749933  7758 debug-util.cc:398] Leaking SignalData structure 0x7b080010e960 after lost signal to thread 11594
I20260709 12:26:56.750487  7805 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:56.754580  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.756583  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 128747us
I20260709 12:26:56.663141 11753 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:56.763859  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:56.766294  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69098.4us
W20260709 12:26:56.769877 11810 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 12:26:56.773044  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 715) took 1300 ms. Trace:
W20260709 12:26:56.773399 11816 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
W20260709 12:26:56.774322  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.775821  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 717) took 1242 ms. Trace:
I20260709 12:26:56.776353  7658 rpcz_store.cc:276] 0709 12:26:55.533356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.146101 (+612745us) service_pool.cc:224] Handling call
0709 12:26:56.147862 (+  1761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.158220 (+ 10358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.158250 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.202558 (+ 44308us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.336371 (+133813us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.388631 (+ 52260us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.572419 (+183788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.572494 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.775481 (+202987us) raft_consensus.cc:1685] finished
0709 12:26:56.775505 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.775739 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28153,"prepare.queue_time_us":115626,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":31140,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":9906,"thread_start_us":272263,"threads_started":2,"wal-append.queue_time_us":301882}
W20260709 12:26:56.786178  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.786866  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.789863  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2349) took 1242 ms. Trace:
I20260709 12:26:56.773531  7656 rpcz_store.cc:276] 0709 12:26:55.472300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.109529 (+637229us) service_pool.cc:224] Handling call
0709 12:26:56.109904 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:56.110414 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:56.110776 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.692829 (+582053us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.693064 (+   235us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:56.706019 (+ 12955us) log.cc:844] Serialized 52 byte log entry
0709 12:26:56.737815 (+ 31796us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:56.737894 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.738257 (+   363us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.772719 (+ 34462us) raft_consensus.cc:1685] finished
0709 12:26:56.772744 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.772963 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13119,"prepare.queue_time_us":143288,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":111,"raft.queue_time_us":3061,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":25527,"thread_start_us":567943,"threads_started":2,"wal-append.queue_time_us":231611}
W20260709 12:26:56.794646  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.796823  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2594) took 1260 ms. Trace:
I20260709 12:26:56.797475  7804 rpcz_store.cc:276] 0709 12:26:55.525071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.525235 (+   164us) service_pool.cc:224] Handling call
0709 12:26:56.784829 (+1259594us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400332054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:56.784959 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151636736}
I20260709 12:26:56.694430 11812 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:26:56.813256  7810 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:56.760373 11818 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7.
I20260709 12:26:56.768551 11700 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:56.816910  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:56.789146  7558 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 5, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:56.833446  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:56.805960  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:56.820778  7730 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:56.821609  7733 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:56.822788  7729 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:56.790438  7661 rpcz_store.cc:276] 0709 12:26:55.547289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.164691 (+617402us) service_pool.cc:224] Handling call
0709 12:26:56.208687 (+ 43996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.245880 (+ 37193us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:56.510771 (+264891us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.761026 (+250255us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.769090 (+  8064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.769164 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:56.789465 (+ 20301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.789588 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.789784 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15520,"prepare.queue_time_us":210100,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":29122,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3851,"thread_start_us":370978,"threads_started":2}
I20260709 12:26:56.829064  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.763093 11725 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:26:56.837348  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 142769us
I20260709 12:26:56.849635  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:56.851565  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:56.862983  7558 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:26:56.870136  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 172358us
W20260709 12:26:56.871466  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60241.7us
I20260709 12:26:56.878875  7735 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 12:26:56.877023  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:56.870142 11753 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:56.876279 11828 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:56.880574 11725 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:56.885360 11268 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:26:56.888659  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102521us
I20260709 12:26:56.889393 11825 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.899650 11825 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:56.900148 11825 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.900290  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:26:56.900463  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:56.908169  8406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:26:56.908672  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2348) took 1362 ms. Trace:
I20260709 12:26:56.909264  7654 rpcz_store.cc:276] 0709 12:26:55.546175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.163305 (+617130us) service_pool.cc:224] Handling call
0709 12:26:56.163618 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.206989 (+ 43371us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:56.537537 (+330548us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.908243 (+370706us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.908317 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:56.908365 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.908443 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.908586 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55693,"prepare.queue_time_us":47878,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":554743,"threads_started":2}
I20260709 12:26:56.911693  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:56.911851 11823 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:56.912580 11823 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.912894 11608 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:56.913143  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:56.871809 11700 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:56.876150 11719 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:56.909473  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 80584.1us
I20260709 12:26:56.877272 11818 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:56.913031 11823 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:56.915730  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 96054.3us
W20260709 12:26:56.930239  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58211.8us
I20260709 12:26:56.915202  7735 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:56.934653  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:56.913532 11608 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:56.898451 11725 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.939148  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:56.940548 11704 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:56.940820 11836 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:56.941274 11836 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.941830 11836 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:56.943456 11608 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:56.948704  7735 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:56.955703 11830 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:56.971385  7735 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:56.972044  7735 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:56.972864  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 109450us
I20260709 12:26:56.931882 11702 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:56.990538  8406 raft_consensus.cc:606] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:56.993434  8406 raft_consensus.cc:614] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:56.991466  7558 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 6, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 6 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:57.007867 11819 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:57.008507 11819 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:56.944108 11725 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.009572 11819 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.010969 11819 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.014355  7558 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:26:57.028832  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 99624.2us
W20260709 12:26:57.029475  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.014333  7735 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:56.970942 11268 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:26:57.033365  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98812.6us
W20260709 12:26:57.034085  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59157.8us
W20260709 12:26:57.036552  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 123548us
I20260709 12:26:56.992825 11704 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:57.041170  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 992) took 1036 ms. Trace:
I20260709 12:26:57.041589  8411 rpcz_store.cc:276] 0709 12:26:56.005032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.281713 (+276681us) service_pool.cc:224] Handling call
0709 12:26:56.315611 (+ 33898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.372889 (+ 57278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.372936 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.453032 (+ 80096us) log.cc:844] Serialized 556 byte log entry
0709 12:26:56.592409 (+139377us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.592465 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.592541 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.040949 (+448408us) raft_consensus.cc:1685] finished
0709 12:26:57.040969 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.041108 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":138776,"threads_started":1,"wal-append.queue_time_us":239547}
I20260709 12:26:57.040297  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.048605  7735 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.022527 11830 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:57.050344  7557 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 303) took 31727 ms (client timeout 15000 ms). Trace:
I20260709 12:26:57.030550 11719 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.053917  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.027056  7558 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 7, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.060947  7558 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.039394 11268 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.077538 11716 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:57.076725  7558 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.078382 11716 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.079562  7558 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 7, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.081171 11716 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:57.062345 11624 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.003s
I20260709 12:26:57.092077  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:57.105782 11811 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:57.105988 11820 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:57.106554 11833 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:57.095464  7558 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 6 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 7 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
W20260709 12:26:57.118129  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65593us
I20260709 12:26:57.115453  7558 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 8, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 8 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:26:57.119897  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.121670  7558 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.132565  7558 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 6, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 6 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:57.150360  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 120652us
I20260709 12:26:57.154644  7735 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:57.155405  7735 pending_rounds.cc:85] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:26:57.155848  7735 pending_rounds.cc:107] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:26:57.156322  7735 raft_consensus.cc:2889] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.048336 11704 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.072184 11719 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.165935  7558 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.052796 11700 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.169334  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:57.158797 11650 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.005s
I20260709 12:26:57.142036 11815 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:57.171919 11650 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.172034 11815 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.174309 11815 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:57.174507  7809 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:57.094522 11268 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:57.176329  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 141308us
I20260709 12:26:57.183497  7558 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.181844 11841 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:57.191401 11840 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:57.191262 11841 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.192378 11840 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:57.193075 11840 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:57.193907 11841 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:57.193645 11840 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.194525 11841 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.211617  7558 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.142671 11842 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:26:57.174309 11650 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:57.215735  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 170852us
W20260709 12:26:57.199541 11760 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.001s
I20260709 12:26:57.181371 11759 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:57.217962 11608 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.227001  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:57.227831 11849 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:57.227160  7558 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 5, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 5 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:57.228760 11841 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.228451 11842 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Committing config change with OpId 8.8: config changed from index 4 to 8, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.230800 11818 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:26:57.231848 11834 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
I20260709 12:26:57.216125  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:57.232066 11719 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.233883  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.170480 11624 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:26:57.237753  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118383us
I20260709 12:26:57.233690 11759 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.181757 11702 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.250797  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 196132us
W20260709 12:26:57.052350  7557 rpcz_store.cc:269] 0709 12:26:25.268771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.292426 (+ 23655us) service_pool.cc:224] Handling call
0709 12:26:56.974339 (+31681913us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:48.967853 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:48.969180 (+  1327us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:48.969262 (+    82us) write_op.cc:435] Acquired schema lock
0709 12:26:48.969294 (+    32us) tablet.cc:662] Decoding operations
0709 12:26:49.046802 (+ 77508us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:49.046898 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:49.046933 (+    35us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:26:49.052557 (+  5624us) tablet.cc:701] Row locks acquired
0709 12:26:49.052639 (+    82us) write_op.cc:260] PREPARE: finished
0709 12:26:49.052903 (+   264us) write_op.cc:270] Start()
0709 12:26:49.053079 (+   176us) write_op.cc:276] Timestamp: P: 1783600009052801 usec, L: 0
0709 12:26:49.053104 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:49.055938 (+  2834us) log.cc:844] Serialized 5510 byte log entry
0709 12:26:49.557620 (+501682us) op_driver.cc:464] REPLICATION: finished
0709 12:26:49.975499 (+417879us) write_op.cc:301] APPLY: starting
0709 12:26:49.988904 (+ 13405us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:50.181033 (+192129us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:50.181163 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:50.307273 (+126110us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:50.307638 (+   365us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=28
0709 12:26:50.307688 (+    50us) write_op.cc:312] APPLY: finished
0709 12:26:50.308122 (+   434us) log.cc:844] Serialized 101 byte log entry
0709 12:26:50.308354 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:50.308526 (+   172us) write_op.cc:454] Released schema lock
0709 12:26:50.308609 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:50.309139 (+   530us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:50.309170 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4469,"reported_tablets":47,"spinlock_wait_cycles":4352,"child_traces":[["op",{"apply.queue_time_us":376937,"apply.run_cpu_time_us":7717,"apply.run_wall_time_us":353673,"num_ops":14,"prepare.queue_time_us":711754,"prepare.run_cpu_time_us":15727,"prepare.run_wall_time_us":314293,"raft.queue_time_us":198385,"raft.run_cpu_time_us":21750,"raft.run_wall_time_us":350509,"replication_time_us":500733,"thread_start_us":606151,"threads_started":4,"wal-append.queue_time_us":220020,"wal-append.run_cpu_time_us":16069,"wal-append.run_wall_time_us":2039934}]]}
I20260709 12:26:57.234917 11823 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.272095 11823 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.272784 11547 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:57.272953 11704 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:57.241730 11818 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:57.274434 11818 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.275080  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:57.276186 11818 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:57.276401  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.258116 11650 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:57.277944 11650 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:57.278113  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.278684  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.279140 11839 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:57.279770 11839 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.280174 11839 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:57.267903 11824 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:57.283474 11824 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:57.284741 11608 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.284587 11824 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.285991 11824 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:57.214170 11830 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:57.287583  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70373us
I20260709 12:26:57.268199 11268 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:57.251138 11759 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:57.269735  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.279037 11650 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.290930 11840 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:57.262207 11811 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:57.262288 11849 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:57.307062  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58519.8us
I20260709 12:26:57.243041 11825 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:57.292074  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.291718  7558 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.278925 11841 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:57.236290 11760 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.319036  7558 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 4 to 5, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.326112  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:57.306659 11849 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.347393  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:57.350630  7808 raft_consensus.cc:2368] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:26:57.358592  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:57.349912  7558 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.364867  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:57.365388  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72716.6us
W20260709 12:26:57.364904  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 202210us
I20260709 12:26:57.357439 11608 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 12:26:57.372859  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.309721 11702 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.379604 11704 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:57.384397 11704 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:57.385870 11704 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.387544 11704 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.388972 11760 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:26:57.390610  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.391577 11861 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:57.392530 11861 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:57.393714 11861 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.354712 11849 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:57.380192 11719 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:57.395903 11719 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:57.396378 11825 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:57.397202  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:57.397020 11719 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.354914 11862 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:57.404927  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.406200  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:57.406472  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.407056  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:57.407871  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:57.408655  7808 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:57.411379  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:57.412259  7809 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:57.414661  7558 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 2 to 7, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.416272  7654 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:57.418824  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:57.419445  7659 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:57.428071  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.429788  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:57.430789  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:57.431053  7658 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:57.434005  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.434512  7655 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:57.434661  7656 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:57.441522  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.443537  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:57.443696  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.439173  7809 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:57.437975  7558 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.448989  7659 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:57.449673  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:57.450126  7657 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:57.451296  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.452207  7653 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:57.441522  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:57.462666  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48254.2us
W20260709 12:26:57.463577  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.464740  7654 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:57.467489  7803 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:57.467893  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.468600  7655 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:57.353969 11846 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:26:57.405833 11862 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:57.472061  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:57.483601  7811 raft_consensus.cc:2376] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10. Re-sending same reply.
I20260709 12:26:57.473546  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.436355 11665 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 12:26:57.477224 11753 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.002s
I20260709 12:26:57.457860 11139 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:26:57.457989 11760 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:57.419839 11255 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:26:57.382225 11268 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:26:57.435541 11272 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:26:57.481932  7662 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:57.485862  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:57.488775  7808 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:57.463936  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:26:57.491904  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63972.7us
I20260709 12:26:57.492156  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.503984  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:57.458110 11830 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:57.452968 11845 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:57.531962  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.448617 11825 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:57.543699  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:57.450484 11608 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.544380  7803 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:57.484629 11836 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.003s
I20260709 12:26:57.506829  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.507380  7658 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:57.457707 11823 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.420449 11815 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:26:57.532802  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:57.457337 11821 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:57.542563  7657 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:57.566639  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.547518  7811 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:57.504706  7558 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 5 to 9, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:26:57.493937 11862 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.563014  7655 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
W20260709 12:26:57.562502  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55782.4us
W20260709 12:26:57.567713  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 103917us
I20260709 12:26:57.457566 11837 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:26:57.570258  7659 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:57.531152 11716 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:57.477316 11759 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.578944  7658 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:57.535566 11845 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.586393 11873 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.494668 11725 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.629909  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:57.631210  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.633183  7659 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:57.637881  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.579916 11821 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.651965  7655 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:57.649768  7659 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:57.578449 11760 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:57.644953  7656 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:57.661875  7810 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:57.660507  8406 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:57.660392  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.667589  7658 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
W20260709 12:26:57.639902 11870 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:26:57.734050  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:57.655468  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 108443us
W20260709 12:26:57.720927  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.722757 11871 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.723310  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.587249 11825 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.749434  7662 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:57.724234  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.724614  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.637951 11845 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.751601  8411 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:26:57.751606  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.757668  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:57.766592  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.767302  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.647902 11837 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:57.633810 11830 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:57.660821 11821 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.765355  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:57.772221  7662 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:57.770143  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.587796 11823 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.675179 11760 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.780371  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.571241 11836 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.749069 11846 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:57.803615 11139 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:57.785053 11855 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:57.804334 11139 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.640654 11608 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.812426  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:57.813503 11841 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:57.814284  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.816044 11830 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:57.816665 11830 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.817092 11830 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.818135  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:57.819530 11869 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:26:57.799621 11872 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.804448 11855 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:57.796196  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 289447us
I20260709 12:26:57.798803 11760 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.833758  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:57.772383 11837 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:57.838910  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:57.844187  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.795316 11840 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:57.851197  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.852945  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:57.854890  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.641249 11873 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.857673  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:57.859002  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.835741 11872 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.862380  8414 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 4 to 8, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.864049 11872 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.865171  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:57.865542 11704 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:57.865960 11704 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:57.866362 11704 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:57.866417 11825 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:57.871085 10979 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.871737 10979 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:57.873409 11873 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.873951 10979 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.874751 11837 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:57.876893 11864 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:57.877846  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:57.878839  7732 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
W20260709 12:26:57.878880  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51941.7us
I20260709 12:26:57.880636  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:57.881843 11867 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:26:57.882339  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:57.887111 11819 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.859s	user 0.007s	sys 0.006s
W20260709 12:26:57.888612  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52455.4us
I20260709 12:26:57.890357  7657 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:57.891738  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.891986  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:57.893282  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:57.894148  7657 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:57.894598  7657 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:57.895509  7657 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.896885  7657 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:57.897689 11859 raft_consensus.cc:2764] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:57.914638  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.876787 11837 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:57.899837  7660 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:57.899013  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.790278 11871 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.903491  7662 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 4 to 8, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:57.909192 11702 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.912696  7656 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:57.875007  7653 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:26:57.920274  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.926506  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:57.956071  7655 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:57.959376  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.966988 11139 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:57.847338 11877 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:57.919143 11842 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.000s
I20260709 12:26:57.981410  7659 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:57.983465  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:57.984899  8413 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 5.
I20260709 12:26:57.987749  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:57.988977  7734 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:57.989498  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.983513  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.991084  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:58.002002  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.970744 11872 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.987560  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.017234  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:57.970956 11846 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:58.008128  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:58.009027  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:57.981503 11139 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.051312  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:57.979403 11873 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:58.031852  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.059698  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:58.060267  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.034312  7806 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:58.061906  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.062538  7806 raft_consensus.cc:2376] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9. Re-sending same reply.
I20260709 12:26:58.062546  8407 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:58.063539  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.056349  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:58.057704  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.065079  7808 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10. Re-sending same reply.
I20260709 12:26:58.066282  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.067509  7808 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:58.068828  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:58.070178  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 187949us
I20260709 12:26:58.071559  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:58.071887  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:58.072579  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55310.5us
I20260709 12:26:58.076130  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:58.017851 11547 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.005s
I20260709 12:26:57.991036 11840 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.024463 11819 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.077263  7806 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:26:58.008145 11841 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.981752 11833 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 12:26:58.071264 11881 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:26:58.074733  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:58.101060  7655 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:26:58.102283  7734 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:57.981010 11821 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.107434  7662 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:58.085187  7732 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:57.983408 11820 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:26:58.020066 11753 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:58.056483 11828 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:58.088099  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:26:58.091188  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44085.4us
I20260709 12:26:58.096551  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.089529  8412 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:26:58.088284  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.615s (SENT) [suppressed 34 similar messages]
I20260709 12:26:58.057433 11825 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:58.074909  7737 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:58.088835 11839 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.001s
I20260709 12:26:58.087198 11700 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:58.122164  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:58.122980  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46967.8us
I20260709 12:26:58.123718  7657 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:58.124850  7730 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:58.018465 11857 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:26:58.142304 11700 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:58.125370  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 117482us
I20260709 12:26:58.176867 11811 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:58.178764 11811 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:58.180950 11879 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.181511 11879 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:58.181921 11879 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.157344 11839 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.184998  7807 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:58.129925 11861 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:58.130564  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:26:58.131385 11719 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:58.088581 11702 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:58.165444  8415 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:26:58.196072  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65614.5us
I20260709 12:26:58.076000 11873 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:58.160249  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.156114  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:58.079154  7731 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 4 to 8, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:26:58.101830 11841 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.121675 11547 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:58.072613 11139 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.171846  7809 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:58.168211  7626 leader_election.cc:400] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:26:58.175683 11700 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:58.134120 11815 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.087833 11872 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:57.991312 11877 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.189404  8405 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:58.186206  7730 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:58.094848 11840 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.098806 11819 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.196141  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.169751  7806 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:26:58.220010  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.156733 11665 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:58.201462  7728 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:58.226090  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:26:58.389470 11888 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:26:58.389981  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.122980 11845 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.392980 11845 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.228689 11833 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
W20260709 12:26:58.393790 11760 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.003s
I20260709 12:26:58.394241 11760 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:58.396751 11883 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 12:26:58.399245 11886 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
W20260709 12:26:58.400084 11704 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.003s
I20260709 12:26:58.400679 11704 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.401257 11896 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.402078 11896 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.403338 11866 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:58.405253 11891 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
I20260709 12:26:58.408823 11872 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.409751 11841 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.410282 11873 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.410435 11841 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.412979 11821 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.414278 11848 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.414825 11848 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.255540  7730 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.186733 11823 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.233968  7660 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:26:58.185480 11839 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.199046 11861 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:58.224227 11836 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.229216  7626 leader_election.cc:403] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:58.410931 11841 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.393062 11665 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.415832  7730 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:58.416183  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.417918  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2358) took 1011 ms. Trace:
W20260709 12:26:58.425134  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.805s (SENT)
I20260709 12:26:58.428210  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:58.429512  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 193484us
I20260709 12:26:58.435177  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2169) took 1014 ms. Trace:
I20260709 12:26:58.455535  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:26:58.458277  7807 rpcz_store.cc:276] 0709 12:26:57.420982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.421118 (+   136us) service_pool.cc:224] Handling call
0709 12:26:58.434956 (+1013838us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54001b1f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:58.435107 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689042688}
W20260709 12:26:58.420279  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 252532us
I20260709 12:26:58.463842  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:58.441319  7808 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:58.452414  7660 rpcz_store.cc:276] 0709 12:26:57.406129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.406242 (+   113us) service_pool.cc:224] Handling call
0709 12:26:57.899379 (+493137us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b540028c4a8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007cff5f4 00007fc006ee8227 00007fc006ee7ab8 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:58.417852 (+518473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11006720}
W20260709 12:26:58.449348  7778 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.525s (SENT)
W20260709 12:26:58.432906  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.462051  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.470886  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.408979 11824 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.122s	user 0.013s	sys 0.001s
I20260709 12:26:58.473642  7803 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:58.477977  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 347167us
W20260709 12:26:58.444509  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 356240us
W20260709 12:26:58.475144  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.479820  7700 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.717s (SENT)
W20260709 12:26:58.421054 11862 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.005s
I20260709 12:26:58.481124  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:58.483777  7660 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:58.480284  7700 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:58.485656  7660 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:58.485616  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:58.486603  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 293722us
I20260709 12:26:58.487116  8405 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:58.488327  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.488840  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.490800 11839 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.491444 11839 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.491801  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:58.493808  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.494105 11839 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.494791 11839 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.499089  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:58.500061  7656 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:58.499606 11839 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.502626 11839 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.502826 11833 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:58.503343 11833 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.496503  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:58.497969  7660 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:58.503878 11833 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:58.506820  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.507130 11894 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:26:58.510006  7656 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:58.515470  7806 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:58.516371  7811 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:58.415346 11863 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.515729  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:58.435521 11849 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.040s	user 0.013s	sys 0.000s
I20260709 12:26:58.415191 11848 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.432528 11819 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.522009 11849 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.522862 11819 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.523857 11819 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:58.524485 11819 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:58.525213  7805 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:58.431454 11716 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:58.517261  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.420301 11823 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:26:58.524538 11849 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.439934 11841 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.423347 11547 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
W20260709 12:26:58.527364  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 78059.4us
I20260709 12:26:58.431743 11896 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.520815  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:58.422518 11665 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.554574  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.435566 11704 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.462863 11139 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.429339 11840 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.434625 11877 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.436455 11608 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.253090 11815 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:58.602083  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49415.9us
I20260709 12:26:58.602949  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.604049 11824 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:58.533886  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:58.605736  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71923.6us
W20260709 12:26:58.606689  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50637.8us
I20260709 12:26:58.606623  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:58.607755 11866 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.608296 11866 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.538363  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.609377  7807 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:58.610546 11841 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.478737 11846 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:58.548921 11873 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.611152 11841 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.611814 11873 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:58.612883 11823 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:58.613420 11823 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:58.431627 11871 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.548796 11882 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:26:58.513970 11821 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:58.559307  7812 raft_consensus.cc:606] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.518641 11872 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.518309 11845 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:26:58.564512 11844 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 12:26:58.567399 11868 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:26:58.568288  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.568483 11719 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:58.569377 11887 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:26:58.615965 11872 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.634459 11719 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.613827 11823 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:58.635113 11719 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:58.625830  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2368) took 1127 ms. Trace:
I20260709 12:26:58.636211  7654 rpcz_store.cc:276] 0709 12:26:57.489340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.489511 (+   171us) service_pool.cc:224] Handling call
0709 12:26:57.489840 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.788655 (+298815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.788710 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:57.788772 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:57.788820 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.615951 (+827131us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400116554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:58.615997 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.616153 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675777408}
I20260709 12:26:58.633402  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.637619 11872 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.518836 11862 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:58.572558  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.580396  7730 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:58.640347  7728 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:58.569813 10979 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:58.585672  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.641714 11862 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.641875  7811 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:58.643095  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.643778  7811 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:58.644819  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.645483  7811 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:58.646474  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.590524 11650 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:58.592722  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2167) took 1182 ms. Trace:
W20260709 12:26:58.560717 11818 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.284s	user 0.010s	sys 0.005s
I20260709 12:26:58.651248  7805 rpcz_store.cc:276] 0709 12:26:57.410645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.410950 (+   305us) service_pool.cc:224] Handling call
0709 12:26:57.439860 (+ 28910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.525662 (+1085802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.592436 (+ 66774us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003c50d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:58.592649 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495631616}
I20260709 12:26:58.653348  7805 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:58.569686 11833 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.656718 11833 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.601172 11272 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:58.600503  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.658370  7809 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:58.603528 11759 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.605091 11824 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:58.659917 11824 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:58.657644 11823 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.661144 11824 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.610488  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.662967 11700 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.664157  7807 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:26:58.561501 11760 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.531677 11716 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.666133 11700 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.568293 11858 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:26:58.668865  7733 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:58.608496  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.618592  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2164) took 1212 ms. Trace:
I20260709 12:26:58.438819 11879 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:58.670300  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61870.1us
I20260709 12:26:58.672029 11879 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.525909 11819 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.611152 11846 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.672817  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.673007 11846 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:58.672065  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.973s (SENT)
I20260709 12:26:58.673743  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.673807  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.670411 11818 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.675439 11877 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.675889 11877 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.676411 11877 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.679474 11608 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:58.679879  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.680189 11608 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.680801 11608 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:58.681903  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.682783  7807 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:58.419183 11861 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.683451 11759 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:58.684314 11759 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:58.684799 11759 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.684367 11846 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.685168 11759 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:58.421516 11890 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 12:26:58.613797 11841 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.674894  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:58.656852 11830 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.000s
W20260709 12:26:58.694191  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 200472us
I20260709 12:26:58.695746  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.670251  7810 rpcz_store.cc:276] 0709 12:26:57.405775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.407363 (+  1588us) service_pool.cc:224] Handling call
0709 12:26:58.618340 (+1210977us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54000681d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:58.618516 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499824896}
I20260709 12:26:58.627461  8409 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:26:58.662833 11824 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:58.706679  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:58.708002  7805 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:26:58.711242  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 219507us
I20260709 12:26:58.698088  7806 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:26:58.704305  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.699283  7811 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:58.712417  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:58.643935 11837 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.714s	user 0.004s	sys 0.008s
I20260709 12:26:58.730994 11760 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:58.731163 11856 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:26:58.731587 11760 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:58.732121 11760 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:58.732311  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 252499us
I20260709 12:26:58.725123 11841 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.736776 11855 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.737421 11841 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.530085 11863 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.714162  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:26:58.740898  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44688.9us
I20260709 12:26:58.747701  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.740949  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:26:58.748142  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.693941 11823 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.705111 11704 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.796983  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.712682 11872 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.749094  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:58.744009  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:26:58.795542  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:26:58.801831  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57942.9us
I20260709 12:26:58.715379 11700 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.801985  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52981.6us
I20260709 12:26:58.797470  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:58.830861 11815 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:58.824594 11871 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:58.831751 11871 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.831887 11842 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:58.832543 11893 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:58.832857  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.833323 11893 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:58.830929  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.834111 11893 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:58.750098 11844 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:26:58.793473 11855 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:58.835749 11855 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:58.836632  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88607.4us
I20260709 12:26:58.837033  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:26:58.837173  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 96305.7us
I20260709 12:26:58.837541  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:26:58.835827  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.832214 11871 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:58.741940 11879 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:58.731181 11908 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:58.793685 11873 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:58.841140  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.711812 11830 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.675892 11818 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:58.836400 11855 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:58.851411 11841 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.851969 11854 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:58.836681 11846 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:58.855824 11907 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:58.796770 11704 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.808279 11866 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:58.837743 11700 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:58.804472 11872 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.865911  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.798398 11896 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:26:58.848246 11882 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:58.887176  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65482.6us
I20260709 12:26:58.824873 11624 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:26:58.849226 11879 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:58.889797  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 275051us
I20260709 12:26:58.860057 11873 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:58.892594  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55144.8us
I20260709 12:26:58.846343  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:58.941054  7806 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:58.896133  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 280285us
W20260709 12:26:58.901724  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.901662  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:26:58.910097  7700 leader_election.cc:336] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.950s (SENT)
I20260709 12:26:58.943979  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:58.952955  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:58.940083  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.894836 11704 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.887526 11849 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:26:58.973922  7661 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:26:58.897535 11872 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.896378 11844 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:58.975818  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69073.7us
I20260709 12:26:58.900178 11854 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:26:58.985445  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83889.8us
I20260709 12:26:58.986537  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:58.987035  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.987686  8415 raft_consensus.cc:2376] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10. Re-sending same reply.
I20260709 12:26:58.988237  8405 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:58.988478  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.988636  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.989259  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.990285 11901 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:58.991744  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.992010 11901 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:58.994020  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:58.991026  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.994513  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:26:58.979331  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.980022  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:58.966799  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:58.928042 11841 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:58.982719  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.000970  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.001479  7736 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:59.001921  7656 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
W20260709 12:26:58.978255  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 175155us
I20260709 12:26:59.000166  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.002301  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2096) took 1369 ms. Trace:
I20260709 12:26:59.000979  7731 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:58.984344  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:59.006050  7661 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:58.958068 11873 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:59.015460  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75546.2us
I20260709 12:26:59.002749  7659 rpcz_store.cc:276] 0709 12:26:57.632484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.765760 (+133276us) service_pool.cc:224] Handling call
0709 12:26:57.766144 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.990817 (+224673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.002040 (+1011223us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540028e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:26:59.002239 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494627200}
I20260709 12:26:59.013468  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.005530  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:59.018790  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.019550  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:59.020483  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.016592  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:58.970067 11882 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:26:59.040438  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 130388us
I20260709 12:26:59.027441  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:59.028064  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.026579  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:59.057330  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:59.004307 11753 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:59.068214  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.086005  7734 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:26:59.121358  7731 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:26:59.132717  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.145181  7731 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:59.166993  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:59.180706  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.186239  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2627) took 1149 ms. Trace:
W20260709 12:26:59.189437  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 172923us
I20260709 12:26:59.190420  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.022408 11904 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:59.191983  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.186827  7808 rpcz_store.cc:276] 0709 12:26:58.037222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.068347 (+ 31125us) service_pool.cc:224] Handling call
0709 12:26:59.185969 (+1117622us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:59.186159 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853588864}
I20260709 12:26:59.208781  7659 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:26:59.213541  7806 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:26:59.209241  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.214556  7809 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:26:59.215271  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:59.215708  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 684) took 1299 ms. Trace:
I20260709 12:26:59.216181  8409 rpcz_store.cc:276] 0709 12:26:57.888309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.891698 (+  3389us) service_pool.cc:224] Handling call
0709 12:26:59.187873 (+1296175us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:59.188001 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436820736}
I20260709 12:26:59.027813 11901 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.060226 11862 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:59.004233 11857 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:26:58.983351 11716 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:59.221676  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 233568us
W20260709 12:26:59.208380  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 201576us
I20260709 12:26:59.057789 11846 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.027660 11844 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:59.238529  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:59.252328  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:59.253090  7659 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:59.228247  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:59.253014  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:26:59.254599  7808 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:59.219367 11608 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.002s
I20260709 12:26:59.068418 11879 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.260107  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.264822  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:26:59.264171  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.072177 11836 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 12:26:59.212095 11904 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:26:59.284171  7729 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:59.229321 11139 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:59.311625  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.238946 11863 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:59.311879  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.223975 11907 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:59.255487 11725 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:59.265573 11849 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:59.330662  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50788 (request call id 21) took 1309 ms. Trace:
I20260709 12:26:59.332048  7812 rpcz_store.cc:276] 0709 12:26:58.028568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.028769 (+   201us) service_pool.cc:224] Handling call
0709 12:26:59.318864 (+1290095us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400373cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:59.319062 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992738560}
W20260709 12:26:59.333828  7807 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.004s
I20260709 12:26:59.334244  7807 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
W20260709 12:26:59.336802  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 331345us
W20260709 12:26:59.338156  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 147931us
I20260709 12:26:59.270409 11862 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:59.343488  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.342229  7808 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
W20260709 12:26:59.345782  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98159.2us
I20260709 12:26:59.348304  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:26:59.354560  8409 raft_consensus.cc:2376] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10. Re-sending same reply.
W20260709 12:26:59.358191  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.320370 11820 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:59.328680 11760 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.003s
I20260709 12:26:59.377400 11760 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.264398 11846 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.312749 11919 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:59.379927 11919 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:59.383442 11846 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.327148 11879 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:59.265499 11608 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.389110 11608 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.393160 11700 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.328647 11139 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:26:59.394759  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 179583us
I20260709 12:26:59.396531 11139 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:59.396589  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:59.397958  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:26:59.327775 11849 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.400946  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:59.402233  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.304648 11645 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 12:26:59.414873  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56767.2us
I20260709 12:26:59.416889  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.426420  7737 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:26:59.347524 11929 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.431627  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.430898  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.352334 11907 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:26:59.374825 11820 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.386075 11760 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:59.443321  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.414532 11849 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:59.467154  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:59.483421  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.484232  7654 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:59.485096  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.492462  7737 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:59.428107 11608 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:59.478652  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.428886 11846 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:59.357551 11839 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:58.990154 11854 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.260874 11901 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.370543 11844 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.328909 11882 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.661250  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:26:59.523761  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.674805  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:59.567830  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.566780  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.437165 11909 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:59.546932  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.715338  7732 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:59.745232  7654 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:26:59.573943  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86360.8us
I20260709 12:26:59.464419 11919 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:59.747644  7659 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:26:59.472926 11812 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.237s
W20260709 12:26:59.550669 11255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.000s	sys 0.011s
W20260709 12:26:59.685972 11719 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.050s	user 0.013s	sys 0.000s
W20260709 12:26:59.755771  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.687817 11922 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 12:26:59.481617 11820 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:59.681404 11910 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
W20260709 12:26:59.541323 11916 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
W20260709 12:26:59.611230 11759 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.926s	user 0.009s	sys 0.006s
I20260709 12:26:59.610543 11862 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.512229 11716 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:59.668538 11927 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:59.798393  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 137299us
I20260709 12:26:59.797070  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.763109  7806 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:26:59.803591  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:26:59.804425  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.623355 11895 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:59.810699  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.018s
W20260709 12:26:59.812384  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.000s	sys 0.018s
I20260709 12:26:59.813908  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:26:59.660019 11938 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:26:59.784720 11877 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.108s	user 0.006s	sys 0.008s
W20260709 12:26:59.785131 11255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.000s	sys 0.013s
I20260709 12:26:59.817086  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:26:59.828111 11877 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.823421  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.831087  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.831825 11760 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:26:59.442720 11926 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.332s
I20260709 12:26:59.832774 11760 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.826850  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:26:59.829335 11866 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:59.833024  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.837920 11760 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:59.835183  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:26:59.851616  7810 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:59.839841  7730 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.033s	sys 0.003s
I20260709 12:26:59.444449 11929 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.840618  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:26:59.854031  7730 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.033s	sys 0.003s
I20260709 12:26:59.853971  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.854854  7736 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:26:59.852793  7737 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:59.624919 11906 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.405s
I20260709 12:26:59.811452 11895 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:59.716025 11941 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 12:26:59.751719 11839 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.806494 11719 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.837344 11877 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.877852  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:26:59.886313  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:59.863157 11824 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.151s	user 0.014s	sys 0.002s
I20260709 12:26:59.836119 11866 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.850920 11716 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:59.812914 11759 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:59.865029  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61523.8us
I20260709 12:26:59.886605  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.891701  7654 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:59.860394 11272 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:26:59.905555  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:26:59.459270 11904 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.915937 11904 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:59.919641  8412 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:59.920090 11941 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:59.920540 11941 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:59.920598  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.921981  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.922307  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 803) took 1117 ms. Trace:
I20260709 12:26:59.922710  7731 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:59.905565 11866 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:59.924278  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.925705  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.927378 11937 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:59.927891 11937 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:59.929342 11719 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.930588  7728 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:26:59.931318 11912 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.408s	user 0.001s	sys 0.000s
I20260709 12:26:59.932461  7656 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:26:59.911693 11913 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.722s	user 0.001s	sys 0.005s
I20260709 12:26:59.934191 11759 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.914724  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2110) took 1432 ms. Trace:
I20260709 12:26:59.908286  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.936414 11942 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:59.936923 11942 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:26:59.937350 11942 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:59.937219  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.938129  7736 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:59.928916 11937 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.934988  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.905184 11645 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:59.895190  7658 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:26:59.926878  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.939106  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:26:59.948946  7736 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:59.950122 11830 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:26:59.950439  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:26:59.951946 11918 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.916s	user 0.002s	sys 0.000s
I20260709 12:26:59.952301  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.953940  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.810065 11919 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.945773 11903 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.956043  7659 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:26:59.956359 11903 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.957382 11919 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:26:59.959303 11645 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:26:59.702750 11907 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:26:59.910044 11704 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.926s	user 0.008s	sys 0.005s
I20260709 12:26:59.962319  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:26:59.863099 11927 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:26:59.964763  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.951994  7732 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:26:59.921605 11941 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:26:59.965334  7729 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:26:59.998214  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.886819 11846 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:26:59.932485  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.965945  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:26:59.967280  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.942536  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.932528  7806 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:27:00.000643  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.002149  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.002933  7806 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:59.935456  7657 rpcz_store.cc:276] 0709 12:26:58.482646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.482771 (+   125us) service_pool.cc:224] Handling call
0709 12:26:58.483333 (+   562us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:58.483713 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.483741 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:58.518510 (+ 34769us) log.cc:844] Serialized 52 byte log entry
0709 12:26:58.894128 (+375618us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:59.006130 (+112002us) log.cc:844] Serialized 14 byte log entry
0709 12:26:59.007105 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.007198 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.914421 (+907223us) raft_consensus.cc:1685] finished
0709 12:26:59.914454 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.914639 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":813,"prepare.queue_time_us":371395,"prepare.run_cpu_time_us":1514,"prepare.run_wall_time_us":8355,"raft.queue_time_us":31532,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":21128,"thread_start_us":446778,"threads_started":2,"wal-append.queue_time_us":830364}
I20260709 12:26:59.922880  8412 rpcz_store.cc:276] 0709 12:26:58.805009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.805252 (+   243us) service_pool.cc:224] Handling call
0709 12:26:59.921996 (+1116744us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400321c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:26:59.922214 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468131584}
I20260709 12:26:59.988684 11608 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:00.007136 11818 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:26:59.983935  8409 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:27:00.007995  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:26:59.984288  8408 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:27:00.009142 11915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.001s	sys 0.019s
W20260709 12:27:00.009783 11915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.001s	sys 0.019s
I20260709 12:26:59.938936 11815 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:26:59.983956  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.011680  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.012861  7733 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:00.015571  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.015736  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.839067 11938 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.017006  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.017728  7737 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:59.940670 11833 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:59.959029 11903 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.018505  7736 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:00.019202  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.045606  7731 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:00.005678  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.003934  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:00.023967  7734 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:00.007995  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.960542 11645 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.024130  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118680us
I20260709 12:27:00.024004  7729 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:59.939368 11934 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.059937  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56079.8us
I20260709 12:27:00.066349  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:00.067078  8407 raft_consensus.cc:2376] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5. Re-sending same reply.
I20260709 12:27:00.056751  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 818) took 1123 ms. Trace:
I20260709 12:27:00.076625  7810 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:00.077690  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.078299  7810 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.081669  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:26:59.988328 11919 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.099365  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.008s	sys 0.012s
W20260709 12:27:00.099908  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.008s	sys 0.012s
W20260709 12:27:00.100646  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.07412e+06us
I20260709 12:27:00.101617  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.019331 11759 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.026276 11819 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:00.128865  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67176.1us
I20260709 12:27:00.025884 11719 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.064043 11650 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.416s	user 0.013s	sys 0.004s
I20260709 12:27:00.142058  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:00.142174  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:00.041585 11833 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:00.156710  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:00.157666  7657 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:00.059484 11716 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:59.996017 11704 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.126338 11268 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:00.163832  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.164044  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:00.119251 11939 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:00.140168 11893 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.297s	user 0.016s	sys 0.000s
I20260709 12:27:00.168996  7804 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:00.186537 11954 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.216679 11840 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.217474 11954 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.218730 11901 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.169871  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.128022 11941 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.104758 11952 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:00.224690  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.226563  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:00.227399  7657 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:00.174890  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:00.165979 11849 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.653s	user 0.004s	sys 0.009s
I20260709 12:27:00.007038 11608 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.188119 11895 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.249737 11895 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.251178 11268 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:00.251711 11268 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:00.117003 11824 consensus_queue.cc:1175] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 7 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: [6.8-6.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:00.252167 11268 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:00.144132 11903 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:00.254941 11903 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.257424 11903 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.091789 11818 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 12:27:00.264040 11839 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.129361 11919 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.217697 11840 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:26:59.987318 11272 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.155853 11938 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.147378  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.214993 11894 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.232852  7806 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:27:00.287715  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:00.295253  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.301944  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.242491  7662 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:00.242486  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49909.4us
I20260709 12:27:00.241475 11936 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.248786  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.145923 11759 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.160571 11719 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.271090  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.160908 11645 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:00.215401 11939 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:00.272763  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.236579  7654 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:00.331620 11877 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.332477 11877 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.334360 11953 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.335280 11953 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.341327  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:00.270524  7810 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:00.305991  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.241797 11952 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.220142 11819 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:27:00.282718 11871 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.427s	user 0.007s	sys 0.010s
I20260709 12:27:00.108630 11934 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.317535  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:00.189528 11846 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.228739 11879 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:27:00.271875 11665 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:00.323057  7806 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:00.343338  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:00.306270 11938 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:27:00.324256  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.270614 11849 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.352597  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46720.1us
I20260709 12:27:00.353469 11819 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:27:00.353955 11871 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.178285 11927 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.344612 11952 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:00.357121 11934 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.357661 11759 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.302274 11894 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.361495 11855 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.468s	user 0.014s	sys 0.001s
I20260709 12:27:00.362809 11954 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.320489 11936 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.341771 11848 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.819s	user 0.017s	sys 0.000s
I20260709 12:27:00.220503 11893 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.342221 11939 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.373560  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:00.376987  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 310100us
I20260709 12:27:00.422248  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:00.347309 11820 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.529s	user 0.004s	sys 0.008s
W20260709 12:27:00.380363  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 249579us
I20260709 12:27:00.452003  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:00.381882  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:00.384069  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.384568  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.386082  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:00.453486  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.409951  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:00.411681  7661 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:00.383131 11956 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.397198 11934 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:00.453655  7806 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:00.416671 11896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.015s	sys 0.012s
W20260709 12:27:00.456878  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75097.7us
I20260709 12:27:00.399531 11819 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:00.408850 11929 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.391722 11938 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:00.325598 11833 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.497644  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:00.384533 11665 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.506397  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.456895 11818 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.492686 11956 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.493880 11896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.225s	user 0.016s	sys 0.013s
I20260709 12:27:00.411206 11855 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.514102 11272 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.514304 11704 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.514833 11961 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:27:00.516389 11942 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
I20260709 12:27:00.493763 11624 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:00.519655 11929 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:00.520851 11893 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:00.513552 11945 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:27:00.539562  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.487617 11914 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:00.459614 11820 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.580276  7655 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 7: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:00.421330 11848 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.551071  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 200604us
I20260709 12:27:00.506804 11934 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:00.531168 11719 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:00.582293 11955 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:27:00.583878 11665 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.584966 11938 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.585165 11819 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.586937 11873 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.529s	user 0.006s	sys 0.009s
W20260709 12:27:00.591902 11911 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:00.615756 11929 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.585309 11855 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.621368 11961 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:00.618981 11914 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.667968 11914 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.588342 11936 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.586977 11956 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.628737  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:00.672466  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43833us
I20260709 12:27:00.634218 11871 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.674005 11871 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:00.674456 11871 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.608374 11624 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.674857 11871 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:00.676793 11624 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.640295 11894 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.593552 11833 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.635867 11820 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.640578  7737 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:00.640877  8405 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:00.641274 11268 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.642468  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 658) took 1644 ms. Trace:
W20260709 12:27:00.630824 11963 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:27:00.646775 11872 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.642s	user 0.016s	sys 0.000s
W20260709 12:27:00.682314  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101254us
I20260709 12:27:00.671718  7806 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:00.690665  7654 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:00.549427 11818 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.642637 11866 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.719s	user 0.015s	sys 0.000s
W20260709 12:27:00.581441  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 129461us
I20260709 12:27:00.684746  7812 rpcz_store.cc:276] 0709 12:26:58.998132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.401681 (+403549us) service_pool.cc:224] Handling call
0709 12:26:59.443371 (+ 41690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.443637 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.443670 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.466519 (+ 22849us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.823057 (+356538us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:59.857799 (+ 34742us) log.cc:844] Serialized 14 byte log entry
0709 12:26:59.858217 (+   418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.858335 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.642151 (+783816us) raft_consensus.cc:1685] finished
0709 12:27:00.642181 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.642383 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":64093,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":2787,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":14988,"thread_start_us":258264,"threads_started":1,"wal-append.queue_time_us":618471}
W20260709 12:27:00.689357  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 191865us
I20260709 12:27:00.695492  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:00.696480  7812 raft_consensus.cc:606] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.696658 11839 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:00.696899  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:00.697336 11839 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.698721  7812 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:27:00.640219 11848 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.629741 11919 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.593482 11962 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:00.700709  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.639808 11970 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.709383  7806 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:00.709383  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:00.707247  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2181) took 1016 ms. Trace:
I20260709 12:27:00.668110 11961 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.711549  7812 rpcz_store.cc:276] 0709 12:26:59.686596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.697899 (+1011303us) service_pool.cc:224] Handling call
0709 12:27:00.698331 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.702473 (+  4142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.703189 (+   716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.773644  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:00.710829 11973 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.774839  7812 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:27:00.774996 11849 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:00.775466 11944 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.339s	user 0.004s	sys 0.000s
I20260709 12:27:00.775765  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2183) took 1086 ms. Trace:
I20260709 12:27:00.776482  7812 rpcz_store.cc:276] 0709 12:26:59.689211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.725079 (+1035868us) service_pool.cc:224] Handling call
0709 12:27:00.774421 (+ 49342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.775457 (+  1036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.775698 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.777202  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 655) took 1780 ms. Trace:
I20260709 12:27:00.607832 11844 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:27:00.607071 11942 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.778410  7812 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:00.780175  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2184) took 1087 ms. Trace:
I20260709 12:27:00.780596  7812 rpcz_store.cc:276] 0709 12:26:59.692543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.777440 (+1084897us) service_pool.cc:224] Handling call
0709 12:27:00.778163 (+   723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.779855 (+  1692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.780078 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.781597 11942 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.782142  7812 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:00.782976  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2187) took 1036 ms. Trace:
I20260709 12:27:00.783360  7812 rpcz_store.cc:276] 0709 12:26:59.746775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.781509 (+1034734us) service_pool.cc:224] Handling call
0709 12:27:00.781926 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.782681 (+   755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.782911 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.784682  7812 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:00.785310  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2188) took 1006 ms. Trace:
I20260709 12:27:00.785429 11818 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.665506 11873 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.777693  7808 rpcz_store.cc:276] 0709 12:26:58.997258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.352949 (+355691us) service_pool.cc:224] Handling call
0709 12:26:59.353364 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.374429 (+ 21065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.374483 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.407356 (+ 32873us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.754600 (+347244us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:59.754715 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.754840 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.631686 (+876846us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:00.776838 (+145152us) raft_consensus.cc:1685] finished
0709 12:27:00.776863 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.777126 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4021,"raft.queue_time_us":46920,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":26067,"spinlock_wait_cycles":6819584,"thread_start_us":229033,"threads_started":1,"wal-append.queue_time_us":719649}
I20260709 12:27:00.790022  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.789642  7812 rpcz_store.cc:276] 0709 12:26:59.779356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.784121 (+1004765us) service_pool.cc:224] Handling call
0709 12:27:00.784504 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.785059 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.785252 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.790994  7808 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:27:00.791249  7734 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:00.792209  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2182) took 1024 ms. Trace:
I20260709 12:27:00.792825  7806 rpcz_store.cc:276] 0709 12:26:59.687818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.708687 (+1020869us) service_pool.cc:224] Handling call
0709 12:27:00.709107 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.711903 (+  2796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.712111 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.632086 11934 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.707903 11962 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:00.773404 11909 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.972s	user 0.013s	sys 0.001s
I20260709 12:27:00.796455  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:00.772528 11719 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:00.652033 11952 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.700788 11872 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.809204  7660 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:00.705427 11866 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.818207  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:00.804636 11818 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:00.826771  7653 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:00.803107 11962 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:00.809829 11909 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:00.825793 11719 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.833130  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2635) took 2746 ms. Trace:
I20260709 12:27:00.800997 11939 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:00.836306  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.889629  7812 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.890637  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.891407  7806 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:00.837869  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:00.838749 11818 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.841146 11914 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.898797 11914 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.905871 11823 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.881735 11839 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.908787 11877 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.909348 11939 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.909267 11962 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.882707  7803 rpcz_store.cc:276] 0709 12:26:58.086597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.477888 (+391291us) service_pool.cc:224] Handling call
0709 12:26:58.478241 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.516285 (+ 38044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.516318 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.531654 (+ 15336us) log.cc:844] Serialized 26 byte log entry
0709 12:26:58.884727 (+353073us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:58.957318 (+ 72591us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.987415 (+ 30097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.987491 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.458500 (+1471009us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b18002d0b20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:00.826231 (+367731us) raft_consensus.cc:1685] finished
0709 12:27:00.832860 (+  6629us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.833051 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30352,"prepare.queue_time_us":74626,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":35539,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":37224,"spinlock_wait_cycles":18984832,"thread_start_us":334405,"threads_started":1,"wal-append.queue_time_us":454182}
I20260709 12:27:00.912034  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.912923  7803 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:00.913818 11871 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.914270  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.915958 11962 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.916513 11866 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:00.917610  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.918390 11873 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.919270 11873 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:00.919878 11873 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:00.921564 11818 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.915061  7803 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:00.929355  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.930433  7803 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:00.931634  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.932654  7803 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:00.933516  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.850045 11953 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.859248  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2126) took 1793 ms. Trace:
I20260709 12:27:00.860917 11942 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.943092  7803 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:00.868651 11952 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.943560 11942 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:00.871274 11904 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.955s	user 0.015s	sys 0.000s
I20260709 12:27:00.945544  7807 raft_consensus.cc:2410] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:27:00.872735 11268 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.873881 11929 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:00.947073  7657 rpcz_store.cc:276] 0709 12:26:59.065710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.488757 (+1423047us) service_pool.cc:224] Handling call
0709 12:27:00.489168 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.489578 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.489605 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.490107 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.711570 (+221463us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:00.711626 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.711706 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.858924 (+147218us) raft_consensus.cc:1685] finished
0709 12:27:00.858952 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.859140 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4292,"raft.queue_time_us":3615,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6472,"thread_start_us":164382,"threads_started":1,"wal-append.queue_time_us":344830}
I20260709 12:27:00.881680  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:00.949124  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:00.949606  8412 pending_rounds.cc:85] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 9
I20260709 12:27:00.949781  7657 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:00.885869  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:00.888725  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.891896  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:00.951315  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.951661  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:00.892840  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55127.3us
W20260709 12:27:00.954003  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.954646 11929 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:00.954777  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:00.900113  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:00.957221  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:00.957683 11952 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.958112 11952 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:00.958184  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62864.8us
I20260709 12:27:00.902714  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:00.825465 11970 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:27:00.959311  8409 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:27:00.959916 11970 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:27:00.960496 11970 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:00.961634 11970 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:00.903060  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.902693 11937 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:00.964408  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.965776  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.848829 11872 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:00.937088  7729 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:00.939743 11840 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:00.946656 11942 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:00.949589  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69277.8us
I20260709 12:27:00.884824  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.950287  8412 pending_rounds.cc:107] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:27:00.984473  8412 raft_consensus.cc:2889] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.068730  7730 rpcz_store.cc:276] 0709 12:26:58.933882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.934478 (+   596us) service_pool.cc:224] Handling call
0709 12:26:58.934939 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.935252 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:59.856354 (+921102us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.056406 (+200052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.056461 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:00.056507 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.056569 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.056692 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1050038,"prepare.run_wall_time_us":54,"raft.queue_time_us":34372,"thread_start_us":1056237,"threads_started":2}
I20260709 12:27:00.894192  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.955328 11929 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:00.955468  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:00.955698  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:00.956223  8407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:27:00.996299  8407 raft_consensus.cc:606] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:00.996793  8407 raft_consensus.cc:614] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.956355  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:00.997865  8415 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:27:00.956835  8414 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:27:00.958309 11824 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.958535 11952 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.968097  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:00.936461 11811 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:00.963114 11970 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:00.990428  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:00.986434  8406 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:01.016330  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.011142  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:00.967026  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:01.017766  7808 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:01.012156  8409 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:27:01.009984  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:01.021059  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:00.923132 11873 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:01.024210  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.028616  7803 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:01.016108  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.026466  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.029536  7810 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:27:01.029695  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.030370  7803 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:01.031437  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.032210  7803 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:01.030583  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.035125  7657 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:01.036316  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.039986  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.015005 11856 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:01.052903  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.012691 11811 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:01.051097  7808 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:01.026013 11837 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:01.025033 11973 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.056991  7657 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:01.037736 11903 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:01.061960  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.079517  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.080756  7657 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:01.081921  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.052145 11873 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:01.073014  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.026292 11824 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:27:01.091290  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.092222  7810 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.053471 11927 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:27:01.086771  7626 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45918 dropped due to backpressure: service queue is full with 50 items
I20260709 12:27:01.052187 11841 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:01.073814  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.088572  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.088662  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.054868 11856 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:01.054543 11962 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:27:01.084132 11903 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.139931  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.137949  7808 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:01.084553 11854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.007s	sys 0.037s
I20260709 12:27:01.085274 11837 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:01.149881  7657 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:01.137533 11841 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.090310 11914 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.137338 11901 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.016s	sys 0.061s
I20260709 12:27:01.143535  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:01.165048  7808 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:01.097671 11931 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:01.151329  7803 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:01.144140 11973 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.143730 11856 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:01.151928 11854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.008s	sys 0.037s
I20260709 12:27:01.101893 11873 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.177690  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.143774 11927 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:01.059125 11929 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.148358 11871 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:01.156948 11833 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:01.165298 11901 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.017s	sys 0.062s
I20260709 12:27:01.216054  7803 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:01.156038 11903 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:01.191962  7809 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:27:01.163137 11841 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:01.227583 11979 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:27:01.229277 11873 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:01.220794 11983 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:01.230242  8415 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:01.231340 11837 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.232266  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:01.232733  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:01.233343 11946 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.925s	user 0.002s	sys 0.000s
I20260709 12:27:01.233661  8406 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:27:01.231918  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2118) took 2219 ms (client timeout 1772 ms). Trace:
I20260709 12:27:01.234946  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 659) took 2236 ms. Trace:
W20260709 12:27:01.235097  7656 rpcz_store.cc:269] 0709 12:26:59.012277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.013136 (+   859us) service_pool.cc:224] Handling call
0709 12:27:01.231671 (+2218535us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b540002fb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.231813 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105392896}
I20260709 12:27:01.235883  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2387) took 1170 ms. Trace:
I20260709 12:27:01.236363  7737 rpcz_store.cc:276] 0709 12:27:00.065675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.089151 (+ 23476us) service_pool.cc:224] Handling call
0709 12:27:01.235631 (+1146480us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002f1554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.235784 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830305152}
W20260709 12:27:01.237573  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.238298  7661 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2119) took 2221 ms (client timeout 1751 ms). Trace:
W20260709 12:27:01.238708  7661 rpcz_store.cc:269] 0709 12:26:59.017406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.017668 (+   262us) service_pool.cc:224] Handling call
0709 12:27:01.238056 (+2220388us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540028e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.238211 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3177890688}
I20260709 12:27:01.235404  7811 rpcz_store.cc:276] 0709 12:26:58.998473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.418883 (+420410us) service_pool.cc:224] Handling call
0709 12:26:59.420078 (+  1195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.441162 (+ 21084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.441220 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.507425 (+ 66205us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.821231 (+313806us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:00.269059 (+447828us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.270151 (+  1092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.270245 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.232636 (+962391us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:01.234634 (+  1998us) raft_consensus.cc:1685] finished
0709 12:27:01.234738 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.234883 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36308,"prepare.queue_time_us":175174,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":1036,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":44707,"spinlock_wait_cycles":6701440,"thread_start_us":621715,"threads_started":2,"wal-append.queue_time_us":649081}
I20260709 12:27:01.190192 11931 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:01.247979  7810 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:01.248370  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:01.162878 11811 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.250029 11811 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.254947 11845 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:01.256153 11845 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:01.257402 11845 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.265734 11863 raft_consensus.cc:993] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:01.266393 11842 raft_consensus.cc:1081] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:01.246964 11907 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:01.267704 11928 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:01.256261  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1008) took 1033 ms. Trace:
I20260709 12:27:01.269779  7811 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:01.271852  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.268368  8405 rpcz_store.cc:276] 0709 12:27:00.222628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.223732 (+  1104us) service_pool.cc:224] Handling call
0709 12:27:01.256009 (+1032277us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.256193 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756029312}
I20260709 12:27:01.167164 11914 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.135977  7626 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45918 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2131)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2137)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2144)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2145)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2149)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2138)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2130)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2148)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 747)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 748)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2379)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2381)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2388)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 749)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 750)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2129)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2132)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2136)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2390)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2393)
I20260709 12:27:01.280062  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.281349  7811 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:01.243681 11840 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.283811  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.286007  7803 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:01.286197  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.267585 11907 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.181028 11973 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.288746  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.290115  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.290850  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.291581  7811 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:01.268169 11928 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:01.292526  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.293358  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.296738  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.297675  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.297122  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.301440  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.302191  7810 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.294737 11608 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:01.294437 11934 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:01.299551  7803 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:01.308810  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.293172 11873 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:01.307192  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:01.299021  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.312491  7808 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:01.297932  7811 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:01.303553 11877 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:01.313413  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.316359 11877 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:01.317051 11823 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:01.317586 11823 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:01.317394 11877 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:01.318742  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.318826 11907 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:01.316118 11882 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:01.298415 11938 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:01.320335  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:01.318568 11823 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.298359 11872 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:01.289644 11840 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.320748 11975 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:01.336858 11975 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:01.338124 11975 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:01.339594  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.343935  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.344376 11914 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:01.325716 11931 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.319581 11608 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:01.347646 11608 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:01.348635 11608 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.313767 11873 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.351886 11873 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.310786 11716 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:27:01.356784  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.335549  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.358915  7803 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:01.326920 11981 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:27:01.303574 11927 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:01.327117 11982 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:27:01.372682 11842 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.372545  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.335435 11871 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:27:01.377496  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.392299  7732 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.378760  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:01.390445  7810 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:01.372568  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:01.385115  8405 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:01.331133 11938 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.396376  8405 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:01.316146 11962 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:27:01.400545 11842 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.402834 11973 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:01.403244  7627 connection.cc:256] RPC call timeout handler was delayed by -2.08683202300017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 12:27:01.405932  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.408416  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.407125  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.421449  7808 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
W20260709 12:27:01.410172  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97889.5us
I20260709 12:27:01.411538  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.424496  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.411125  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.425357  7728 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:01.426337  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.414462  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:01.416467  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:01.434978  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.436877  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.441228  7736 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:01.441457  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:01.334810 11907 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:27:01.445144  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.408525 11938 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.405022 11753 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:01.446638  7737 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:01.427878  8405 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.445125  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.458853  8405 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:01.458890  7728 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:01.462076 11837 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:01.474614  7732 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:01.451830  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:01.456650  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:01.447588  7803 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:01.403892 11856 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.479458  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:01.484146  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
I20260709 12:27:01.489477  7734 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:01.490079  7734 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:01.490465  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.484746  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.485364 11914 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:27:01.516675  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.554153  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.423923 11871 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:01.568931  7810 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:01.569777  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:01.571249  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.449501 11953 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:01.476197 11904 raft_consensus.cc:2764] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:27:01.584043  7808 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:27:01.587060  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.586025  7808 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:01.591379  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.595429  7808 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:27:01.599399  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:01.600409  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 745) took 2359 ms. Trace:
I20260709 12:27:01.552891 11907 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:27:01.604009  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2120) took 2585 ms (client timeout 1588 ms). Trace:
W20260709 12:27:01.604565  7660 rpcz_store.cc:269] 0709 12:26:59.018647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.027723 (+  9076us) service_pool.cc:224] Handling call
0709 12:27:01.603748 (+2576025us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b540033e5d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.603918 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4056182144}
W20260709 12:27:01.603291  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 141794us
I20260709 12:27:01.600945  7654 rpcz_store.cc:276] 0709 12:26:59.240614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.491144 (+1250530us) service_pool.cc:224] Handling call
0709 12:27:00.512259 (+ 21115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.691558 (+179299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.600010 (+908452us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54002198d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:01.600314 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409605888}
I20260709 12:27:01.555583 11914 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.605789  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.426105 11928 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.573119 11882 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 12:27:01.607669  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.374s (SENT)
I20260709 12:27:01.629437  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.627846  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.690238  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.691100  7659 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:01.691491  7660 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:01.663884  7810 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:01.693253  7728 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:01.614447  8405 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:01.611074  7734 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.689203  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:01.723039 11928 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:01.723464 11993 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:27:01.730734  7659 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:01.731581  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2131) took 1108 ms. Trace:
I20260709 12:27:01.732009  7659 rpcz_store.cc:276] 0709 12:27:00.622895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.627413 (+1004518us) service_pool.cc:224] Handling call
0709 12:27:01.731508 (+104095us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:01.732656  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 106213us
I20260709 12:27:01.733242  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.637730  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.734603  7654 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:01.736586  7660 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:01.696341 11914 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:01.739090  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.740161  7659 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
W20260709 12:27:01.613719 11937 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.000s
I20260709 12:27:01.741603  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:01.744141 11992 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:01.759586  8408 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:01.604192 11907 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:01.159576  7625 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45926 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2131)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2137)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2144)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2145)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2149)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2138)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2130)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2148)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 747)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 748)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2379)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2381)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2388)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 749)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 750)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2129)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2132)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2136)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2390)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2393)
I20260709 12:27:01.751353 11882 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:01.751957  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2127) took 2678 ms (client timeout 1867 ms). Trace:
W20260709 12:27:01.766217  7658 rpcz_store.cc:269] 0709 12:26:59.073699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.430586 (+356887us) service_pool.cc:224] Handling call
0709 12:27:01.751608 (+2321022us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54000f4454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:01.751813 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060090624}
I20260709 12:27:01.768160  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.769243  7658 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:01.768308  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.770656  7734 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:01.673162 11970 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.004s
W20260709 12:27:01.750907 11841 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 12:27:01.756605  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:01.744544 11932 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.595779 11953 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.752862 11856 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:01.676906 11997 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:01.634480 11991 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:01.798321  7733 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:27:01.798982  7733 pending_rounds.cc:85] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:27:01.791582  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.800129  7660 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:27:01.767622 11842 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:01.801138 11997 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.634678 11962 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.145s
W20260709 12:27:01.772356 11868 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.510s	user 0.006s	sys 0.010s
I20260709 12:27:01.762018 11941 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.792672  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.802958  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2130) took 1180 ms. Trace:
I20260709 12:27:01.803831  7733 pending_rounds.cc:107] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:27:01.855926  7733 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:01.807293 11991 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.797035 12001 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.812388  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:01.812551  7654 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:01.684274 11871 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.814596 11938 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:01.818550 10979 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:27:01.819953  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.867431  7656 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:01.868487  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.869086  7656 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:01.869666  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2139) took 1064 ms. Trace:
I20260709 12:27:01.869997  7656 rpcz_store.cc:276] 0709 12:27:00.805644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.868203 (+1062559us) service_pool.cc:224] Handling call
0709 12:27:01.869580 (+  1377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:01.780270 11882 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.871987 11961 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:01.872471 11961 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:01.873153 11882 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:01.833629  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.841364 11986 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.681s	user 0.003s	sys 0.000s
I20260709 12:27:01.842115  7734 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:01.856036 11941 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.823196  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:01.896293  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73154.1us
W20260709 12:27:01.807049 11903 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.002s
I20260709 12:27:01.847108  7737 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:01.855736  7660 rpcz_store.cc:276] 0709 12:27:00.622633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.791212 (+1168579us) service_pool.cc:224] Handling call
0709 12:27:01.802862 (+ 11650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:01.835623 11819 raft_consensus.cc:993] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:01.777565 11841 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.840001 11997 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:01.860463  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2137) took 1158 ms. Trace:
I20260709 12:27:01.842212 11833 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:01.836752 11995 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 12:27:01.854928 11868 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.601s	user 0.006s	sys 0.011s
I20260709 12:27:01.871052  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:01.908362  7656 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:01.845827 11856 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:01.898792  7659 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:01.901152  7654 rpcz_store.cc:276] 0709 12:27:00.701811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.621410 (+919599us) service_pool.cc:224] Handling call
0709 12:27:01.860366 (+238956us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:01.901780  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 205986us
W20260709 12:27:01.905151  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 275686us
I20260709 12:27:01.914273  7803 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:01.914912  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 748) took 2540 ms. Trace:
I20260709 12:27:01.873368 11961 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.917412  7654 rpcz_store.cc:276] 0709 12:26:59.374212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.913972 (+2539760us) service_pool.cc:224] Handling call
0709 12:27:01.914343 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.914551 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.914574 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:01.914624 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:01.914656 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.914711 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.914846 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:01.917806  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.920612  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:01.903622 11952 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.880s	user 0.007s	sys 0.008s
W20260709 12:27:01.908295 11873 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.005s
I20260709 12:27:01.903247 11833 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.908623 11904 raft_consensus.cc:1081] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:01.903395 11842 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:01.945212  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.928817  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:01.857841 11907 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.947777  7654 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:01.950523  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2371) took 2265 ms. Trace:
I20260709 12:27:01.950879  7654 rpcz_store.cc:276] 0709 12:26:59.685544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.926565 (+2241021us) service_pool.cc:224] Handling call
0709 12:27:01.947572 (+ 21007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.950237 (+  2665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.950461 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:01.916481 11856 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.942562 11995 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.900769 10979 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:27:01.937368 11952 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.903258 11903 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.944715 11938 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:01.956887 11991 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:01.957113 11933 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 2.383s	user 0.000s	sys 0.005s
I20260709 12:27:01.924074 11970 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.959357  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:01.970114 11862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.617s	user 0.003s	sys 0.018s
I20260709 12:27:01.938657 11873 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:01.962949 12006 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:01.962759 11995 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:01.962848 11907 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:01.967862 10979 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:01.968817  7654 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
W20260709 12:27:01.970517  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.013023  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2372) took 2324 ms. Trace:
W20260709 12:27:01.970544  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 362870us
I20260709 12:27:02.013411  7654 rpcz_store.cc:276] 0709 12:26:59.688855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.953042 (+2264187us) service_pool.cc:224] Handling call
0709 12:27:01.968617 (+ 15575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.012689 (+ 44072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.012951 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:01.947642 11962 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.945374 12005 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.982527  7803 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:27:01.992784 11862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.636s	user 0.003s	sys 0.018s
I20260709 12:27:01.961002 11842 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:01.937986 11932 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:02.014034  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8141320230001838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:27:01.976197  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.014830  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119126us
I20260709 12:27:01.994114  7658 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:02.078938  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.103365  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 653) took 3106 ms. Trace:
I20260709 12:27:02.109777  7805 rpcz_store.cc:276] 0709 12:26:58.996643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.999335 (+  2692us) service_pool.cc:224] Handling call
0709 12:26:58.999807 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.007537 (+  7730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.007567 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.022155 (+ 14588us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.339881 (+317726us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:59.339946 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.340053 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.961083 (+621030us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b4800091158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:27:01.956336 (+1995253us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:02.102963 (+146627us) raft_consensus.cc:1685] finished
0709 12:27:02.103000 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.103262 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7279,"raft.queue_time_us":17075,"raft.run_cpu_time_us":9959,"raft.run_wall_time_us":954275,"spinlock_wait_cycles":165471232,"thread_start_us":115239,"threads_started":1,"wal-append.queue_time_us":292832}
I20260709 12:27:02.112775  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.109797  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:02.115811  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.106051  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:02.100807  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.012339 10979 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:01.981036 12007 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 12:27:02.113631  7805 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:02.150097  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:02.150894  7805 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:02.151285  7805 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:02.145478  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.144317  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:02.065280 11753 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:02.157923  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 747) took 2916 ms. Trace:
I20260709 12:27:02.158613  7656 rpcz_store.cc:276] 0709 12:26:59.241454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.909168 (+2667714us) service_pool.cc:224] Handling call
0709 12:27:01.909553 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.909782 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:02.076920 (+167138us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.157486 (+ 80566us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.157546 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:02.157592 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.157696 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.157845 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6566,"raft.queue_time_us":231,"thread_start_us":213385,"threads_started":2}
I20260709 12:27:02.152282  7805 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.164883  7805 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:02.160465  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:02.167316  7803 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:02.169924  7803 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:27:02.172781  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2115) took 3178 ms (client timeout 1628 ms). Trace:
W20260709 12:27:02.173342  7655 rpcz_store.cc:269] 0709 12:26:58.994069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.994239 (+   170us) service_pool.cc:224] Handling call
0709 12:27:02.172542 (+3178303us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001a40d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.172699 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3644864256}
I20260709 12:27:02.177333  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.178700  7656 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:02.180879  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:02.181744  7654 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:02.070190 11962 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.195001  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.180670 10979 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.198423  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 826) took 2859 ms. Trace:
I20260709 12:27:02.267261  7735 rpcz_store.cc:276] 0709 12:26:59.338911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.015286 (+676375us) service_pool.cc:224] Handling call
0709 12:27:00.015698 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.015924 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:00.398755 (+382831us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.197967 (+1799212us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b18003a1440. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:27:02.198069 (+   102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.198117 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:02.198164 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.198235 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.198365 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2635,"prepare.queue_time_us":566377,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":90724,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4945,"spinlock_wait_cycles":18370816,"thread_start_us":577131,"threads_started":3,"wal-append.queue_time_us":373235,"wal-append.run_cpu_time_us":1433,"wal-append.run_wall_time_us":1045878}
W20260709 12:27:02.342121  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 248138us
I20260709 12:27:02.209721 11997 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:02.353335 12010 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:02.353607 11997 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:02.354012 12010 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.354640 11997 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.356912 12010 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.345011 11879 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:02.343858  7656 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:02.176903 12014 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:02.348112  7805 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:02.192324 11868 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.366264 11868 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.272825 11953 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.367707  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:02.376322  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.370918 11868 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.371814  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2171) took 4911 ms. Trace:
W20260709 12:27:02.371173  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2121) took 3351 ms (client timeout 3028 ms). Trace:
I20260709 12:27:02.373533 11840 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.287144 11841 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.352209  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.352530  7730 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:02.403597  7737 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:02.181471 11943 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
I20260709 12:27:02.415601  7654 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:02.358634 10979 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:02.360378  7735 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:02.173425 11753 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.285230 11842 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.425951 11753 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.380558  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2670) took 1060 ms. Trace:
I20260709 12:27:02.380857  7728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
I20260709 12:27:02.381335  7803 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [6.5-9.6]
W20260709 12:27:02.428632 11928 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 12:27:02.426824  7810 rpcz_store.cc:276] 0709 12:27:01.319864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.414066 (+ 94202us) service_pool.cc:224] Handling call
0709 12:27:02.380258 (+966192us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001347d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.380452 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323674368}
I20260709 12:27:02.430612  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:02.389050 11903 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.431571  7810 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:02.433166  7810 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:27:02.434965  7654 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:27:02.436266  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2377) took 2617 ms. Trace:
I20260709 12:27:02.377106  7656 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:02.436794  7654 rpcz_store.cc:276] 0709 12:26:59.819075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.417165 (+2598090us) service_pool.cc:224] Handling call
0709 12:27:02.434508 (+ 17343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.435794 (+  1286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.436172 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.387130  7655 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [7.6-8.7]
I20260709 12:27:02.351495  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:02.438546  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2375) took 2633 ms. Trace:
I20260709 12:27:02.439301  7655 rpcz_store.cc:276] 0709 12:26:59.805580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.386092 (+2580512us) service_pool.cc:224] Handling call
0709 12:27:02.386732 (+   640us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:02.438162 (+ 51430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.438459 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.388378  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 811) took 1432 ms. Trace:
W20260709 12:27:02.438982  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 87779.8us
I20260709 12:27:02.378279  7805 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:27:02.413323 12025 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
I20260709 12:27:02.438333 11928 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.447826 11928 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.449370  7805 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:27:02.449600 11833 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.010s
I20260709 12:27:02.450474 11928 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:02.383260  7662 rpcz_store.cc:269] 0709 12:26:59.020011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.020179 (+   168us) service_pool.cc:224] Handling call
0709 12:27:02.370864 (+3350685us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003999d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.371078 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2807776000}
I20260709 12:27:02.450804 12024 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.451452 12024 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.451949 11873 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.454123 12024 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.454394 11995 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.419679 11962 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.421801  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 746) took 3180 ms. Trace:
W20260709 12:27:02.383446  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2116) took 3384 ms (client timeout 2124 ms). Trace:
W20260709 12:27:02.440521  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.382812  7804 rpcz_store.cc:276] 0709 12:26:57.460213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.460606 (+   393us) service_pool.cc:224] Handling call
0709 12:26:57.500370 (+ 39764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.216487 (+2716117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.371486 (+2154999us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b5400295254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:02.371700 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6116753280}
I20260709 12:27:02.382057  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:02.412896 12011 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.309s	user 0.002s	sys 0.000s
I20260709 12:27:02.440486  8408 rpcz_store.cc:276] 0709 12:27:00.955595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.966776 (+ 11181us) service_pool.cc:224] Handling call
0709 12:27:02.388091 (+1421315us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54002ac6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.388277 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705441536}
I20260709 12:27:02.444166 12022 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:02.445326  7810 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:02.450146 11833 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:02.488328  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.381570 11854 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.464620  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:02.466115 12007 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:27:02.466929 11914 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.001s
W20260709 12:27:02.463440 11877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.011s	sys 0.020s
W20260709 12:27:02.489624  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107722us
I20260709 12:27:02.490461  7728 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:02.490335  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:02.490466  7660 rpcz_store.cc:276] 0709 12:26:59.241043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.903836 (+2662793us) service_pool.cc:224] Handling call
0709 12:27:01.904327 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.904663 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:02.095569 (+190906us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.421333 (+325764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.421407 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:02.421462 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.421546 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.421711 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19893,"prepare.queue_time_us":368643,"prepare.run_cpu_time_us":879,"prepare.run_wall_time_us":6130,"raft.queue_time_us":30890,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":4006,"thread_start_us":475112,"threads_started":2,"wal-append.queue_time_us":379494}
I20260709 12:27:02.497624  7728 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:02.492363  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.498873  7660 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:27:02.495374  7654 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 12:27:02.495637  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:02.491473  7653 rpcz_store.cc:269] 0709 12:26:58.999584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.999864 (+   280us) service_pool.cc:224] Handling call
0709 12:27:02.383057 (+3383193us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540002ec54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.383319 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4194385792}
I20260709 12:27:02.500079  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2383) took 2604 ms. Trace:
I20260709 12:27:02.500487  7660 rpcz_store.cc:276] 0709 12:26:59.896038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.498214 (+2602176us) service_pool.cc:224] Handling call
0709 12:27:02.498603 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.499817 (+  1214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.500017 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.452036 11856 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.502890  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2382) took 2615 ms. Trace:
I20260709 12:27:02.506942  7654 rpcz_store.cc:276] 0709 12:26:59.887643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.487011 (+2599368us) service_pool.cc:224] Handling call
0709 12:27:02.494942 (+  7931us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:02.499419 (+  4477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.502823 (+  3404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.505223  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:02.508546  7810 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:02.509658  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2384) took 2603 ms. Trace:
I20260709 12:27:02.510139  7660 rpcz_store.cc:276] 0709 12:26:59.906291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.501646 (+2595355us) service_pool.cc:224] Handling call
0709 12:27:02.502456 (+   810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.509302 (+  6846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.509328 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.509374 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:02.509404 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.509463 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.509595 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:02.496837 11877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.011s	sys 0.020s
I20260709 12:27:02.519106  7804 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:27:02.519160  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.523558  7662 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:27:02.526407 12013 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.539s	user 0.001s	sys 0.000s
I20260709 12:27:02.513459 11995 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:02.497709 11928 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:27:02.530856  7656 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:02.514122 11824 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:02.538141  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.514449 11845 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:27:02.539379  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.523193  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:02.512827 11896 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.532317  7728 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.502192 11914 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:02.546671  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.549242 11896 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.551565 11896 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:02.552143 12001 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.013s
I20260709 12:27:02.552239 11896 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:02.500749 11854 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.546490 11928 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:02.554352 11928 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.555022 11854 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.556269 11914 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.541343 11845 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.539469 11995 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.557114 11845 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:02.557312 11995 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:02.557636 11952 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.558735  7804 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:02.528483 11862 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:02.559793 11862 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:02.560432  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.557787 11833 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.562182 11862 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.552759 11896 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.562999 11896 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:02.567276  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:02.568250  8408 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:02.569950  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52265.5us
I20260709 12:27:02.574712  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:02.576390  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.584157  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 662) took 3582 ms. Trace:
I20260709 12:27:02.584775  7809 rpcz_store.cc:276] 0709 12:26:59.002566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.689960 (+687394us) service_pool.cc:224] Handling call
0709 12:26:59.692583 (+  2623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.223330 (+1530747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.583790 (+1360460us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400132ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:02.584069 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446910336}
W20260709 12:27:02.586959 11984 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.360s	user 0.001s	sys 0.002s
W20260709 12:27:02.585335 12021 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:27:02.532332 11841 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:27:02.552784 12001 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:02.590499  7810 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:02.597647  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.598534  7735 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:02.595422  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 130890us
I20260709 12:27:02.544866 11824 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.591037 11842 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.600698  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:02.611162  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2388) took 2667 ms. Trace:
I20260709 12:27:02.612138  7654 rpcz_store.cc:276] 0709 12:26:59.943897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.515221 (+2571324us) service_pool.cc:224] Handling call
0709 12:27:02.519621 (+  4400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.519819 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:02.520088 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.610758 (+ 90670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.610822 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:02.610877 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.610947 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.611093 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4159,"prepare.queue_time_us":2266,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":429,"thread_start_us":89740,"threads_started":1}
I20260709 12:27:02.614818  7654 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:27:02.615545  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 750) took 2609 ms. Trace:
I20260709 12:27:02.615899  7654 rpcz_store.cc:276] 0709 12:27:00.006014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.614175 (+2608161us) service_pool.cc:224] Handling call
0709 12:27:02.614557 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.615266 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.615479 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.617259  7654 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:02.617894  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 751) took 2611 ms. Trace:
I20260709 12:27:02.618229  7654 rpcz_store.cc:276] 0709 12:27:00.006364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.616738 (+2610374us) service_pool.cc:224] Handling call
0709 12:27:02.617074 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.617624 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.617837 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.620054  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 752) took 2613 ms. Trace:
I20260709 12:27:02.620415  7654 rpcz_store.cc:276] 0709 12:27:00.006740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.618997 (+2612257us) service_pool.cc:224] Handling call
0709 12:27:02.619474 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.619695 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.619725 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.619774 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:02.619810 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.619872 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.619996 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.625304 12034 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:02.626302 12034 consensus_queue.cc:1243] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:02.626824 12034 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:02.627167 12034 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:02.627851 12028 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:27:02.628158 12034 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.629881 12034 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:02.610363 11716 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:02.634987 11716 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:02.636013 11716 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.636812 11824 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.607663 11842 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.640018 11842 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:02.610823 11907 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
I20260709 12:27:02.642774  7660 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
W20260709 12:27:02.644102 12019 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 12:27:02.640162 11996 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:02.645483  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2132) took 2020 ms. Trace:
I20260709 12:27:02.646072  7660 rpcz_store.cc:276] 0709 12:27:00.625467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.642258 (+2016791us) service_pool.cc:224] Handling call
0709 12:27:02.642576 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.645114 (+  2538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.645404 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.652436  7660 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:27:02.653514  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2136) took 2025 ms. Trace:
I20260709 12:27:02.653784  7653 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:02.653911  7660 rpcz_store.cc:276] 0709 12:27:00.628419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.647747 (+2019328us) service_pool.cc:224] Handling call
0709 12:27:02.648071 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.653232 (+  5161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.653455 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.657708  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.660665  7660 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:27:02.661413  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2390) took 1777 ms. Trace:
I20260709 12:27:02.661919  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.662967  7731 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:02.661760  7660 rpcz_store.cc:276] 0709 12:27:00.884248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.659971 (+1775723us) service_pool.cc:224] Handling call
0709 12:27:02.660307 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.661128 (+   821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.661351 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.666035  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:02.666721 12036 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:27:02.666911  7735 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:02.668473  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2381) took 2801 ms. Trace:
I20260709 12:27:02.655802 11907 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:27:02.668965  7655 rpcz_store.cc:276] 0709 12:26:59.866928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.459629 (+2592701us) service_pool.cc:224] Handling call
0709 12:27:02.460190 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.463976 (+  3786us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:02.496989 (+ 33013us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.579090 (+ 82101us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.592828 (+ 13738us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.593043 (+   215us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:02.593068 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.593123 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.668235 (+ 75112us) raft_consensus.cc:1685] finished
0709 12:27:02.668253 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.668410 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21376,"prepare.queue_time_us":32991,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":4025,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":2557,"thread_start_us":42441,"threads_started":1,"wal-append.queue_time_us":81721}
I20260709 12:27:02.671784  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 759) took 1714 ms. Trace:
I20260709 12:27:02.672225  7655 rpcz_store.cc:276] 0709 12:27:00.957087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.670886 (+1713799us) service_pool.cc:224] Handling call
0709 12:27:02.671272 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.671461 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.671489 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.671536 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:02.671568 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.671621 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.671727 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.665153  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.673776  7731 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:02.674458  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 761) took 1716 ms. Trace:
W20260709 12:27:02.676601  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.675108  7655 rpcz_store.cc:276] 0709 12:27:00.957770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.673265 (+1715495us) service_pool.cc:224] Handling call
0709 12:27:02.673676 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.673948 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.673990 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.674055 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:02.674131 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.674212 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.674370 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.678661  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.681556  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.682498  7731 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:02.692826  7810 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:27:02.693456  7655 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:27:02.694150  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2393) took 1651 ms. Trace:
I20260709 12:27:02.694525  7655 rpcz_store.cc:276] 0709 12:27:01.042474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.692763 (+1650289us) service_pool.cc:224] Handling call
0709 12:27:02.693094 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.693880 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.694089 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.695901  7655 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:02.698514  7655 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:02.707392  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:02.708809  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.711457  7811 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:02.652396 11996 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:02.715719  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.611255 12023 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 12:27:02.716202  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.720053  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.720413  7809 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:27:02.720817  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:02.716580  7810 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:02.721796  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.715808  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.723335  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.716920  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:02.724081  7808 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:02.724202  7735 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:27:02.723073  7655 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
W20260709 12:27:02.730401  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 656) took 3732 ms (client timeout 2557 ms). Trace:
W20260709 12:27:02.730898  7807 rpcz_store.cc:269] 0709 12:26:58.997525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.344759 (+347234us) service_pool.cc:224] Handling call
0709 12:27:02.730158 (+3385399us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.730318 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3533659520}
I20260709 12:27:02.732642  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:02.737190  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.738719  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.729193  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 758) took 1818 ms. Trace:
I20260709 12:27:02.706734  8408 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:02.743510  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:02.743965  7731 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:02.739991  7655 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:02.746371  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:02.747684  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.715986 11862 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:02.759667 11837 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:02.760845  7661 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2391) took 1876 ms (client timeout 1653 ms). Trace:
I20260709 12:27:02.771463  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:02.771970  7661 rpcz_store.cc:269] 0709 12:27:00.884548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.760602 (+1876054us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:02.760739 (+   137us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:27:02.762809  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 665) took 2773 ms. Trace:
I20260709 12:27:02.773820  7812 rpcz_store.cc:276] 0709 12:26:59.989883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.791498 (+801615us) service_pool.cc:224] Handling call
0709 12:27:00.889390 (+ 97892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.892390 (+  3000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.762597 (+1870207us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b54003728d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:02.762732 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212252928}
I20260709 12:27:02.763619  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.775741  7735 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:02.764235  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.764689 11877 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:02.764878 11909 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:02.717639 11848 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:27:02.721043 11716 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:27:02.778964 11837 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.786582 11837 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:02.716455 11907 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:27:02.792321 11845 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.766346  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.794782  7730 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:27:02.795956  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.797231  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.767253 11996 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.671999 11995 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.798295  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.800367  7656 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 9. Ops: []
I20260709 12:27:02.773167  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.780841 11877 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:02.782708  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.766966  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2404) took 1338 ms. Trace:
I20260709 12:27:02.805963  7733 rpcz_store.cc:276] 0709 12:27:01.429147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.486042 (+ 56895us) service_pool.cc:224] Handling call
0709 12:27:01.486384 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.856542 (+370158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.766692 (+910150us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54001c52d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:02.766895 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703247104}
I20260709 12:27:02.807694  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.809173  7733 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:02.804477  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.814953  7661 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:02.820644  7729 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:02.822750  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 76556.6us
I20260709 12:27:02.824826  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.833389  7655 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:02.801512 11862 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:02.835192  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2397) took 1038 ms. Trace:
I20260709 12:27:02.835960  7655 rpcz_store.cc:276] 0709 12:27:01.796732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.812300 (+1015568us) service_pool.cc:224] Handling call
0709 12:27:02.833039 (+ 20739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.834888 (+  1849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.835131 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:02.820981 11907 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.839460  7812 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:02.839874  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:02.841970  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2385) took 2899 ms. Trace:
I20260709 12:27:02.842463  7653 rpcz_store.cc:276] 0709 12:26:59.942761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.512864 (+2570103us) service_pool.cc:224] Handling call
0709 12:27:02.522013 (+  9149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.654303 (+132290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.841706 (+187403us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003999d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:02.841903 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302598016}
I20260709 12:27:02.792501 11896 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:02.844694 11987 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.571s	user 0.005s	sys 0.001s
I20260709 12:27:02.834158 11845 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.854202  7653 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:02.854390  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:02.859607  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.859573  7655 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:02.862771  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:02.872056 11985 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.587s	user 0.000s	sys 0.004s
I20260709 12:27:02.873829 11842 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.874578 11842 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.876142 11845 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:02.877084 11845 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:02.877218 11842 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.850045 11995 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:02.878782 11823 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:02.878564 11845 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.881937 11928 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.882596 11928 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.885125 11928 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:02.850562 11903 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:02.887984 11903 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.890473 11903 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.846484 11907 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.898224 11896 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:02.870323 11877 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:02.900271  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 707) took 1948 ms. Trace:
I20260709 12:27:02.894927 11907 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.900914  8411 rpcz_store.cc:276] 0709 12:27:00.952365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.952525 (+   160us) service_pool.cc:224] Handling call
0709 12:27:00.952901 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.953320 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.953353 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.953873 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.148237 (+194364us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:01.148291 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.148381 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.840496 (+1692115us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:02.899776 (+ 59280us) raft_consensus.cc:1685] finished
0709 12:27:02.899829 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.900145 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2897,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":6576512,"thread_start_us":193896,"threads_started":1,"wal-append.queue_time_us":240579}
I20260709 12:27:02.918642  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:02.926200  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55133.3us
I20260709 12:27:02.927968  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:02.929522  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58557.4us
W20260709 12:27:02.930076  7777 connection.cc:256] RPC call timeout handler was delayed by -1.1531787740000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3028s for a response
W20260709 12:27:02.932922  7777 connection.cc:256] RPC call timeout handler was delayed by -1.0646725359997617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 12:27:02.933873  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:02.934489  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.838377 11862 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.933122 11842 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:02.936605 11896 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:02.935315 12044 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:27:02.930920 11823 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:02.939172 11929 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:02.940691 12043 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:27:02.925035 11903 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.942209 12008 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:02.943470 11907 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:02.944751  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:02.945536 12034 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:02.931947 11995 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:02.976576  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.958613 12007 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:27:02.966769 11896 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:02.985105  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:02.975781 11823 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:02.991443  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.991679  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.992444  7655 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:02.973646 12044 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:27:02.962001 11909 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:27:02.995651  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.005707  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.006227  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.997256 11896 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:02.971558 11903 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.013792 11903 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:02.960291 11842 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.004338  7812 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:27:03.008805  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:02.980631 11873 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:27:02.995391  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.036892  8411 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:03.044051  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:03.044360  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.046541  8408 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:03.049448  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.050634  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:03.051411  8408 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:03.052610  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.054179  7655 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:03.059319  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.059579  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.060555  7655 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:03.063143  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.064491  8411 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:03.073163  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.078884  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:03.083508  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.080489  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.086093  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.085035  8411 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:03.087845  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152716us
I20260709 12:27:03.088326  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:03.088797  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:03.075933 11951 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.055335 11929 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:27:03.083328 11823 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:03.055554 11837 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.085919 11904 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:27:03.127779  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 168095us
W20260709 12:27:03.131683  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 143350us
I20260709 12:27:03.086627 11928 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:03.135983  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:03.089813 11995 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:03.139976  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:03.140127 11995 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.141283  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:03.092479 11840 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:03.144418  7731 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:03.122630 11841 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:03.143985  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 760) took 2186 ms (client timeout 1660 ms). Trace:
W20260709 12:27:03.148492  7657 rpcz_store.cc:269] 0709 12:27:00.957430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.143744 (+2186314us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:03.143909 (+   165us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:27:03.138083 12015 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.738s	user 0.004s	sys 0.003s
W20260709 12:27:03.155001  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66238.2us
I20260709 12:27:03.155668  7657 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:03.156162  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.129093 11904 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:03.159174  7655 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:03.160236  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.121908 11951 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.175791  7807 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:03.172442  7657 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:27:03.147169 11841 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.146473 11840 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:03.183733  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.155004 11928 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:03.186753  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.187858  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:03.192152  7810 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:03.193555  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52352.8us
I20260709 12:27:03.144287 11907 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:27:03.194689  7702 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Timed out: Call waited in the queue past client deadline [suppressed 27 similar messages]
I20260709 12:27:03.197640  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.172042 11904 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:03.199554  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:03.201499  7807 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:03.185760 11841 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:03.204282  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:03.208379  8411 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:03.185200 11909 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:03.186250 11840 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:03.214551  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.219327  8412 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:27:03.210858 11909 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:03.225325 11909 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:03.226934  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:03.230175  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46517.7us
I20260709 12:27:03.231145  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.222062  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:03.232393  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.234562  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.236011  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.199258 12052 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:03.240613  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:03.247786  7807 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:03.250267  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:03.251683  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:03.260782  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:03.263731  7807 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:27:03.177662 12053 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:27:03.205488 11903 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:03.212126 12066 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:03.211515 11907 consensus_queue.cc:1243] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:27:03.268496  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 70597us
I20260709 12:27:03.270754  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:03.217697 11882 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:03.248986 11914 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:27:03.249404 11842 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.238647 12067 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.218284 11823 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.282323 11882 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.274658 12064 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:03.288281 11882 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:03.283713  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89041.9us
I20260709 12:27:03.287756 12067 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.267155  8406 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:27:03.294003 12064 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:03.297859 12056 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:03.298452 12067 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.305966 12052 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.306811 12052 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.283461 11914 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:03.277570 11903 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.310544 11824 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:27:03.339020  7728 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:03.340803 12053 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:03.341053 12059 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:27:03.341423 12053 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.342949 11928 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:03.319717 12064 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:03.344218 12053 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.345044 11951 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:03.339542  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.352140  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2392) took 2467 ms (client timeout 2569 ms). Trace:
W20260709 12:27:03.352522 12061 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:27:03.352869  7659 rpcz_store.cc:269] 0709 12:27:00.884785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.768059 (+883274us) service_pool.cc:224] Handling call
0709 12:27:03.351737 (+1583678us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54000f4454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:03.351975 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294790400}
I20260709 12:27:03.358296 11903 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:03.320240 11914 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:27:03.365774  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:02.742937  7660 rpcz_store.cc:276] 0709 12:27:00.910748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.664363 (+1753615us) service_pool.cc:224] Handling call
0709 12:27:02.664829 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.665024 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:02.668542 (+  3518us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.728782 (+ 60240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.728843 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:02.728893 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.729005 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.729131 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7079,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":94,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13811,"thread_start_us":59768,"threads_started":1,"wal-append.queue_time_us":83107}
I20260709 12:27:03.373966  7657 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [7.6-8.7]
I20260709 12:27:03.376713 12052 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:03.376802 11929 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.380556  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.381244  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:03.337855  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 106780us
I20260709 12:27:03.388079  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:03.392001  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.377823 11929 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.398804 11929 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:03.383265 12068 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.410239 12068 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.412636 12052 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.406163  7659 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:03.413357 12052 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.417403 11941 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:03.419471 12024 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.420506 12066 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:03.420327 12024 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.423398 11845 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:03.425532  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2214) took 1449 ms. Trace:
I20260709 12:27:03.434904  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:03.434764  7805 rpcz_store.cc:276] 0709 12:27:01.975850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.442418 (+466568us) service_pool.cc:224] Handling call
0709 12:27:02.442784 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.449873 (+  7089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.425174 (+975301us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b54001b1f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:03.425450 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13807488}
I20260709 12:27:03.437323 11854 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:03.437867 11854 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:03.426352 11879 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:03.447865  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.447484 12078 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:27:03.413213 12068 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:03.453830  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.454111  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66660.1us
I20260709 12:27:03.456264  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:03.458684  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.494385  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 81988.3us
I20260709 12:27:03.496739  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.423969 11841 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.479012  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:03.434638 11845 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:03.438782 11927 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:27:03.484894 11962 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:03.452674 11879 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:03.453109 11990 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:03.500810 12066 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:27:03.487763  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.459841  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:03.498746  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.506870  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:03.501420 11842 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:27:03.620810  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122194us
I20260709 12:27:03.621825  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.485769 11854 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.503480 12052 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:03.517593  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.521181 12053 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:03.513345 11990 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:03.562340  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 324503us
W20260709 12:27:03.575443  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.697618  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 214287us
W20260709 12:27:03.609071  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.510355 11962 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:03.636970  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:03.635087  7806 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:03.711057  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:03.714061  7657 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:03.716650  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.718081  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.619666 12066 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:03.727372 12066 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.729351 11841 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.709525 12076 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:27:03.710772 11854 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:03.732522 11914 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:03.733500 11904 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.005s
I20260709 12:27:03.734015 11904 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.700599 11842 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:03.735186 12034 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:03.735931 12034 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:03.736289 11842 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.737174 12034 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.738619 12034 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:03.701773 11845 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.725857 11990 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.745311 11914 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:03.682308 12075 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:03.745973 11990 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:03.743765 11962 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.751566  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:03.752760  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2221) took 1041 ms. Trace:
I20260709 12:27:03.753142  7804 rpcz_store.cc:276] 0709 12:27:02.711425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.711574 (+   149us) service_pool.cc:224] Handling call
0709 12:27:03.751351 (+1039777us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:27:03.752680 (+  1329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902400}
I20260709 12:27:03.746304 12023 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:03.745002 11973 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:03.765092 11973 consensus_queue.cc:1243] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:03.765228 12024 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.692946 11951 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:03.727447 11903 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:03.766458 11951 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.767251 11903 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:03.767000  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:03.767838 11903 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:03.768936 11903 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:03.770340 11903 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:03.766975 11951 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:03.775713 11962 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:03.745149 11845 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:03.727906 11879 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:03.785383  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.788437  7729 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:27:03.793308  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156446us
I20260709 12:27:03.796321  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:03.801318  8411 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:03.798713  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2577) took 1111 ms. Trace:
W20260709 12:27:03.803511  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 307275us
I20260709 12:27:03.803922  7729 rpcz_store.cc:276] 0709 12:27:02.687822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.821667 (+133845us) service_pool.cc:224] Handling call
0709 12:27:03.766764 (+945097us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:27:03.798638 (+ 31874us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13160704}
I20260709 12:27:03.818266  8408 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:27:03.799691 11840 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.007s
I20260709 12:27:03.835345  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:03.829695  8411 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:03.801338 12073 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:03.844211  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:03.839133  7657 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:03.826819  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:03.842450 11909 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.615s	user 0.017s	sys 0.000s
I20260709 12:27:03.851547  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:03.843640 12024 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:03.865988  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.824124 11856 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:03.843966 12073 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:03.867497  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 442319us
I20260709 12:27:03.833887 11873 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:27:03.900440  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.887362  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.901396  7657 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:03.905560  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:03.841969 11840 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:03.913429  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2138) took 3112 ms (client timeout 2736 ms). Trace:
W20260709 12:27:03.915256  7658 rpcz_store.cc:269] 0709 12:27:00.800720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.770241 (+969521us) service_pool.cc:224] Handling call
0709 12:27:03.913128 (+2142887us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:03.913334 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311357696}
I20260709 12:27:03.849371 11872 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:03.917171  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:03.902060 12087 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:03.914086  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:03.958345  7657 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
W20260709 12:27:03.953202  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.959476  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:03.949473  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:03.960404  7658 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:03.960520  7732 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:03.924609  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.875072 11909 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:03.908286 12073 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:03.962652  7733 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:03.920622 11904 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.964335  7657 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [4.3-7.5]
I20260709 12:27:03.969465  7657 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:03.971946  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:03.972708  7733 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 6.
I20260709 12:27:03.973168  7657 raft_consensus.cc:1217] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Deduplicated request from leader. Original: 5.4->[6.5-9.6]   Dedup: 6.5->[9.6-9.6]
I20260709 12:27:03.975081  7657 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 12:27:03.977876  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:03.985486 11840 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:03.992377  7729 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:03.995744  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:03.998716  7657 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:04.002313  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:04.004058  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 277617us
I20260709 12:27:03.971118 11872 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:04.014671  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:04.015815  7733 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:04.003715 11823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.090s	sys 0.150s
I20260709 12:27:03.977290 11907 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:27:03.978782 11862 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:27:04.043861 12084 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.009641 11879 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:04.045071  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.063534  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.028650 11823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.090s	sys 0.150s
I20260709 12:27:04.064637 11872 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.065680  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 850) took 1252 ms. Trace:
I20260709 12:27:04.066182  7731 rpcz_store.cc:276] 0709 12:27:02.813261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.861933 (+ 48672us) service_pool.cc:224] Handling call
0709 12:27:02.862272 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.144871 (+282599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.065326 (+920455us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400212d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:04.065601 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648355200}
I20260709 12:27:04.072238  7658 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:04.073975  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:04.075038  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 749) took 4094 ms. Trace:
I20260709 12:27:04.075704  7656 rpcz_store.cc:276] 0709 12:26:59.980454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.540994 (+2560540us) service_pool.cc:224] Handling call
0709 12:27:02.541334 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.800934 (+259600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.074703 (+1273769us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540021a7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:04.074935 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595149568}
I20260709 12:27:04.078718 12024 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:04.079882 12024 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:04.081278  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.081200 12024 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.083443  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:04.084223  7776 connection.cc:256] RPC call timeout handler was delayed by -1.2893804800000908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:27:04.046156 11882 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.002s
I20260709 12:27:04.040350 12085 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:04.094275  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.096745  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.097754  8411 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:27:04.100942  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:04.101890  8408 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:04.115694  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 2.02255e+06us
W20260709 12:27:04.116371  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5564396939998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:27:04.116761  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4836381380000603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 12:27:04.117132  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4061396940001032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:27:04.117431  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3584396940000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 12:27:04.117833  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3215396939999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 12:27:04.121991  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2882396939999126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:27:04.122639  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2471460669999033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:27:04.123106  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0275320230002762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2741s for a response
I20260709 12:27:04.092594 11882 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.125612  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:04.125916  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:04.176326  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51159us
I20260709 12:27:04.178692  8408 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:04.178948  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.103688 11716 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:04.213812  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 208494us
I20260709 12:27:04.224014  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:04.225095  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.233381  8405 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:04.176528 12093 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:27:04.240321  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61471.8us
I20260709 12:27:04.242925  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.243119  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2379) took 4390 ms. Trace:
I20260709 12:27:04.243835  7662 rpcz_store.cc:276] 0709 12:26:59.852391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.451489 (+2599098us) service_pool.cc:224] Handling call
0709 12:27:02.451871 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.524218 (+ 72347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.242722 (+1718504us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54003994d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:04.243026 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163706752}
I20260709 12:27:04.086748 12084 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.098539 11951 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.176605 12091 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.262032 12091 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.213356 12045 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:27:04.251914 11909 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.261083 11882 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.250600 11995 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:04.126549 12052 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:27:04.235661 11716 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:04.280443  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.295435  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:04.296916  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 210863us
I20260709 12:27:04.279134 12005 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:04.308224 11862 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:04.309005  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85064.2us
I20260709 12:27:04.270998 12045 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:04.290925 11716 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:04.294615 12008 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:04.308073 12005 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.320027 12005 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.324987  7661 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:04.326579 12070 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.327555 12070 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:04.330152  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.333871 12095 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:27:04.337297 12084 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.336038 11837 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.343863 12070 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.298146 11929 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:04.312048  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:04.346740  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:04.347792 12070 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:04.348554 12070 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:04.349135 12070 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:04.349665 11927 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:04.350210 11927 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:04.350633 11927 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:04.351778 11914 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:27:04.345927  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.362545  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.365602  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:04.316666 11990 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.000s
W20260709 12:27:04.359302 12034 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.006s
I20260709 12:27:04.313656 12103 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.374696 12008 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 12:27:04.396623 12096 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:27:04.397503 12045 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.344666 11951 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:04.398443 12091 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:04.358978 12074 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:04.358119 11995 raft_consensus.cc:2764] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:27:04.387060 11962 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:27:04.425829 12091 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:04.435940 12091 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:04.437529 12042 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.687s	user 0.004s	sys 0.005s
I20260709 12:27:04.437443 12091 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.402701 12103 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.439831  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2129) took 3817 ms. Trace:
I20260709 12:27:04.441510 12103 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.445811 12029 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:27:04.449127 11845 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.004s
I20260709 12:27:04.317126 11862 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.449826 12073 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:04.450513 11903 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.007s
I20260709 12:27:04.452354 11862 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.453660 12045 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:04.459915 11961 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:04.461573 11716 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.442800 12074 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:04.464946 12074 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.400295 11941 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:27:04.440250  7654 rpcz_store.cc:276] 0709 12:27:00.622334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.621450 (+1999116us) service_pool.cc:224] Handling call
0709 12:27:02.621823 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.652171 (+ 30348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.652201 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.660588 (+  8387us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.769704 (+109116us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:02.769749 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.769832 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.349879 (+580047us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4800080358. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:27:04.436813 (+1086934us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:04.439542 (+  2729us) raft_consensus.cc:1685] finished
0709 12:27:04.439565 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.439747 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":3251,"raft.run_wall_time_us":727710,"spinlock_wait_cycles":90967168,"thread_start_us":31232,"threads_started":1,"wal-append.queue_time_us":52592}
I20260709 12:27:04.475955 12010 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:04.476553 12010 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:04.477571 12010 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.478816 12010 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:04.482798  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:04.470892  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2403) took 2405 ms. Trace:
I20260709 12:27:04.487553  7661 rpcz_store.cc:276] 0709 12:27:02.065797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.829418 (+763621us) service_pool.cc:224] Handling call
0709 12:27:04.076721 (+1247303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.325557 (+248836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.469611 (+144054us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b540021a7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:04.470632 (+  1021us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540021a7d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:04.470817 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3431044096}
I20260709 12:27:04.491135  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:04.492079  7655 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:04.517592  7661 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:04.526075  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:04.545595  7661 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [4.3-7.5]
I20260709 12:27:04.475505 12070 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.535477 11719 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 12:27:04.552963  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 257742us
I20260709 12:27:04.554529  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.573704  7655 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:04.575855  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:04.578456  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.585134  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.535030 12052 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:27:04.591480  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.561089 11995 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.589952 12109 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:04.572382 11856 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:04.589057 11862 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:04.583047 11823 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:04.599728 12052 consensus_queue.cc:1243] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:04.609754 12008 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:04.619813 11996 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:04.639209 11862 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:04.599480 11995 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.683395  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.642211 12073 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:04.613163 11856 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:04.717659  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 429826us
W20260709 12:27:04.721550  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2414) took 1349 ms (client timeout 1535 ms). Trace:
W20260709 12:27:04.725294  7659 rpcz_store.cc:269] 0709 12:27:03.372521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.380208 (+  7687us) service_pool.cc:224] Handling call
0709 12:27:04.721222 (+1341014us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b540028d054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:04.721432 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56773504}
I20260709 12:27:04.727473  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:04.730003  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 364480us
W20260709 12:27:04.731882  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48635.3us
I20260709 12:27:04.732975  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:04.742177  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.741847 11868 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:04.742518 12112 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:04.749210 11716 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.751868  7653 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:27:04.755578  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:04.758689  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.759486  8409 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 5. Current term is 6. Ops: []
I20260709 12:27:04.760259  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:04.760591  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.760429  8414 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:04.751344 11868 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:04.762214  8411 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:04.779526  8411 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:04.781352  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:04.784140  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:04.783999  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.785877  8406 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:04.756657  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:04.766759  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.768673  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:04.785346  8413 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:04.793268  8412 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:04.744395 11941 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:27:04.799072  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:04.803273  7655 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:04.780345 11868 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:04.813207  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.599720 12109 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.817984  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:04.818672  7661 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:04.820643 11856 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.803006 12070 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.823413  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.824182  8405 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:04.832044  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:04.832242  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99365us
I20260709 12:27:04.832927  7655 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:04.834599  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.839861  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:04.829144 12110 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:27:04.773245 11862 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:04.848166  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:04.847803  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.853088  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:04.859504  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:04.860805  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:04.844485 12111 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 12:27:04.862257  8406 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:04.847365 11837 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:04.865906  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:04.865262 12070 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:04.864460 12074 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:04.875800 11927 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.001s
I20260709 12:27:04.868778 12008 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:04.908305 12109 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:04.932132 11995 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:04.933133 11995 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:04.933655 11995 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:04.934100 11995 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:04.866798 12001 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:27:04.913602  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.875403 11837 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:04.945277 11837 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:04.887487  7730 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:27:04.923774 11927 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:04.920285 11823 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:27:04.954783  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:04.921495 12074 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.959303  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:04.969340  8408 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:04.952344  7661 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
W20260709 12:27:04.955749 12113 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:27:04.986517  8405 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:04.990892  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:04.955651 12007 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.010658  7661 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:27:04.977187 12077 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.796s	user 0.004s	sys 0.000s
I20260709 12:27:04.970697 12074 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:05.046110 12128 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:27:05.034564 12122 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:05.048274 12122 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:05.049211 12122 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:05.047700 12132 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:27:05.064662  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 198846us
I20260709 12:27:05.036698 12109 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:27:05.036489 12007 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:05.095927 12094 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.857s	user 0.004s	sys 0.001s
W20260709 12:27:05.090240 12010 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.584s	user 0.016s	sys 0.000s
W20260709 12:27:05.126610 12045 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.672s	user 0.017s	sys 0.000s
I20260709 12:27:05.127655 12074 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.066116  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:05.170890  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104746us
I20260709 12:27:05.188773  7732 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:05.189615  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.190402  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2421) took 1185 ms. Trace:
W20260709 12:27:05.191676  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2569) took 2506 ms (client timeout 2627 ms). Trace:
I20260709 12:27:05.192200  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:05.192243  7735 rpcz_store.cc:269] 0709 12:27:02.685061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.779558 (+ 94497us) service_pool.cc:224] Handling call
0709 12:27:05.191044 (+2411486us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002f1554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:05.191412 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670890240}
W20260709 12:27:05.190467  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.209697  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2408) took 2502 ms. Trace:
I20260709 12:27:05.211381 11868 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.191077  7732 rpcz_store.cc:276] 0709 12:27:04.004644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.014001 (+  9357us) service_pool.cc:224] Handling call
0709 12:27:04.014333 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.188129 (+1173796us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b540001e120. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f241c0 00007fc001f25231 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c
0709 12:27:05.189785 (+  1656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.190229 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167074176}
I20260709 12:27:05.210373  7653 rpcz_store.cc:276] 0709 12:27:02.702297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.855472 (+153175us) service_pool.cc:224] Handling call
0709 12:27:02.857788 (+  2316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.752409 (+1894621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.186766 (+434357us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400116554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:05.204367 (+ 17601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4355497344}
I20260709 12:27:05.220175 11929 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:05.222312 12143 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:27:05.228438  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.225360 12074 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.227514 12112 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.114899 11927 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.250252 12146 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:05.251906 12146 consensus_queue.cc:1243] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:05.252624 12146 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:05.253039 12146 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:05.223132 11871 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:05.263677 12044 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:05.264412 12044 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:05.269654  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.275563 11856 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.275830 11863 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:05.277017 11856 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.277278 11863 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.274849 11842 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.279987 11840 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.280186 11842 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.281461 11840 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:05.282533 12131 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
W20260709 12:27:05.280308 12135 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.322s	user 0.000s	sys 0.004s
I20260709 12:27:05.286177 11896 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:27:05.297278  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.298170  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.299450 12024 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.269697  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2685) took 1766 ms. Trace:
W20260709 12:27:05.287125  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2572) took 2600 ms (client timeout 2682 ms). Trace:
I20260709 12:27:05.266342 12044 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:05.304515  7730 rpcz_store.cc:269] 0709 12:27:02.686048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.796978 (+110930us) service_pool.cc:224] Handling call
0709 12:27:05.286418 (+2489440us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540039b7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:05.286847 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4804107520}
I20260709 12:27:05.308470 11840 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.312558 11842 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:05.319298  7732 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:05.320367 11952 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:05.322093 11997 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:05.322921  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:05.323632 12023 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.323943  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:05.324760 12023 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.258085 12146 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.325974  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:05.326563  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:05.329358  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 101279us
W20260709 12:27:05.333729  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88241.5us
I20260709 12:27:05.288775 11863 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.334954  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:05.335157  7735 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:05.364313 11842 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.358018  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:05.348443  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.359416  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:05.360538  7656 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.352264  7729 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
W20260709 12:27:05.371417  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 100293us
I20260709 12:27:05.358712 11863 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:05.354522 12146 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:05.375703  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:05.352876  7659 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:05.378502  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.383303  7811 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [9.9-10.10]
I20260709 12:27:05.362110 11840 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.404523  7808 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:27:05.406872 11863 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.407481  7653 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:05.408329  7732 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:05.336969 12007 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.348802 12024 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.411178  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:05.340462 11824 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:27:05.413046  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:05.414109 12010 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.415159  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.419445 12081 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.420109  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:05.421283  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.424209  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.426048  7811 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:27:05.365417 11842 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:05.404508  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46810.6us
I20260709 12:27:05.354693 11868 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.377604 12023 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:05.440480  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.359865 12022 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:27:05.454165  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:05.480754  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60956.5us
I20260709 12:27:05.490190 12153 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.490469  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.491875  7811 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:27:05.491274 12153 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.492317  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:05.494010  7729 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:05.496563 11863 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:05.497082 12024 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.498068  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:05.497808 12122 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:05.499719 12122 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:05.500479 12122 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:05.502501 12122 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.483120  7804 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.504684 11896 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:05.505529 12005 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:05.478120 12022 consensus_queue.cc:1243] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:05.505522 12122 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.510537 12023 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:27:05.511096 12023 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:05.512537 12023 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.514540 12023 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:27:05.514807 12126 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.255s	user 0.004s	sys 0.000s
W20260709 12:27:05.516860  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.518249 12153 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.523048  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.523814 12081 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.524999  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.526563  7811 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:05.524518  7808 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.521122  7806 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:05.483771  7803 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:27:05.528680  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.544855  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:05.547631  8405 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:05.529208  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.539029  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:05.552380  7803 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:05.527640  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:05.537536 12150 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.297s	user 0.001s	sys 0.003s
I20260709 12:27:05.488109 12010 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.577075  8406 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:05.577562 11856 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.578406  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:05.578693  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104713us
I20260709 12:27:05.580073  7733 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:05.302161  7805 rpcz_store.cc:276] 0709 12:27:03.503727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.503898 (+   171us) service_pool.cc:224] Handling call
0709 12:27:03.504284 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:03.504692 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.504721 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:03.512198 (+  7477us) log.cc:844] Serialized 52 byte log entry
0709 12:27:03.679595 (+167397us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:03.837773 (+158178us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.842260 (+  4487us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.267793 (+1425533us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b1800278260. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:05.268713 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.268894 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.269093 (+   199us) raft_consensus.cc:1685] finished
0709 12:27:05.269109 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.269369 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1445995,"prepare.queue_time_us":407372,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":4776,"raft.queue_time_us":1431668,"raft.run_cpu_time_us":868,"raft.run_wall_time_us":10523,"spinlock_wait_cycles":4071808,"thread_start_us":319997,"threads_started":2,"wal-append.queue_time_us":268115,"wal-append.run_cpu_time_us":6269,"wal-append.run_wall_time_us":1246887}
I20260709 12:27:05.580966  7811 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:05.582129  7803 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.527122 12081 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:05.583417  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 149153us
I20260709 12:27:05.535071 11896 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:05.588147 12134 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:05.591180 11842 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:05.593771  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:05.598639  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.77167e+06us
I20260709 12:27:05.599977  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:05.578693  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:05.602397  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:05.603334  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.603724  7729 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:05.607069  7806 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:05.609977  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:05.544142 12109 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:05.651445 12109 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:05.652427  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:05.652794 11863 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:05.653402  8409 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:05.653278 12109 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.654589  8405 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:05.657687  7662 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:05.505671  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.659909  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.661433  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.662626  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.601222  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:05.662925 11951 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.665426 11951 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:05.665977 11951 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:05.668270  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:05.634845 11995 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.700s	user 0.020s	sys 0.007s
I20260709 12:27:05.537225 11840 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.680840  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:05.635641  7803 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:05.636025  7655 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:05.636425  7731 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.639803  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.639567 11896 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.629951  7729 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:05.641762  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.618875 12044 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:05.642227 12074 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:05.706511  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.707533  7811 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:05.539423 12155 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:05.643779  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.644670  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:05.645891  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.646319  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:05.647578 11837 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.010s
I20260709 12:27:05.648931 12153 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.649931 11904 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.531937 12156 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:05.692384  7653 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:05.693809  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:05.695245 11995 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.698894  7810 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:27:05.700052  7729 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.717864  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:05.705034  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:05.643522 12108 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.710754  7657 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:05.649982  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:05.715397  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.720785  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:05.741254  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95541.9us
I20260709 12:27:05.750917  7654 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:05.752435  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:05.753283  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:05.753924  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104076us
I20260709 12:27:05.754161  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:05.754882  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.756138 11854 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:27:05.756659 11854 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:05.757094 11854 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:05.757052  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:05.758348  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:05.759210  7729 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.760447  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:05.761144  7811 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11. Re-sending same reply.
I20260709 12:27:05.762161  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.762800  7811 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:27:05.763953 12108 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:05.764540 12108 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:05.765805 12108 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.767697  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.768626  7807 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:05.769723  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.770376  7807 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:05.771431 11904 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.776845  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.722867 11856 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.734807 11842 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:05.735284 12156 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:05.798055  7808 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:05.775698  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:05.796800  7662 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:05.793449  7729 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:05.774247 11904 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.800982  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.796321  7803 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:05.806514  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:05.811384  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 171755us
I20260709 12:27:05.812588  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.791272 12022 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:05.815927  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.816701  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.775370 12134 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.722055 11837 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.819401  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.794420 11842 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:05.734346 12153 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.858777  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:05.824678  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 343172us
I20260709 12:27:05.825922  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:05.823954  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:05.865170  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51829.9us
I20260709 12:27:05.814707 12109 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.818388 11840 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:05.821573 12022 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.874930  7656 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:05.883934  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.885658  7808 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:27:05.876950  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51078.6us
I20260709 12:27:05.877909  7730 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:05.818921 11863 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:05.871593  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:05.790158 12155 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.868625  7804 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:05.860116 11842 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.893584  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:05.897822  7555 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 6 to 7, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:05.900933  7555 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:05.902314  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.742591 11951 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:05.908602  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 262478us
I20260709 12:27:05.911652  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.889287 12022 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.922590 11975 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.922331  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:05.923825 12073 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.924314 11841 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 12:27:05.925153 11996 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:05.925581 12075 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:05.908414 11970 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.186s
I20260709 12:27:05.909425 12010 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:05.873242 11856 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:05.933374  7806 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.933991  7731 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:05.903256 12154 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:05.933276  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:05.937212  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:05.908262 11932 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.891954 11863 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:05.908356 11872 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:05.883065 11854 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.947057  7808 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:05.939891 11833 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:27:05.887969 12134 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:05.931609 11896 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:05.951928 11863 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.937836 11996 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.962337  7811 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:27:05.967406  7558 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 284) took 23919 ms (client timeout 15000 ms). Trace:
I20260709 12:27:05.967692 12074 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:05.968528 11823 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:05.970081  7804 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 6. Current term is 7. Ops: []
W20260709 12:27:05.978813  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.949975 12154 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:05.960578 11841 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:27:05.994531  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 183351us
I20260709 12:27:05.960660 11951 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:05.990963  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 155581us
I20260709 12:27:05.959992 11975 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.001098  7804 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:27:05.981948 11996 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:06.006230  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.997306  7555 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 8, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:06.009356  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 145820us
I20260709 12:27:05.988476 11823 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:06.007594  7807 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:06.000018 11896 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:05.991046 12154 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:05.995913 12074 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:06.012868 11997 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.011s
W20260709 12:27:06.046713  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67312.6us
I20260709 12:27:06.028708 11823 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:06.033102  7555 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 2 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.078251 12082 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.061609 12108 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:06.065230 11837 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:06.080798  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 131811us
I20260709 12:27:06.074986  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:06.017545 12073 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:27:05.993552 12156 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.083549 12075 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:06.114563 12167 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
W20260709 12:27:06.115442 12162 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 12:27:06.116256 12153 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.117182 12074 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.094381  7555 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:06.079069 12082 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.123164 11932 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.144048 11932 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.144362  7737 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:27:06.146445 11932 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.123692 11896 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:06.126541  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 113260us
I20260709 12:27:06.127887 12085 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 12:27:06.129642 12163 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:27:06.150887 12085 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:06.151517 12085 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:06.151898 12022 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.152657 12085 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.154103 12085 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:06.154671 12022 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.142968  7555 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:27:06.165007  7810 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:06.114178  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:06.173355 11833 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:06.092197 12006 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.179576  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89283.2us
W20260709 12:27:06.179926  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 262231us
I20260709 12:27:06.116267 12155 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.107635 11970 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:27:06.202188  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:06.175112 11833 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
W20260709 12:27:06.207983 12166 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:27:06.208498 12173 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:06.209311 11837 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.209159 12173 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.209780 11837 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.210192 11837 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.211859 12173 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:06.203333 11932 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 12:27:05.978102  7558 rpcz_store.cc:269] 0709 12:26:42.048318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.052926 (+  4608us) service_pool.cc:224] Handling call
0709 12:26:43.391175 (+1338249us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b5400002210. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfe0b2 00007fc007d77759 00007fc007d0fa76 00007fc007ce0d58 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc
0709 12:27:05.966968 (+22575793us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:59.266077 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:59.581698 (+315621us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:59.581776 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:26:59.581798 (+    22us) tablet.cc:662] Decoding operations
0709 12:26:59.682331 (+100533us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:59.682466 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:59.682508 (+    42us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:26:59.794670 (+112162us) tablet.cc:701] Row locks acquired
0709 12:26:59.794734 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:26:59.795689 (+   955us) write_op.cc:270] Start()
0709 12:26:59.795865 (+   176us) write_op.cc:276] Timestamp: P: 1783600019794854 usec, L: 0
0709 12:26:59.795911 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:26:59.976436 (+180525us) log.cc:844] Serialized 9768 byte log entry
0709 12:27:01.190581 (+1214145us) op_driver.cc:464] REPLICATION: finished
0709 12:27:01.682555 (+491974us) write_op.cc:301] APPLY: starting
0709 12:27:01.682681 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:02.582382 (+899701us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:02.582563 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:02.803845 (+221282us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:02.885979 (+ 82134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 12:27:02.886045 (+    66us) write_op.cc:312] APPLY: finished
0709 12:27:02.886751 (+   706us) log.cc:844] Serialized 220 byte log entry
0709 12:27:03.038767 (+152016us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:03.058314 (+ 19547us) write_op.cc:454] Released schema lock
0709 12:27:03.077864 (+ 19550us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:03.135164 (+ 57300us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:03.137091 (+  1927us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":337505,"reported_tablets":49,"spinlock_wait_cycles":6061696,"child_traces":[["op",{"apply.queue_time_us":482361,"apply.run_cpu_time_us":65168,"apply.run_wall_time_us":1408936,"cfile_cache_hit":98,"cfile_cache_hit_bytes":102410,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":2,"lbm_read_time_us":56672,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":8,"num_ops":25,"prepare.queue_time_us":397422,"prepare.run_cpu_time_us":26531,"prepare.run_wall_time_us":1197412,"raft.queue_time_us":307197,"raft.run_cpu_time_us":56985,"raft.run_wall_time_us":415747,"replication_time_us":1365208,"thread_start_us":1097962,"threads_started":5,"wal-append.queue_time_us":691685,"wal-append.run_cpu_time_us":26829,"wal-append.run_wall_time_us":2562923}]]}
I20260709 12:27:06.201428 11997 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:27:06.220072 11975 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.220875  7811 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
W20260709 12:27:06.221810 12044 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 12:27:06.221863  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.222566 11833 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:06.222673  7811 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:06.222997  7735 raft_consensus.cc:2410] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:27:06.228526 12082 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.230175  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 843) took 3717 ms. Trace:
I20260709 12:27:06.230358  7811 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:06.230614  7737 rpcz_store.cc:276] 0709 12:27:02.512761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.513011 (+   250us) service_pool.cc:224] Handling call
0709 12:27:02.531776 (+ 18765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.145107 (+3613331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.229901 (+ 84794us) spinlock_profiling.cc:243] Waited 3.61 s on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:06.230074 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8306425216}
I20260709 12:27:06.189600 12168 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:06.223278 11856 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.226460  7808 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:06.226622  7807 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:06.216812 12169 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:06.267683 12109 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:06.268352 12109 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:06.269480 12109 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.234207 12154 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:06.273932  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.274281 12154 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:06.279886 12134 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.281379 12146 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:06.282440  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.282413 12146 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:06.283664  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.288946  7806 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
W20260709 12:27:06.286283  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 553446us
I20260709 12:27:06.290647  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:06.175207 12010 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:27:06.201869 12122 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.694s	user 0.021s	sys 0.004s
W20260709 12:27:06.201642 12023 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.686s	user 0.017s	sys 0.003s
W20260709 12:27:06.303256  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.307907  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:06.291997  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 175274us
I20260709 12:27:06.223165 11833 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:06.234711 11854 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.224723  7555 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 7 to 8, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195), now has a pending config: VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194), VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195), VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:06.325917  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:06.328236  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.320963 11975 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:06.204728 12006 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.347394 12010 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:06.348709 12022 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.349504 12022 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.350589 12122 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.350042 11833 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.348766 12023 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.351204 12122 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.351773 11833 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.349536 12006 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.353587 12022 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:06.360131  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 138369us
I20260709 12:27:06.362943  8405 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:06.215822 12075 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:06.385782  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95648.8us
I20260709 12:27:06.356598 12074 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:06.353595 12122 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:06.393359  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 338958us
I20260709 12:27:06.352074 12023 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.396878  7555 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.402966  7804 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:06.405532  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.408033  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.409008  7735 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:06.415750  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:06.416826  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:06.419044  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.420775  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:06.421947  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110634us
I20260709 12:27:06.423684  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.424701  7731 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:06.421070  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.432942  7730 raft_consensus.cc:2410] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:27:06.396018 11975 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 12:27:06.441535  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.434954  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.433589  7555 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:06.444623  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.444859 12146 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:06.467563  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53523.3us
W20260709 12:27:06.470280  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51325us
I20260709 12:27:06.474323 11716 raft_consensus.cc:2764] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:06.475350  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:06.475908  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.477334 12112 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.484033  7555 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.485924  7555 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 5, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:27:06.487736 11975 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:06.488266 11975 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:06.489401 11975 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.491179 12112 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:06.434487 12074 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:06.499577 12074 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.455987 12022 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:27:06.509755 12006 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:06.434531 11868 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 12:27:06.520009  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 106196us
I20260709 12:27:06.532783  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:06.497956  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67439us
I20260709 12:27:06.532434  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:06.472227 12178 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 12:27:06.542742  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66915.5us
I20260709 12:27:06.559875  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.544034  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.570099  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:06.571188  8405 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:06.531098 12112 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:06.502710 11863 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.538954 12122 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:27:06.572943 11996 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.002s
I20260709 12:27:06.537820 12023 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:06.589061  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.583822  7555 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 8, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:06.595067  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.559796 11975 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:06.563390 12010 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.602982  7811 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:06.604485  7807 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:06.604449  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 180264us
I20260709 12:27:06.576273 12156 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:06.582239 12112 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:06.535532  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62459.3us
I20260709 12:27:06.569062 11840 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:06.581176 11868 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.585381 12006 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:06.583628 12122 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:06.577111 12186 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:27:06.609936 12112 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:06.614936  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71025us
I20260709 12:27:06.602332 12010 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.576865 11823 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.611137  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:06.623518  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 84960.4us
I20260709 12:27:06.599743 11996 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.624856 12170 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:27:06.625370  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:06.625515  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.631788 12171 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:06.633741 11837 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.614002 11840 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:06.634960  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 702) took 1044 ms. Trace:
I20260709 12:27:06.621624 12122 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:06.613042 11868 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.641439  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:06.645423  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:06.651183  8405 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:06.635298  7555 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.636430 12010 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.662601  7555 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.663657 11868 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:06.664057  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53004.9us
I20260709 12:27:06.651765  7812 rpcz_store.cc:276] 0709 12:27:05.604036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.124209 (+520173us) service_pool.cc:224] Handling call
0709 12:27:06.124702 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.413009 (+288307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.413590 (+   581us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.443735 (+ 30145us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.582141 (+138406us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:06.582179 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.582253 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.634563 (+ 52310us) raft_consensus.cc:1685] finished
0709 12:27:06.634585 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.634773 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45310,"raft.queue_time_us":149965,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":11626,"thread_start_us":360008,"threads_started":2,"wal-append.queue_time_us":176572}
I20260709 12:27:06.665472 11840 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:06.666280 11951 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.649s	user 0.014s	sys 0.000s
I20260709 12:27:06.667909 11837 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.687116  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.687350  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:06.670449  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.671245  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2695) took 1063 ms. Trace:
I20260709 12:27:06.671564  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.608242 12172 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:06.682160  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:06.608122 12182 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:27:06.680600  8409 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:06.684559  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:06.678932 12156 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:06.681582  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:06.693749  8406 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:27:06.695389  7659 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 12:27:06.689644  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 81906.9us
I20260709 12:27:06.689054  7806 rpcz_store.cc:276] 0709 12:27:05.607361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.434728 (+827367us) service_pool.cc:224] Handling call
0709 12:27:06.456287 (+ 21559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.531558 (+ 75271us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:06.532033 (+   475us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.669154 (+137121us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.669844 (+   690us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.670814 (+   970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.670870 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:06.670923 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.671004 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.671153 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16165,"prepare.queue_time_us":16646,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":127,"raft.queue_time_us":3329,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":11061,"spinlock_wait_cycles":45952,"thread_start_us":119829,"threads_started":1}
W20260709 12:27:06.738015  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56522.3us
I20260709 12:27:06.738864  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.739368  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2698) took 1131 ms. Trace:
W20260709 12:27:06.739694  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.740932 11716 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:06.647914 12146 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:27:06.682006 12073 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.005s	sys 0.018s
I20260709 12:27:06.658272 12006 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:06.762092  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.679383 11996 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:06.701254  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 699) took 1111 ms. Trace:
I20260709 12:27:06.709076  7555 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:27:06.745142  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.701431  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:06.698879 12187 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:06.768244  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:06.767242  7804 rpcz_store.cc:276] 0709 12:27:05.589646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.035487 (+445841us) service_pool.cc:224] Handling call
0709 12:27:06.402707 (+367220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.403611 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.700993 (+297382us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001329d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:06.701180 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785197312}
I20260709 12:27:06.772805  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.775759  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:06.777128  7806 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:06.691454 12195 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.739847  7811 rpcz_store.cc:276] 0709 12:27:05.608330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.612207 (+1003877us) service_pool.cc:224] Handling call
0709 12:27:06.612646 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.622422 (+  9776us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:27:06.667456 (+ 45034us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.738971 (+ 71515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.739030 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:06.739072 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.739143 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.739308 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27420,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":71061,"threads_started":1}
I20260709 12:27:06.720331 12081 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:06.795046  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 143570us
W20260709 12:27:06.795325  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 108331us
I20260709 12:27:06.795473  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2694) took 1188 ms. Trace:
I20260709 12:27:06.796128  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:06.796166  7810 rpcz_store.cc:276] 0709 12:27:05.607041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.312803 (+705762us) service_pool.cc:224] Handling call
0709 12:27:06.333901 (+ 21098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.344080 (+ 10179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.344130 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.359622 (+ 15492us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.541593 (+181971us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:06.541665 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.541826 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.795117 (+253291us) raft_consensus.cc:1685] finished
0709 12:27:06.795148 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.795391 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1548,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":64917,"threads_started":1,"wal-append.queue_time_us":203433}
W20260709 12:27:06.799932  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 56502.2us
I20260709 12:27:06.800012 12186 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:06.800918 12182 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.801224  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.802008 12081 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:06.803663 12186 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:06.803383 12081 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.804174 12186 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:06.804647 12186 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:06.805729  7555 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 1 to 4, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:06.770905 11840 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.807482 12085 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:06.808053 12085 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.807783 11872 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:06.808470 12085 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:06.801646 12182 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.700695 11823 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.812975  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:06.755641 11837 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:06.763751 12170 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.788830 11842 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.858s	user 0.009s	sys 0.006s
I20260709 12:27:06.825214  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:06.763792 11951 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:06.796135  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:06.827111  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:06.789189 12195 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.820008 11837 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.831125 11951 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.860646  7812 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:06.833392 12082 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.862317 12170 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.866019  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:06.862145 12185 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:27:06.845607 12195 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.854463 11837 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:06.870615  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52354.5us
I20260709 12:27:06.845984 12075 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:06.868135  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:06.893473  7737 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:27:06.879405  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 196576us
I20260709 12:27:06.833526 11842 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.868984 11951 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:06.890947  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:06.905179  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:06.907881  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 111840us
I20260709 12:27:06.848203 12010 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.902523  7732 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:06.915817  7735 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:27:06.918917  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.869982 11716 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:27:06.924929  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 131906us
I20260709 12:27:06.928547  7803 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:06.892526 12075 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:06.931818 12190 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:27:06.902549 12176 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:06.933080 11995 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:06.994232 12203 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:06.994134 12186 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:06.934075 11904 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:27:06.936429 12200 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:27:06.995200 11904 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:06.902633 12198 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:06.912801 12023 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:27:06.974022  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47619.3us
I20260709 12:27:06.981726  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.922160 12082 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:06.993508  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.993839  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:06.996812  8412 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:07.005968  7735 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 12:27:06.964313  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59239.7us
I20260709 12:27:06.993919  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:06.994752 11995 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:07.002336  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.009419  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.003357  7737 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:06.944466 12075 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.015559  7812 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:07.037982  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.014026 12185 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:07.010057 12023 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:07.043300 12023 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:07.045027  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2247) took 1452 ms. Trace:
I20260709 12:27:07.045634  7809 rpcz_store.cc:276] 0709 12:27:05.592699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.137067 (+544368us) service_pool.cc:224] Handling call
0709 12:27:06.137604 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.190192 (+ 52588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.190229 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.199037 (+  8808us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.433241 (+234204us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:06.614339 (+181098us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.617543 (+  3204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.623430 (+  5887us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.044547 (+421117us) raft_consensus.cc:1685] finished
0709 12:27:07.044588 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.044804 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56914,"prepare.queue_time_us":46686,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":146095,"raft.run_cpu_time_us":968,"raft.run_wall_time_us":13647,"thread_start_us":269606,"threads_started":2,"wal-append.queue_time_us":332934}
I20260709 12:27:07.000173 11823 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.047351  7812 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:27:07.048132  7809 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:07.047909 11823 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.050722  8408 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:07.051723 11823 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.060534 12023 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.037875  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2243) took 1514 ms. Trace:
I20260709 12:27:07.038352 12212 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:07.040444  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2246) took 1448 ms. Trace:
I20260709 12:27:07.029623  7805 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:07.066458  7808 rpcz_store.cc:276] 0709 12:27:05.523065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.973728 (+450663us) service_pool.cc:224] Handling call
0709 12:27:06.226008 (+252280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.235373 (+  9365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.037608 (+802235us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54001338d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:07.037797 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521913088}
I20260709 12:27:07.067288  7803 rpcz_store.cc:276] 0709 12:27:05.592302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.816305 (+224003us) service_pool.cc:224] Handling call
0709 12:27:07.040105 (+1223800us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:07.040293 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456136960}
I20260709 12:27:07.070477  7732 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:27:07.002660 11824 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:27:07.070945  7732 pending_rounds.cc:85] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 4
I20260709 12:27:06.995003 12203 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.073944  7732 pending_rounds.cc:107] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:27:07.078758  7732 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.078706 12199 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.079524  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.027607 11995 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:07.098701  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89962.9us
I20260709 12:27:07.078881 11970 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:27:07.102892  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 105679us
I20260709 12:27:07.064086 11904 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.113768  7735 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:07.090837 12204 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.103412 12201 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.105528  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:07.122727  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.104424 11970 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:07.125283 11970 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.126328 11932 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:27:07.126533 11970 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:07.126945 11932 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:07.115125 12023 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.121421 11904 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:07.132118 11833 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:07.132675 11833 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:07.121763  7734 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:07.133788 11833 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.175401 12010 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.175595 11833 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:07.177213 12082 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:07.177551  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53666.5us
I20260709 12:27:07.177821 11932 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.178705  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:07.179636 11932 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:27:07.179884 12182 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.180845 11842 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.106894 11716 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:27:07.096823 12199 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.121553 12074 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.107257 12194 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:27:07.186297 12074 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.188612 12186 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.216519  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.217604  7734 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:27:07.135273 12204 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:07.246522  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 112735us
I20260709 12:27:07.257088 12085 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.245051 11716 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:07.258867 11716 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:07.260242 12204 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.260043 11716 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:07.261689 12112 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.015s
I20260709 12:27:07.262164 12074 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.263288 12109 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:07.263688 12203 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:07.263921 11975 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.663s	user 0.020s	sys 0.006s
I20260709 12:27:07.215425 12182 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.270859  7728 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:07.273531  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.276520  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 171089us
W20260709 12:27:07.278692  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100098us
I20260709 12:27:07.239466 11823 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.214210 12122 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.285882  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:07.292234  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.296118  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.297282  7653 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:07.292511  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.301128  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2693) took 1694 ms. Trace:
I20260709 12:27:07.301610  7807 rpcz_store.cc:276] 0709 12:27:05.606736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.293614 (+686878us) service_pool.cc:224] Handling call
0709 12:27:06.319713 (+ 26099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.608390 (+288677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.300822 (+692432us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400332054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:07.301054 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645117184}
I20260709 12:27:07.307459  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.315550  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.316289  7662 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:07.309441  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.319743  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:07.295161 11841 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.208s	user 0.018s	sys 0.016s
I20260709 12:27:07.320536  7809 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:07.326303  7660 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:07.278400 12203 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:07.275202 12186 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:07.301939 11951 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.314357 11823 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.287889 12204 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:07.355875  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:07.358105  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.239848 12201 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:07.326978 11841 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.247s	user 0.018s	sys 0.016s
W20260709 12:27:07.364977  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68404.6us
I20260709 12:27:07.366408  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.262215 12112 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.315887 12205 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.363193  7662 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:07.379763  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.380438  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.381124  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:07.367847  7653 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:07.369271  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.377070  8409 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 6: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:07.283941 12182 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.370255  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.302386 12168 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:27:07.382545  7657 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:07.388784  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:07.388569  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.389508  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:07.302039 12073 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:07.352430 12146 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:07.411412  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.359355 12186 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:07.415412  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97461.6us
W20260709 12:27:07.417992  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 124660us
I20260709 12:27:07.419189  7729 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:07.313472 11896 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
W20260709 12:27:07.427922 11840 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.010s
I20260709 12:27:07.428550 11840 raft_consensus.cc:2764] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:07.429925  7810 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:27:07.412684 11868 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.726s	user 0.023s	sys 0.011s
I20260709 12:27:07.426126  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.441983  7734 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:07.381312 12199 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:07.447633  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.451588  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.391548 12081 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.413038 12085 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.455529  7654 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:07.465747  7729 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:07.419075 12186 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:07.468822  7735 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:07.474439  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58621.8us
I20260709 12:27:07.370834 12203 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.425119 11975 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.392412 12204 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.479329  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.392266 12205 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.483794  7729 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:07.461213 11995 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.491253  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.441251 12146 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.468174 12223 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:07.472246 12217 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:07.512180 12170 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.518675  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.532776 12211 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:27:07.543979  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 704) took 1939 ms (client timeout 1713 ms). Trace:
I20260709 12:27:07.552402  7734 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:07.553692  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.556857  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:07.558045  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.560554 11863 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:07.601737  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:07.599669  7805 rpcz_store.cc:269] 0709 12:27:05.604863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.667883 (+ 63020us) service_pool.cc:224] Handling call
0709 12:27:07.543614 (+1875731us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400295754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:07.543821 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348912512}
W20260709 12:27:07.561246  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.565274  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 706) took 1816 ms. Trace:
W20260709 12:27:07.568137 12214 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:27:07.569972 12218 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:27:07.573035  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.574193 12182 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.616768  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.617545  7653 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:07.617653  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 261552us
I20260709 12:27:07.583112  7731 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:27:07.619827  7731 pending_rounds.cc:85] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:07.620328  7731 pending_rounds.cc:107] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:27:07.620644  7731 raft_consensus.cc:2889] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:07.587564  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.589974  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:07.455659 11868 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.472417 12081 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.599241  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.604143  7655 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:07.604002  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:07.601388  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.604754  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:07.595875  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 151025us
I20260709 12:27:07.635986  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:07.606786  7810 rpcz_store.cc:276] 0709 12:27:05.749304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.798597 (+1049293us) service_pool.cc:224] Handling call
0709 12:27:06.799066 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.430322 (+631256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.564861 (+134539us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400132ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:07.565168 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449875840}
I20260709 12:27:07.637903  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.638805  7810 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:07.613140  7661 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
W20260709 12:27:07.602380  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 84153.7us
I20260709 12:27:07.616282  7654 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:27:07.595584 12223 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.628664  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:07.644572  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.601691 12181 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.653548  8412 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:07.654723  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.656453  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.601526 11975 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:07.660184  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70319.8us
W20260709 12:27:07.666071  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 276631us
I20260709 12:27:07.658221  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.599493 12211 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:07.668351  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:07.680023  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.680823 12134 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:07.681751  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.681947 12146 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:07.682636  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46799.5us
I20260709 12:27:07.683111  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.628381 11823 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.596135 11896 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:27:07.661863 11716 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.612848 12226 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.671267  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.671123  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.625299 11951 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.674559 12174 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:07.675246 12112 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.676259  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.677651 12203 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:07.647200 12176 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.617720 12122 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.666225 11863 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:07.655339 12073 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:07.652964 12223 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:07.683836  8415 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:07.683152  7733 raft_consensus.cc:2368] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:07.683178  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.697953  7735 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:07.701181  7653 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:07.713917  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.693902  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:07.760701  7804 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
W20260709 12:27:07.761127  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67340.1us
I20260709 12:27:07.698244  7734 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:07.721534  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:07.674553  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.713299  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.756613  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.687527  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.741052  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.764424  7728 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:07.764518  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.768796  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:07.769491  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:07.721675 12218 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:27:07.690442  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.688655 11872 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.772851  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.773211  7661 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:07.773975  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.691190  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.685659 12201 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:07.693578 12204 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:07.685256 12224 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:07.692554 12205 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:07.772032  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50581.3us
I20260709 12:27:07.771077 12112 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:07.785673  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.789144  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.691617 12182 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:27:07.688380 12230 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:27:07.796201  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:07.781283  7805 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:07.791679  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:07.794536  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.830780  7728 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:27:07.834228  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:07.812220  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:07.785138 11833 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.609s	user 0.018s	sys 0.002s
I20260709 12:27:07.693506 12146 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:27:07.693361 11995 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
W20260709 12:27:07.819176 11970 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.005s
I20260709 12:27:07.722957 12156 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:27:07.692561 12170 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:27:07.794066  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.818535  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.672748 12181 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.786397 11872 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.820041  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.694065 11716 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.820683  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:07.819901  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:07.886562  7654 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:27:07.819828  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.726151 12226 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:07.840269  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44192.9us
I20260709 12:27:07.888748  8412 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:07.841801  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.889981  7653 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:07.844002  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.848124  7700 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:07.848941  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.796271  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:07.796614 11823 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.816362 12112 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:07.851900  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.871538  7730 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:07.766997 12073 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.905843  7661 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:07.906826  7732 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:07.850044 12193 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:27:07.911425  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115423us
I20260709 12:27:07.912489  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:07.874877  8413 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:07.874925  8407 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9. Re-sending same reply.
W20260709 12:27:07.860410 11837 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.014s
W20260709 12:27:07.825624 11904 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.016s
I20260709 12:27:07.778297 12229 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:27:07.862511 12174 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:07.897892  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:07.819141 12224 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:07.766544 12176 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.944363  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.877591  8415 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:07.872834  7728 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:07.923763  7731 raft_consensus.cc:2410] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:27:07.930401  7734 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:27:07.934283  7661 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:07.945678  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:07.953759  7732 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:07.910782 11842 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:07.940968  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.975931  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.921072 11997 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:07.942655  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.927177 11970 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.999831  7657 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:07.908082 12112 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.945853  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:07.946202  7662 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:27:07.946311  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:07.905349 12186 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:07.938534  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91341.5us
I20260709 12:27:07.942271  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.009876  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:07.921427 12201 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.929944 12085 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:07.929551 12022 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:08.017616  8413 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:08.018607  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:08.020210  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.905699  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:07.989409  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.022500  7653 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:08.022590  7805 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:27:08.001083  7654 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:27:08.002139  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 154142us
I20260709 12:27:07.905985 11840 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:27:07.933727 12174 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:07.949132  7734 pending_rounds.cc:85] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:08.052381  7734 pending_rounds.cc:107] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:08.052812  7734 raft_consensus.cc:2889] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.053195  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:07.949343 11868 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.055403  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:07.908219 11823 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.006822  7807 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:08.082305  7732 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:07.956624 11904 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:07.933424 11837 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:07.988888 11975 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.055748  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.056285  7728 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:08.094280  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:08.056968  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.007817 11970 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.099170  7732 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:27:08.083802  8415 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:27:08.004560 12081 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.084610  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.094410  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:08.067346  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.005510 12156 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:27:08.074743 12174 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.016696 12085 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.099985  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.109112  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.095760  7806 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:08.081818 12108 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.134354 12108 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.135421  7732 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:27:08.137104 12108 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:08.138023  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.138988  7807 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:08.140594  7807 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:08.142220  7807 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:27:08.099408 12172 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:08.145232 11970 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.135056 12176 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:08.147465  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:08.148571 12239 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:08.148912 11837 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.115887  7728 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
W20260709 12:27:08.124934  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 227119us
I20260709 12:27:08.163726  7728 pending_rounds.cc:85] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:27:08.164024  7728 pending_rounds.cc:107] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:27:08.164335  7728 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.126508  7735 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:08.008522 11842 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:08.128377  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 188592us
I20260709 12:27:08.129366  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.173352  7806 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:08.006217 12232 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 12:27:08.095059 12209 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:08.147528 11716 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:08.088202 12234 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
I20260709 12:27:08.092144 11951 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:08.087661 11904 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:08.161680  7807 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:08.090739 12198 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:27:08.175850  7653 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:08.053345 12201 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.186408  7662 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:08.215492  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 262638us
I20260709 12:27:08.220871  8413 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
W20260709 12:27:08.171296  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86794.2us
I20260709 12:27:08.150707 12085 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.224938  7778 leader_election.cc:400] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:08.098639 12074 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:27:08.235972 12172 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:08.183851 12209 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.237427  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.210992 11951 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:08.239045 11842 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:08.239050 12073 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.188864 12134 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:08.240700 12254 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:08.211036 12108 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:08.190544 12178 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:08.226567 11904 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:08.188388 11840 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:27:08.280882 12172 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:08.262274  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.266801  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:08.232483  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.265861 12181 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:08.271799 12109 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.002s	user 0.013s	sys 0.002s
I20260709 12:27:08.272194  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.273720 11872 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.276389 12226 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.276806 12201 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:08.279548  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100460us
W20260709 12:27:08.281497  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44179.6us
I20260709 12:27:08.284037  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:08.290252  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.293977  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.264493 12212 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.296932  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.286352 12209 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:08.328372  8406 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:08.287022 12023 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:08.323050  7653 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:27:08.333644  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.262180  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 167860us
I20260709 12:27:08.339308  7653 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:27:08.339913  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.334354  7732 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:08.348851  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.350457  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.313620 12178 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:08.325505 12195 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.417s	user 0.022s	sys 0.004s
I20260709 12:27:08.352896  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.345189  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.355335  7734 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:08.320948 12204 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:08.360240  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.295855 12246 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:27:08.368667  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:08.343715 12075 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.308s	user 0.022s	sys 0.010s
I20260709 12:27:08.300165 12134 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.371816  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.374881  7728 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:27:08.381091  7653 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:27:08.383349  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.384411 12211 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.335417 12186 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.327926 12262 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.337450 12217 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.343264 12023 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.392875  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:08.395009  7736 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.004s
I20260709 12:27:08.395057  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.395637  7736 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:08.395978  7653 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:27:08.355829 11975 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:08.379884 12247 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:27:08.368618 12108 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:08.421047  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.343428 12073 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:27:08.394766 11932 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.215s	user 0.021s	sys 0.001s
W20260709 12:27:08.434397  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45244.8us
I20260709 12:27:08.434770 12274 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:08.368644 12201 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:27:08.378953 12081 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:08.373941 11868 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:08.442852  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.394796 12023 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:08.452291  7728 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:08.420717  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.449390  7737 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:27:08.344414 12212 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.460342  8407 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:08.462630  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.423146 12108 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.364656 12195 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.380687 12075 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.438364 11837 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:08.468344  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:08.444209 11716 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:08.456135 12081 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.459400 12073 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:08.437813 11975 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:08.492172 11975 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.437611 12156 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:08.390465 11951 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.474491 12108 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:08.394369 11842 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:27:08.470804 11868 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:08.479271  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:08.481422 12235 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:27:08.481678 12178 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:08.483868 12240 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 12:27:08.481438 12258 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 12:27:08.430315 12262 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.513329  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.436837 12216 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:27:08.515245  8408 raft_consensus.cc:2368] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:08.526050 11868 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.495025 12156 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:08.529460  7778 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.977s (SENT) [suppressed 22 similar messages]
I20260709 12:27:08.529829 12156 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.486117 11837 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.399181 12172 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.534088  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.490696 11716 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.500105 11951 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.417300 12217 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.502242 12081 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.497742 12134 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.514875 12201 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:08.537410 11837 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.548094 11716 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:08.499136 12174 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:08.603416  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 157575us
I20260709 12:27:08.607726  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.609297  7656 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:08.614774  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2707) took 1335 ms. Trace:
I20260709 12:27:08.615726  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.617663  7803 rpcz_store.cc:276] 0709 12:27:07.278972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.279116 (+   144us) service_pool.cc:224] Handling call
0709 12:27:07.308244 (+ 29128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.734157 (+425913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.734204 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.768710 (+ 34506us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.947399 (+178689us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:07.988704 (+ 41305us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.988945 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.989027 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.611945 (+622918us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b18000f5e60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:08.614422 (+  2477us) raft_consensus.cc:1685] finished
0709 12:27:08.614527 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.614691 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11642,"prepare.queue_time_us":71121,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":4915,"raft.queue_time_us":84383,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":19059,"spinlock_wait_cycles":158171648,"thread_start_us":538662,"threads_started":2,"wal-append.queue_time_us":228626}
I20260709 12:27:08.619704  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 875) took 1080 ms. Trace:
I20260709 12:27:08.540529 12112 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.625536  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.606863  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.631048  7654 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:08.633338  7654 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:27:08.634411 12270 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:27:08.635838  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2451) took 1042 ms. Trace:
I20260709 12:27:08.636487  7736 rpcz_store.cc:276] 0709 12:27:07.593360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.644444 (+ 51084us) service_pool.cc:224] Handling call
0709 12:27:08.635590 (+991146us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400121e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:08.635747 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257237760}
I20260709 12:27:08.639120  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.640166  7736 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:08.646483 12266 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:08.620419  7729 rpcz_store.cc:276] 0709 12:27:07.539533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.539797 (+   264us) service_pool.cc:224] Handling call
0709 12:27:07.540317 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.542643 (+  2326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.542678 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.543633 (+   955us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.816887 (+273254us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:07.817799 (+   912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.817889 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.619323 (+801434us) raft_consensus.cc:1685] finished
0709 12:27:08.619356 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.619607 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":244,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":272776,"threads_started":1,"wal-append.queue_time_us":702817}
I20260709 12:27:08.651814 11932 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.653542 12266 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:08.654902 12266 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.655741  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1044) took 1300 ms. Trace:
I20260709 12:27:08.656277  8408 rpcz_store.cc:276] 0709 12:27:07.355088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.355323 (+   235us) service_pool.cc:224] Handling call
0709 12:27:08.655455 (+1300132us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:08.655630 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586985344}
W20260709 12:27:08.661154  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.661427  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.662191 11823 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.663264 11824 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:08.663784 11824 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.664191 11824 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:08.668217 12272 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:27:08.668955 12262 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.670697 12227 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:08.671268 12227 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:08.640787 12186 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:08.672813 12186 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:08.673913 12186 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.675495 12186 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.676043  7662 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:08.672322 12227 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.610936 12212 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.691850  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2708) took 1412 ms. Trace:
I20260709 12:27:08.693555  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.694728  7653 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:08.692328  7808 rpcz_store.cc:276] 0709 12:27:07.279461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.293228 (+ 13767us) service_pool.cc:224] Handling call
0709 12:27:07.319897 (+ 26669us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:07.422279 (+102382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.422329 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:07.451858 (+ 29529us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.696344 (+244486us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:07.696390 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.696467 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.691563 (+995096us) raft_consensus.cc:1685] finished
0709 12:27:08.691589 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.691762 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67877,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5277,"spinlock_wait_cycles":1920,"thread_start_us":277610,"threads_started":2,"wal-append.queue_time_us":789711}
I20260709 12:27:08.705605 12075 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:08.708371 12248 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.524s	user 0.002s	sys 0.003s
I20260709 12:27:08.710362  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2264) took 1081 ms. Trace:
I20260709 12:27:08.710693  7655 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:08.712000 12112 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.710883  7811 rpcz_store.cc:276] 0709 12:27:07.629194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.654105 (+ 24911us) service_pool.cc:224] Handling call
0709 12:27:07.696523 (+ 42418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.730244 (+ 33721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.730300 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.741334 (+ 11034us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.889690 (+148356us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:07.996829 (+107139us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.997261 (+   432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.997380 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.707689 (+710309us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:08.709945 (+  2256us) raft_consensus.cc:1685] finished
0709 12:27:08.710067 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.710275 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10854,"prepare.queue_time_us":50586,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":40290,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":60589,"spinlock_wait_cycles":11509504,"thread_start_us":172323,"threads_started":2,"wal-append.queue_time_us":258881}
I20260709 12:27:08.724081  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:08.682107  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.636585 12223 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.860s	user 0.013s	sys 0.000s
I20260709 12:27:08.662195 12174 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:08.684841 12262 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:08.679800  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.753111  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 223678us
I20260709 12:27:08.756676  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:08.752161 12223 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.769163 11824 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.771130 12223 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:08.773005  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:08.715267 12209 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.776132  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.777145  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:08.777482 12209 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.779356  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:08.784739  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.771620 11824 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.793035  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.796016  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.798168  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.800314  7656 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:08.759377 12227 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:08.802405  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 430669us
I20260709 12:27:08.813982  7736 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:08.820365  7656 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:08.759845 12262 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.818501  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.793990 12023 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:08.827445  7733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2431) took 1889 ms (client timeout 1977 ms). Trace:
W20260709 12:27:08.827991  7733 rpcz_store.cc:269] 0709 12:27:06.936543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.980160 (+ 43617us) service_pool.cc:224] Handling call
0709 12:27:08.823749 (+1843589us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540037cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:08.825597 (+  1848us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522050432}
I20260709 12:27:08.829514  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:08.830381 11975 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:08.822476 12195 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.831161  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:08.843473  7733 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:08.847798 12195 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.848492 12195 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.853507 12262 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.843617  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:08.826929 12085 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.003s
I20260709 12:27:08.858055 12085 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.862267  7732 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:27:08.831836 12204 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:08.864076 12204 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.836154 11975 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:08.865949 12085 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.866907 11975 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.868245 12085 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:08.836347 12023 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:08.840924 11904 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 12:27:08.871438 11904 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.814051 12081 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.873829 12081 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:08.873847 11904 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:08.874712  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.869190 12085 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:08.813795 12108 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:08.875571 12085 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:08.875134  7733 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:08.878513  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 853) took 1357 ms. Trace:
I20260709 12:27:08.882045  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 744) took 1097 ms. Trace:
W20260709 12:27:08.883389  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.883829  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2282) took 1216 ms. Trace:
I20260709 12:27:08.884685 12122 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.885293  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.886214  8413 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:08.884341  7806 rpcz_store.cc:276] 0709 12:27:07.667211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.311564 (+644353us) service_pool.cc:224] Handling call
0709 12:27:08.311892 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.312106 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:08.324273 (+ 12167us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.883411 (+559138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:08.883474 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:08.883553 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.883642 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.883766 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400338,"prepare.queue_time_us":11592,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"thread_start_us":178106,"threads_started":1,"wal-append.queue_time_us":271913}
W20260709 12:27:08.887627  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44118us
I20260709 12:27:08.888043  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.882593  8415 rpcz_store.cc:276] 0709 12:27:07.784646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.990481 (+205835us) service_pool.cc:224] Handling call
0709 12:27:07.990815 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.128212 (+137397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.881746 (+753534us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:08.881970 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212989824}
I20260709 12:27:08.893808  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.879041  8406 rpcz_store.cc:276] 0709 12:27:07.517966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.518221 (+   255us) service_pool.cc:224] Handling call
0709 12:27:08.874942 (+1356721us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:08.875154 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680964992}
I20260709 12:27:08.896000  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.897068  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.898015  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2593) took 1292 ms. Trace:
I20260709 12:27:08.898288  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.899096  8413 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:08.897091  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:08.898423  7731 rpcz_store.cc:276] 0709 12:27:07.605523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.682904 (+ 77381us) service_pool.cc:224] Handling call
0709 12:27:08.897832 (+1214928us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:08.897962 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369305472}
I20260709 12:27:08.897643  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:08.904395  7730 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:27:08.904989  7730 pending_rounds.cc:85] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:27:08.905247  7730 pending_rounds.cc:107] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:27:08.905509  7730 raft_consensus.cc:2889] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.906787  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.907711  7806 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.917325  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:08.918479  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.917346  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.919009  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.919595  7805 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:08.919791  7660 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.913290  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:08.907953  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:08.921005  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:08.921893  7733 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:08.878078 12108 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:08.924911  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2717) took 1023 ms. Trace:
I20260709 12:27:08.926393  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:08.928957  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:08.866600 11837 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:08.929669  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:08.931912  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:08.929491  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2281) took 1286 ms. Trace:
W20260709 12:27:08.940120  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 154495us
I20260709 12:27:08.932749  7803 rpcz_store.cc:276] 0709 12:27:07.901801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.620019 (+718218us) service_pool.cc:224] Handling call
0709 12:27:08.630310 (+ 10291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.630561 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:08.630908 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.924405 (+293497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:08.924483 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:08.924552 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.924673 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.924838 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165526,"prepare.queue_time_us":7633,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"thread_start_us":127551,"threads_started":1,"wal-append.queue_time_us":255547}
I20260709 12:27:08.944325  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:08.945232  7731 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:08.951275  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.953012  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.954164  7811 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:08.958287  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.959132  7809 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.923453 12134 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:08.952257  7808 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:08.932164 11823 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:27:08.972033  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.975379  7733 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:27:08.975361  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.976363  7803 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:08.929322 12265 raft_consensus.cc:2764] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:27:08.942409  7807 rpcz_store.cc:276] 0709 12:27:07.643362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.222908 (+579546us) service_pool.cc:224] Handling call
0709 12:27:08.241325 (+ 18417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.246691 (+  5366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.246737 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.247244 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.370635 (+123391us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:08.448368 (+ 77733us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.877457 (+429089us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18002ccb00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:08.928563 (+ 51106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.928653 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.928699 (+    46us) raft_consensus.cc:1685] finished
0709 12:27:08.928843 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.929404 (+   561us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10989,"prepare.queue_time_us":37847,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":429836,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":278,"spinlock_wait_cycles":12423936,"thread_start_us":211284,"threads_started":3,"wal-append.queue_time_us":214218}
I20260709 12:27:08.967595 12227 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:08.984283  7733 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:08.967552 12134 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.984948 12134 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:08.987759  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2488) took 1363 ms. Trace:
I20260709 12:27:08.988319  7655 rpcz_store.cc:276] 0709 12:27:07.624495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.756457 (+131962us) service_pool.cc:224] Handling call
0709 12:27:07.774139 (+ 17682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.711347 (+937208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.987439 (+276092us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b5400219dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:08.987655 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153418496}
I20260709 12:27:08.987607  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.990361 11842 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.933466 11841 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:27:08.992089  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62484.1us
I20260709 12:27:08.992691 11841 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:08.993086  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50190us
I20260709 12:27:08.993552 11841 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:08.994576  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:08.997658  7808 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.001928  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 714) took 1141 ms. Trace:
W20260709 12:27:09.003978  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 75092us
I20260709 12:27:09.006385  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:09.004359  7810 rpcz_store.cc:276] 0709 12:27:07.860883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.499805 (+638922us) service_pool.cc:224] Handling call
0709 12:27:08.500278 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.500601 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:08.501050 (+   449us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.001478 (+500428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.001558 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:09.001612 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.001698 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.001857 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":330241,"prepare.queue_time_us":1978,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":369449,"spinlock_wait_cycles":17490304,"thread_start_us":169507,"threads_started":2,"wal-append.queue_time_us":266235}
I20260709 12:27:09.013746  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:08.980183 11823 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:09.006728 11837 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:09.020479  7810 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:09.028380  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:08.990989 11842 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.031853  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:09.021917  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:09.029855  7810 raft_consensus.cc:2376] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8. Re-sending same reply.
I20260709 12:27:09.038951  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.039778  7810 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:09.039839  8407 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:09.042099 11837 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:09.042891 11837 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.045531  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 713) took 1184 ms. Trace:
I20260709 12:27:09.046778 11837 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:09.046029  7812 rpcz_store.cc:276] 0709 12:27:07.860632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.495533 (+634901us) service_pool.cc:224] Handling call
0709 12:27:08.495882 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:08.879287 (+383405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.879327 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:08.880118 (+   791us) log.cc:844] Serialized 52 byte log entry
0709 12:27:08.927840 (+ 47722us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:08.928661 (+   821us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.990477 (+ 61816us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.990655 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.990734 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.045251 (+ 54517us) raft_consensus.cc:1685] finished
0709 12:27:09.045277 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.045452 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":423593,"prepare.queue_time_us":5806,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":93,"raft.queue_time_us":445473,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":235,"thread_start_us":47008,"threads_started":1,"wal-append.queue_time_us":138466}
I20260709 12:27:09.048794  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:09.050419  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.001674 12074 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:09.051872  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.051978 12074 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.053059  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.054193 12074 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:09.003916 11995 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:09.056663  7810 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:09.067934  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.054152  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.069327  7812 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.070854  7660 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:09.070920  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 126193us
I20260709 12:27:09.069404  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.072216  7731 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:27:09.078080  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83579.2us
W20260709 12:27:09.078332  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46627.8us
I20260709 12:27:09.079428  7805 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:09.062947  7806 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:09.079730  7775 leader_election.cc:400] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:09.080534  7775 leader_election.cc:403] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.030359 11823 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:09.084934  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.086027  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.083593  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.089465  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.086405  7808 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:09.067495 12227 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:09.089905  7810 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:27:09.098234  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76488.1us
I20260709 12:27:09.078119 12112 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:09.094441  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.101006  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:09.087592 11868 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
W20260709 12:27:09.103438 12292 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:09.108475 12237 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:09.109395 12237 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:09.047070 11716 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.111855 12205 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:09.112399 12205 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:09.112775 11716 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:09.113757  7728 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:27:09.113516 12205 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.118467 11840 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:09.118997 11840 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:09.120429 11840 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.121651 11840 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.125319  8405 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:27:09.084717 12261 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:09.093796 11863 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:09.086210 12081 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.096987 12297 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:09.097476  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.140090  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.158281  7660 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:09.143911  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.130779  7808 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.097932  7805 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:09.130452  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:09.147332  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 715) took 1286 ms. Trace:
W20260709 12:27:09.184867  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54523.6us
I20260709 12:27:09.186028  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:09.184947  7804 rpcz_store.cc:276] 0709 12:27:07.861118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.624636 (+763518us) service_pool.cc:224] Handling call
0709 12:27:08.689570 (+ 64934us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:08.918156 (+228586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.918194 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:08.918985 (+   791us) log.cc:844] Serialized 78 byte log entry
0709 12:27:09.019046 (+100061us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:09.060756 (+ 41710us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.061354 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.061469 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.146995 (+ 85526us) raft_consensus.cc:1685] finished
0709 12:27:09.147027 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.147259 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":267254,"prepare.queue_time_us":14467,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":246079,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":21559,"spinlock_wait_cycles":15902976,"thread_start_us":38030,"threads_started":1,"wal-append.queue_time_us":188966}
I20260709 12:27:09.140434  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.139717  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.190019  7660 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:09.193660  7731 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:27:09.201700  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 189648us
W20260709 12:27:09.126735 12249 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.834s	user 0.003s	sys 0.001s
I20260709 12:27:09.225976  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:09.235383  7656 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.233456  7804 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:09.235948  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.245033  7808 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
W20260709 12:27:09.247411  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.250413  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.109128 11842 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:09.255450  7729 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:09.256606  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.253238  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.134997 12112 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:09.196962 12232 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:09.197149 12108 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:09.263528 11868 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:09.110666 12237 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.267501  7804 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.136720 12085 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.134018 12081 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.252628 11716 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:09.252319 11932 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:09.273100  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.276458  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.279378  7654 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.280668  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.281810  7660 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.264693 12112 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:09.267712 12232 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:09.297726  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.244261 11837 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:27:09.288686  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.304804  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.306072  7655 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:09.276197 12022 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.018s	sys 0.004s
I20260709 12:27:09.273954 11970 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:09.310552  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.133564 12227 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.311676  7809 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:09.309535  7660 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:09.314644  7805 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:09.306882 11975 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:09.277724 11716 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:09.277084 12108 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:09.310263 12022 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.018s	sys 0.005s
I20260709 12:27:09.306257 12232 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.304791 12217 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:09.341393 12085 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:09.323129 11716 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:09.336474  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1046) took 1682 ms. Trace:
I20260709 12:27:09.312657 12023 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:27:09.344914  8409 rpcz_store.cc:276] 0709 12:27:07.654204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.654366 (+   162us) service_pool.cc:224] Handling call
0709 12:27:07.654733 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:07.655236 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.655273 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:07.674682 (+ 19409us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.788028 (+113346us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b1800332500. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:07.923064 (+135036us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:07.923123 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.923210 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.126047 (+1202837us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:09.336146 (+210099us) raft_consensus.cc:1685] finished
0709 12:27:09.336169 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.336371 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7851,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8632,"spinlock_wait_cycles":47369344,"thread_start_us":134786,"threads_started":1,"wal-append.queue_time_us":561374}
W20260709 12:27:09.359608  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70027.8us
I20260709 12:27:09.324923 12237 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.348412 12022 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:09.382808  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:09.383853  8414 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.385545  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:09.392048  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:09.357920 12134 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.405602  7655 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:09.413657  7812 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:09.416345 12314 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:09.389703 11868 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:09.393603 12023 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:27:09.426867  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73483.7us
I20260709 12:27:09.423727 12081 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:09.425668 12315 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:09.387882 12108 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:09.445003 12204 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.003s
W20260709 12:27:09.451453  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53604.2us
I20260709 12:27:09.450227  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:09.434528 11868 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.435513 12023 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.459722  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2452) took 1126 ms. Trace:
I20260709 12:27:09.462693  7732 rpcz_store.cc:276] 0709 12:27:08.333125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.346719 (+ 13594us) service_pool.cc:224] Handling call
0709 12:27:08.347162 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.862852 (+515690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.459408 (+596556us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:09.459653 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183847680}
I20260709 12:27:09.427119 11975 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:27:09.449565 12313 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:27:09.459780 12315 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:09.462006 11868 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.467721 12315 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.471553 12023 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:09.473239  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 799) took 1624 ms. Trace:
I20260709 12:27:09.473734  7657 rpcz_store.cc:276] 0709 12:27:07.848443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.848667 (+   224us) service_pool.cc:224] Handling call
0709 12:27:09.472973 (+1624306us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b540028df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:09.473141 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227340544}
I20260709 12:27:09.445672 12022 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.482569  7654 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
W20260709 12:27:09.487769  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.462395  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:09.496974  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50126us
I20260709 12:27:09.497875 11970 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:09.498534 11970 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:09.499549 11970 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.495713  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:09.514683  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.515079  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.516067  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.493607 12265 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:09.526960  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.547078  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.550468  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.462898 12204 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.553440 12317 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:27:09.573625  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:09.573720  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.581172  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:09.582485  7734 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.581840  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:09.494153 11823 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.607512  7734 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:09.900476  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2456) took 1566 ms. Trace:
I20260709 12:27:09.845425  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:09.900871  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.914986  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.653184  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:09.589304  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 509784us
I20260709 12:27:09.574194  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:09.706027  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.583555  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:09.916651  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.660437  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:09.590317  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 489353us
W20260709 12:27:09.765139  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 238293us
I20260709 12:27:09.597209  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.920802  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:09.669770  8405 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.594226  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:09.921450  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.846218  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.803110  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.922950  7653 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.922205  7812 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:09.914896  7657 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:09.801627  7654 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:09.801785  7660 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:27:09.862833  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:09.694995  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2592) took 2106 ms. Trace:
I20260709 12:27:09.926888  7662 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:09.926832  7735 rpcz_store.cc:276] 0709 12:27:07.588902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.836347 (+247445us) service_pool.cc:224] Handling call
0709 12:27:07.865399 (+ 29052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.165149 (+299750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.694629 (+1529480us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:09.694901 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600015104}
W20260709 12:27:09.645009  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.923563  7655 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:09.595803  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:09.582769  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2598) took 1904 ms. Trace:
I20260709 12:27:09.609958  7803 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:09.863209  7659 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.833612  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:09.930964  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:09.934281  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:09.591971  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2606) took 1873 ms (client timeout 1748 ms). Trace:
I20260709 12:27:09.929680  7730 rpcz_store.cc:276] 0709 12:27:07.678078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.908012 (+229934us) service_pool.cc:224] Handling call
0709 12:27:08.899343 (+991331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.929415 (+ 30072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.582434 (+653019us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540037da50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:09.582664 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278910848}
I20260709 12:27:09.933724  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:09.951637  7737 rpcz_store.cc:269] 0709 12:27:07.718758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.756723 (+ 37965us) service_pool.cc:224] Handling call
0709 12:27:09.591716 (+1834993us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b540039b2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:09.591886 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535449344}
I20260709 12:27:09.596607  7809 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:09.953225  7809 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:09.954187  7729 rpcz_store.cc:276] 0709 12:27:08.334098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.648259 (+314161us) service_pool.cc:224] Handling call
0709 12:27:08.650509 (+  2250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:09.256103 (+605594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.766981 (+510878us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400348854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:09.900040 (+133059us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b5400348850. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:09.900350 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1394416896}
I20260709 12:27:09.596560  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:09.746502  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74501.8us
I20260709 12:27:09.651157  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2459) took 1296 ms. Trace:
I20260709 12:27:09.957460  7736 rpcz_store.cc:276] 0709 12:27:08.355292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.670383 (+315091us) service_pool.cc:224] Handling call
0709 12:27:08.670929 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.814659 (+143730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.650834 (+836175us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002477d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:09.651077 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327965440}
I20260709 12:27:09.846174  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:09.588635  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 507242us
I20260709 12:27:09.919148  8411 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:09.917979  7733 raft_consensus.cc:2368] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
W20260709 12:27:09.963141  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129633us
I20260709 12:27:09.942855  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:09.862219  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:09.522925 11975 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:09.966856  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45502.9us
I20260709 12:27:10.028913  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:10.056605  8409 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:09.635092 12306 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
W20260709 12:27:09.633880 12302 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:27:10.026573 12237 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:09.994055  7728 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:27:10.013025  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70265.5us
I20260709 12:27:10.060415  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:10.062775  7804 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [7.9-9.10]
I20260709 12:27:09.590494 12261 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.065156 11841 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:09.617923 12211 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:09.991880 12229 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:10.079362 12229 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:10.079937 12229 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:09.590724 11863 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:27:09.835295 12303 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.454s	user 0.006s	sys 0.000s
I20260709 12:27:09.627997 12269 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:09.652626 12195 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:27:09.504637 12313 consensus_queue.cc:1243] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:27:10.070338  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 135128us
I20260709 12:27:10.057511  7658 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:09.633700 11840 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.003s
I20260709 12:27:10.070847  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:10.071352  8406 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:10.124004  7730 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:10.124383  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53827us
I20260709 12:27:10.025857 12075 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:27:09.864444 12172 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:10.073724  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:10.135334  7730 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:27:10.081916  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 535631us
I20260709 12:27:10.135926  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:10.136606  7736 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:09.628808 12182 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:09.577003 12204 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:10.139739  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:10.140489  7655 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:27:09.833498 12085 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:09.588717 12176 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:09.654456 12205 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:09.609668 12134 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:09.760105 12081 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:10.092007  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.019s
I20260709 12:27:09.626945 12224 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:10.026087 11995 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:09.666390 12246 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.403s
I20260709 12:27:09.594789 12255 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:10.189241 12255 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.196916  8408 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:10.103334  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.589663 12295 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12.
I20260709 12:27:10.199606 12299 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:10.200114 12295 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:10.200333 12299 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.201133 12295 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:10.201598 12261 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.202615 12295 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:09.597842 12297 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.139544 12182 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:10.204182 12297 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.204944 12242 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.205793  7807 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12. Re-sending same reply.
I20260709 12:27:10.107834  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:10.207311  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 99552.5us
I20260709 12:27:10.209311  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.055605 12217 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:09.588778 12073 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.494s
I20260709 12:27:10.114545  7729 raft_consensus.cc:1217] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Deduplicated request from leader. Original: 6.4->[7.5-9.6]   Dedup: 7.5->[9.6-9.6]
I20260709 12:27:09.604110 12232 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:09.648377 12112 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.125972  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:10.026191  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:09.709401 12266 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.373s
I20260709 12:27:10.082623 12174 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.576781 11868 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.807593 12023 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:09.582895 11716 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.131709  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:10.133868  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:10.268149  7736 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:10.023943 11896 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:27:10.026298 11842 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:27:10.138029  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:10.145511  7730 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:10.062856 11837 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:27:10.072695 12315 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 12:27:10.169631  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.004s	sys 0.019s
W20260709 12:27:10.180689 12322 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:10.181231 12314 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:09.643584 12108 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:10.205539 12242 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.029765 12274 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:10.223341  7729 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:27:10.285254  7731 raft_consensus.cc:2368] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:27:10.258188 11997 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:27:10.150918 11840 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:10.151739 12085 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:09.713040 12320 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.388s
I20260709 12:27:10.241761  8413 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:27:10.243075  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 117195us
I20260709 12:27:10.203032 12295 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.168946 12262 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:10.152341 12176 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.111546 12269 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.227908 12075 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.266134 12073 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 12:27:10.152396 11970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.011s	sys 0.058s
I20260709 12:27:10.271718  7656 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:10.317219 11842 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:10.273378  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 135453us
I20260709 12:27:10.268187  7662 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
W20260709 12:27:10.264853 12022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.007s	sys 0.030s
I20260709 12:27:10.299450  7732 raft_consensus.cc:2368] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:27:10.281042 12227 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:10.283788 12314 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:10.165962 12216 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:27:10.260942 12174 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:10.230903  7808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.020s	sys 0.018s
W20260709 12:27:10.377398  7808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.020s	sys 0.018s
I20260709 12:27:10.204193 12182 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.377940  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:10.378336  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.331084 11823 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:10.334247 11970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.011s	sys 0.059s
I20260709 12:27:10.321416  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:10.272558  7655 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:10.337312  7657 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:10.349609  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2503) took 1097 ms. Trace:
I20260709 12:27:10.363775  8408 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 13. Ops: []
I20260709 12:27:10.351359  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:10.298197 12108 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.407020  7656 rpcz_store.cc:276] 0709 12:27:09.252772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.252929 (+   157us) service_pool.cc:224] Handling call
0709 12:27:10.349393 (+1096464us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000d07d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:10.349533 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332357888}
I20260709 12:27:10.317314 12075 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.334050 11997 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:27:10.376583 12213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.004s	sys 0.038s
W20260709 12:27:10.345142 12022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.008s	sys 0.030s
I20260709 12:27:10.332906 12023 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.444442  7805 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:10.333436 12232 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:27:10.475397 12081 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:10.475901 12081 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:10.348300 11837 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
W20260709 12:27:10.358628 12311 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.010s	sys 0.019s
I20260709 12:27:10.317723 11840 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:10.474970 12326 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:27:10.495448 12176 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.497934 12176 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:10.501178  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2723) took 1541 ms. Trace:
I20260709 12:27:10.502991 12112 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.506011  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:10.506596 12255 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:10.507302  7807 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:10.507442 12255 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:10.507946 12255 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.508337 12269 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.502262  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:10.508857 12255 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.509362  7811 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.510327 12255 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:10.501744  7806 rpcz_store.cc:276] 0709 12:27:08.960096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.184886 (+224790us) service_pool.cc:224] Handling call
0709 12:27:09.185377 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.591149 (+405772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.591177 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.591629 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.280217 (+688588us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:27:10.280288 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.280423 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.500876 (+220453us) raft_consensus.cc:1685] finished
0709 12:27:10.500905 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.501091 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":405331,"raft.queue_time_us":406685,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":62,"spinlock_wait_cycles":6016,"thread_start_us":688110,"threads_started":1,"wal-append.queue_time_us":718732}
I20260709 12:27:10.512903  7806 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:10.518185 12242 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:10.518731 12314 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.520370 12314 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:10.520954 12174 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:10.521651 12174 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.522190 12174 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.524421  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 718) took 1521 ms. Trace:
I20260709 12:27:10.524869  7808 rpcz_store.cc:276] 0709 12:27:09.002545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.253958 (+251413us) service_pool.cc:224] Handling call
0709 12:27:09.256482 (+  2524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.591048 (+334566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.591082 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.591629 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.377829 (+786200us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:10.377869 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.377938 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.524139 (+146201us) raft_consensus.cc:1685] finished
0709 12:27:10.524183 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.524355 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":334110,"raft.queue_time_us":335354,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":785792,"threads_started":1,"wal-append.queue_time_us":876998}
I20260709 12:27:10.457337 11997 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:10.393961 12262 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.461701 12213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.004s	sys 0.039s
W20260709 12:27:10.484670  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120198us
I20260709 12:27:10.490514  8409 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:10.493743 11840 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:10.525750  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:10.627557  7811 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
W20260709 12:27:10.485042 12311 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.011s	sys 0.021s
I20260709 12:27:10.536118 12075 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.540885 11716 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.540685 11997 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.446807 12305 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 12:27:10.508373  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31230d31601445449fab8f0f9943db57"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:10.547462 11932 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.548954 12108 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:10.549757 12285 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.550623 12022 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.552505  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:10.552327 12085 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.555334 12227 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:10.564903  7805 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:10.605916  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.606304  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:10.441064 12237 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:10.386977  7653 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:10.646003  7654 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:10.628831 11823 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:27:10.646473 12134 raft_consensus.cc:2764] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:27:10.659828  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54018.1us
I20260709 12:27:10.630577 12299 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.641294 11932 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:10.654218 12227 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:10.662837 11823 consensus_queue.cc:1243] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:10.661307 11868 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:10.663894 12134 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.664422 12134 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:10.664598 12081 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.665341 12134 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.666222 12081 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:10.666646 12134 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.646543 11997 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:10.668045  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61827.5us
I20260709 12:27:10.668898 12182 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.669759 12205 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:10.670420 12205 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:10.663681 11823 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:10.650000 12075 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.643996 12285 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:10.495483 12295 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.627146 12262 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.646782 12022 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:10.643503 11904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.011s	sys 0.024s
W20260709 12:27:10.695211 11904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.011s	sys 0.025s
I20260709 12:27:10.663787 11932 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:10.694597 12274 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:10.714718  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.700341  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5460778250003386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 12:27:10.687652 11995 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
W20260709 12:27:10.716399  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4686778250002135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 12:27:10.689801 12299 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.018s
W20260709 12:27:10.719031  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3012778250003976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:27:10.720880  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0960778250000658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
W20260709 12:27:10.741776  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0519778250004492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
W20260709 12:27:10.712839 12229 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.632s	user 0.008s	sys 0.004s
I20260709 12:27:10.737498 12295 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:10.689765 12224 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 12:27:10.723538 12172 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.001s
I20260709 12:27:10.724843 11932 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.695724 12205 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:10.743458 12294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.019s	sys 0.009s
I20260709 12:27:10.719177 12285 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:10.764740  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.747776 11841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.013s	sys 0.026s
I20260709 12:27:10.746605 12174 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:10.763864 12294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.019s	sys 0.010s
W20260709 12:27:10.766745  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.028s	sys 0.019s
W20260709 12:27:10.777884  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.778371  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.028s	sys 0.019s
W20260709 12:27:10.779713  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.011s	sys 0.016s
W20260709 12:27:10.780251  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.011s	sys 0.016s
W20260709 12:27:10.778867  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 689590us
W20260709 12:27:10.782208  8415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.011s	sys 0.018s
W20260709 12:27:10.788642  8415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.011s	sys 0.019s
I20260709 12:27:10.723380 12227 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:10.789561  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.009s	sys 0.021s
I20260709 12:27:10.789757  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:10.792047  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.009s	sys 0.022s
W20260709 12:27:10.763929  7737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.016s	sys 0.025s
I20260709 12:27:10.747476 12229 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:10.793684  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.797044  8405 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.010s	sys 0.021s
W20260709 12:27:10.794363  7737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.017s	sys 0.025s
W20260709 12:27:10.797472  8405 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.010s	sys 0.021s
W20260709 12:27:10.770529 11975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.020s	sys 0.027s
I20260709 12:27:10.794878  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:10.804052  7732 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:10.795639  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.006s	sys 0.024s
W20260709 12:27:10.801105  7733 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.012s	sys 0.030s
W20260709 12:27:10.808717  7733 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.012s	sys 0.030s
W20260709 12:27:10.808948  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.007s	sys 0.024s
W20260709 12:27:10.802286  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.813694  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 741473us
I20260709 12:27:10.814939  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.818810 12172 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:10.818921  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.822080  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:10.824532  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.827713  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:10.831048  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.848366  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.849064  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.852458 12172 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:10.853335  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:10.854408  7731 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.857059  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.869161  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:10.870462  7729 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:27:10.873456 12108 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.874666 12108 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:10.873111 12172 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:10.877977 12108 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:10.772071 11841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.014s	sys 0.026s
I20260709 12:27:10.888649  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:10.804764 11975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.021s	sys 0.027s
I20260709 12:27:10.795003 12285 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.890448 11841 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:10.894445  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.909209  7803 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.031s	sys 0.026s
I20260709 12:27:10.946158  7731 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
W20260709 12:27:10.921666  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.001s	sys 0.046s
W20260709 12:27:10.896615  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.041s	sys 0.056s
W20260709 12:27:10.906378  7804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.015s	sys 0.040s
W20260709 12:27:10.911582  7626 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.023s	sys 0.033s
W20260709 12:27:10.922505  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.053436  7626 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.024s	sys 0.056s
W20260709 12:27:11.050741  7803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.036s	sys 0.046s
W20260709 12:27:10.912796  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.006s	sys 0.020s
W20260709 12:27:11.054960  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.011s	sys 0.025s
W20260709 12:27:11.055497  7804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.020s	sys 0.046s
W20260709 12:27:11.048532  7775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.012s	sys 0.010s
W20260709 12:27:10.931967  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54175us
W20260709 12:27:11.032233  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.006s	sys 0.015s
I20260709 12:27:11.056052 12229 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:11.056838  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.007s	sys 0.016s
I20260709 12:27:11.057476  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:10.960265  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.565s (SENT)
W20260709 12:27:11.054247  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 732248us
W20260709 12:27:11.056283  7775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.013s	sys 0.010s
I20260709 12:27:10.881206 12174 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.060345  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.996888 12335 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:11.059808  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2302) took 1406 ms. Trace:
I20260709 12:27:11.071456  7804 rpcz_store.cc:276] 0709 12:27:09.653389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.077347 (+423958us) service_pool.cc:224] Handling call
0709 12:27:10.110059 (+ 32712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.110342 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:10.144831 (+ 34489us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.059387 (+914556us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.059440 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.059485 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.059566 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.059736 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3098,"prepare.queue_time_us":34724,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":10076,"thread_start_us":910786,"threads_started":1}
I20260709 12:27:11.071998  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:10.921921 11896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.016s
W20260709 12:27:11.052266  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.041s	sys 0.056s
W20260709 12:27:11.077590  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.078500  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.059661  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 841905us
I20260709 12:27:11.080138  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:11.061894  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 574620us
W20260709 12:27:11.051729  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.006s	sys 0.049s
I20260709 12:27:11.082057  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 806) took 1276 ms. Trace:
W20260709 12:27:11.063546 12261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.024s
W20260709 12:27:11.093852  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.097364  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.097656  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 137429us
I20260709 12:27:10.895202 11975 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:11.098912  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.691s (SENT)
I20260709 12:27:10.963516 12262 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.082660  7659 rpcz_store.cc:276] 0709 12:27:09.805764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.115948 (+310184us) service_pool.cc:224] Handling call
0709 12:27:10.141888 (+ 25940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.142175 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:10.142516 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.081637 (+939121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.081702 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.081765 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.081849 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.081991 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2951,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":938683,"threads_started":1}
I20260709 12:27:11.012090 12344 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.059731 12297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.005s	sys 0.016s
I20260709 12:27:11.099437  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:10.966945 12172 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:10.897027 12227 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:10.911381 12022 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.064392 12224 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:11.064395 11841 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:11.124811  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.019s	sys 0.026s
I20260709 12:27:11.111896  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:11.123603  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66424.9us
W20260709 12:27:11.090473 11896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.005s	sys 0.025s
I20260709 12:27:11.104902  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:11.089423 12305 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:11.124264  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:11.101190  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:11.061625 12073 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:27:11.102124 11823 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:11.126322  8415 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:11.126695  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.019s	sys 0.027s
I20260709 12:27:11.129297  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:11.129688  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.033s	sys 0.044s
W20260709 12:27:11.142992  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.034s	sys 0.044s
W20260709 12:27:11.143438  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 824649us
I20260709 12:27:11.116664 12344 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.144835  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:11.117362 12297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.006s	sys 0.017s
I20260709 12:27:11.165367  7659 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:11.162343  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 812) took 1246 ms. Trace:
I20260709 12:27:11.076023 12229 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:27:11.147735 12022 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:11.148890  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:11.137415  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:11.195961  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58686us
I20260709 12:27:11.186856  7660 rpcz_store.cc:276] 0709 12:27:09.916379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.132822 (+216443us) service_pool.cc:224] Handling call
0709 12:27:10.133218 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.143803 (+ 10585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.143830 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.163241 (+ 19411us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.077975 (+914734us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:11.078059 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.078175 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.161932 (+ 83757us) raft_consensus.cc:1685] finished
0709 12:27:11.161973 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.162232 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16827,"raft.queue_time_us":13103,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":17269,"thread_start_us":811750,"threads_started":1,"wal-append.queue_time_us":882274}
I20260709 12:27:11.099095 11975 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.201557  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 901) took 1236 ms. Trace:
I20260709 12:27:11.121425 12237 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:27:11.090265 12174 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:11.100409 12261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.000s	sys 0.025s
I20260709 12:27:11.151212  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:11.169858  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:11.192248  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:11.193019  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.200985  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52375.3us
W20260709 12:27:11.363288  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62162.8us
I20260709 12:27:11.207988  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:11.209534  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.126289 12338 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:11.115067 12242 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 12:27:11.157444 12176 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.003s
W20260709 12:27:11.161536 12334 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:27:11.152235 12305 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:11.295935  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:11.397588  7654 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:11.296676 12237 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:11.168349 12349 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:11.159972 12172 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.298841 12174 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:11.300024 12211 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.221s	user 0.009s	sys 0.007s
W20260709 12:27:11.301545  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.301900  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.139396 11842 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.166s
W20260709 12:27:11.305006 12075 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.003s
W20260709 12:27:11.308573 12081 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.642s	user 0.004s	sys 0.009s
W20260709 12:27:11.300025 12182 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.006s
I20260709 12:27:11.132977 12224 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:11.305011 11997 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
W20260709 12:27:11.311697 12333 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.397s	user 0.001s	sys 0.001s
I20260709 12:27:11.166925 11841 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:11.308580 12134 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.641s	user 0.014s	sys 0.003s
W20260709 12:27:11.321216  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.422955  7700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.033s	sys 0.126s
W20260709 12:27:11.423331  7700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.033s	sys 0.126s
W20260709 12:27:11.423610  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 502461us
I20260709 12:27:11.089349 12335 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:11.327132 12295 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.580s	user 0.015s	sys 0.005s
I20260709 12:27:11.329439 12344 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.197780 12108 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:27:11.164876 12348 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:11.288861 12262 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:11.288549 12213 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:11.154982 12255 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.001s
I20260709 12:27:11.140053 12216 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:27:11.166028 12314 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.000s
I20260709 12:27:11.364319  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:11.365051  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.432933 11975 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.386837  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 287917us
W20260709 12:27:11.376199  7737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.029s	sys 0.018s
I20260709 12:27:11.394645  7808 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:27:11.392400  8413 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
W20260709 12:27:11.460747  7737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.031s	sys 0.019s
W20260709 12:27:11.391359 11932 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.008s
W20260709 12:27:11.412081  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.403496 12237 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:11.413452  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 300764us
I20260709 12:27:11.416009  8412 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:11.468001  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:11.419965  7654 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:11.421216  7658 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:11.408185 12299 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:27:11.430711  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.414362 11904 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:11.381810 12338 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.439209  7809 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.473321  7809 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:11.447377  8405 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:27:11.452518  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:11.428570 12337 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:11.437353 12335 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.381695  7810 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: []
W20260709 12:27:11.460119  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95913.2us
I20260709 12:27:11.373584 12339 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.461899  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.943s (SENT)
I20260709 12:27:11.438081 12108 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:11.470902  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.486601  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.466797 12262 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:11.468288  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 899) took 1708 ms. Trace:
I20260709 12:27:11.474298  8405 pending_rounds.cc:85] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 7
W20260709 12:27:11.478713  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.483381  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:11.484436  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:11.458390 12213 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:11.469679 11975 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.526458  8405 pending_rounds.cc:107] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:27:11.526907  8405 raft_consensus.cc:2889] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.527288 11975 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.527516 12237 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.528960 12237 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:11.470285 12299 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:11.531594 12085 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:11.531924  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70045.2us
I20260709 12:27:11.532410 12299 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.533856 12299 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:11.474668 12216 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:11.536576 12216 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:11.488569 12360 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:27:11.489192  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:11.489815 11997 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:11.490209 12355 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:11.491515 12224 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.492657  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2303) took 1839 ms. Trace:
I20260709 12:27:11.560207  7803 rpcz_store.cc:276] 0709 12:27:09.653726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.088044 (+434318us) service_pool.cc:224] Handling call
0709 12:27:10.116550 (+ 28506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.056393 (+939843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.057132 (+   739us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.064663 (+  7531us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.413981 (+349318us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:11.414038 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.414136 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.492396 (+ 78260us) raft_consensus.cc:1685] finished
0709 12:27:11.492423 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.492585 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2898,"raft.queue_time_us":34235,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":18735,"thread_start_us":1202384,"threads_started":2,"wal-append.queue_time_us":376508}
I20260709 12:27:11.495172 12338 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:11.495968 12285 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.000s
W20260709 12:27:11.503141  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.468762 12178 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.471560  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85476.4us
I20260709 12:27:11.471319 12343 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:11.457216 12344 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:27:11.513984 12364 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.503047 12339 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:11.488610 12362 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:27:11.503191 12081 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:27:11.521905 11932 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:11.476452 11904 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.521355 12311 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:11.523197 12213 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.539603  7807 raft_consensus.cc:606] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:11.505331  7737 rpcz_store.cc:276] 0709 12:27:09.759606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.952764 (+193158us) service_pool.cc:224] Handling call
0709 12:27:10.054758 (+101994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.054996 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:10.797801 (+742805us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.467906 (+670105us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.467962 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.468016 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.468092 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.468214 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6543,"prepare.queue_time_us":655514,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7637,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4281,"thread_start_us":1367712,"threads_started":2,"wal-append.queue_time_us":613047}
I20260709 12:27:11.481565 12314 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:11.476608 12022 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.503665 12108 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:11.609160 12022 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:11.572940  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.574187  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:11.561414 12182 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.555435 12242 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:11.568315 12232 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:11.574018 11975 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:11.599017 12364 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:11.626312  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.603488 12311 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:11.614156 12205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.033s	sys 0.025s
I20260709 12:27:11.576870 12343 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:11.486375 12337 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.571830 12178 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:11.631145  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.669420  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.647445  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 131325us
I20260709 12:27:11.515017 11841 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.632504 12351 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:11.634578 11975 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:11.645179 12205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.036s	sys 0.025s
I20260709 12:27:11.632386 12182 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.642680 12311 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:11.676424 12195 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:11.637483 12232 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.707108 12357 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:11.710793  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.718664  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.712419  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.702757  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 725) took 1343 ms. Trace:
I20260709 12:27:11.720525  7806 rpcz_store.cc:276] 0709 12:27:10.358849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.514103 (+155254us) service_pool.cc:224] Handling call
0709 12:27:10.514439 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.514816 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.514846 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.515619 (+   773us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.144453 (+628834us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.362895 (+218442us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.701972 (+339077us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b1800315a60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:11.702215 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.702297 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.702353 (+    56us) raft_consensus.cc:1685] finished
0709 12:27:11.702453 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.702654 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4424,"prepare.queue_time_us":218099,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":213,"raft.queue_time_us":569898,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":4582,"spinlock_wait_cycles":17658752,"thread_start_us":627778,"threads_started":1,"wal-append.queue_time_us":874919}
I20260709 12:27:11.679948 12242 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:11.715060  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 756) took 2663 ms (client timeout 1853 ms). Trace:
I20260709 12:27:11.634858 12262 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.722498  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:11.705586 12364 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.674518 12134 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:27:11.693186 12224 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.736253  8412 rpcz_store.cc:269] 0709 12:27:09.048203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.048434 (+   231us) service_pool.cc:224] Handling call
0709 12:27:11.711721 (+2663287us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:11.711853 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5440812288}
I20260709 12:27:11.739924  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.714298 12195 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.763558  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.764336  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:11.712791 12174 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.802347  7803 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:11.681150  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:11.766268  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2297) took 2147 ms. Trace:
I20260709 12:27:11.809190  7809 rpcz_store.cc:276] 0709 12:27:09.594396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.596006 (+  1610us) service_pool.cc:224] Handling call
0709 12:27:09.596449 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.093851 (+497402us) spinlock_profiling.cc:243] Waited 71 ms on lock 0x7b4c00480280. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc0001eea20 00007fc0001ffa29 00007fc0001c5644 00007fc0001ecf2d 00007fc0001ecb93 00007fc001f1d18b 00007fc001f1dcb5 00007fc001f18f30 00007fc001f1a7de 00007fc001f24b49 00007fc001f25416
0709 12:27:11.438711 (+1344860us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400227ca0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001ea006f 00007fc001f18f60 00007fc001f1a7de 00007fc001f24b49 00007fc001f25416 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62
0709 12:27:11.732461 (+293750us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:11.732494 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.741625 (+  9131us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.741707 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.741837 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.742033 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":200067,"raft.queue_time_us":221020,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":15314,"spinlock_wait_cycles":2379412608}
W20260709 12:27:11.811543  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 237429us
W20260709 12:27:11.775569 12371 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 12:27:11.776845 12373 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:27:11.778627 11863 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 12:27:11.781461  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 292662us
I20260709 12:27:11.784060 12343 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:11.791306  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.774822 12339 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.774808  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 727) took 1410 ms. Trace:
I20260709 12:27:11.894207  7808 rpcz_store.cc:276] 0709 12:27:10.364672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.526429 (+161757us) service_pool.cc:224] Handling call
0709 12:27:10.537411 (+ 10982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.460817 (+923406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.774222 (+313405us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:11.774617 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963373952}
I20260709 12:27:11.794605 12232 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:11.795434 12134 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.795614 11975 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.897094  7803 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:11.813457  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:11.814225  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 201224us
I20260709 12:27:11.906338  7805 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:27:11.814786  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 240333us
I20260709 12:27:11.801046 12195 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:11.818923 12211 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.762450 12022 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.820714 12174 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.758396 12242 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.831657  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:11.833242 12337 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.972517  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:11.690809 12351 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.847625  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:11.885040  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 203966us
I20260709 12:27:11.983968  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:11.984872 12174 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:11.985031  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:11.985177  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.985991  7804 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:11.886075  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.987582  7806 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:11.989022  7806 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:11.989257 12242 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.990314 11997 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:11.912340  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:11.859334 12314 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:11.909209 12350 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.900957 11932 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:11.991771 12350 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.991879 12314 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.920495 12081 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.902494 12348 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.994642 12350 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.880637 12365 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.926584  7732 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:11.927258  7737 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:27:11.998906 12365 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:11.927744  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:11.896006 12364 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.003767 12022 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:12.003651 12364 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.919673 12378 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.005812 12378 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.895169 12346 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.007799 12346 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:11.943940  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.009021 12378 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.845788 12224 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.928315 12211 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.011982 12224 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.798053 12182 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.909935 12134 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:11.996116  7811 raft_consensus.cc:2376] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11. Re-sending same reply.
I20260709 12:27:12.013211 12211 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:11.987048  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:11.928690 11975 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:12.014230  7804 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:27:11.989025 12351 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:11.995029 12348 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.014840 12134 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.994558 12081 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.016957 12348 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:11.910151 12285 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:27:12.019620 12081 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.026988 11997 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:12.028662 11997 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.031507 12022 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:12.033125  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.032874 12022 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.034507 12364 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:11.996788 12213 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:12.038237 12213 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:12.038817 12213 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.047362  7806 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:12.022928 12311 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.052640 12311 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.055009 12174 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.076879  7804 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:27:12.077345  7804 pending_rounds.cc:85] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 6
I20260709 12:27:12.077203  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.077626  7804 pending_rounds.cc:107] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:12.055410 12357 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.082680 12357 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.083295  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.077905  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:27:12.086874  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 917109us
W20260709 12:27:11.923696  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110312us
I20260709 12:27:12.032663 12081 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:12.060737  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:12.046939 12237 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.517s	user 0.014s	sys 0.001s
W20260709 12:27:12.073240  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 225714us
W20260709 12:27:12.102952  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.059049  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.058571 11842 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:27:12.105300  7653 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:12.088496  7654 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:12.089432  7658 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:12.089689 12369 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:12.089789  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.090045  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.107321  7659 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.090950  7811 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:12.091049  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.108717  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.109620  7655 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 5.
I20260709 12:27:12.088510  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.110107  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:12.093179  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.105350  7656 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:12.093163  7806 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:12.042536 12108 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:12.080529 12299 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.002s
I20260709 12:27:12.002980 12365 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.099260  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:12.118327  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.077890  7804 raft_consensus.cc:2889] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:12.121078  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.121490  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.122061  7804 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:12.123986  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.103118  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.126204  7660 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.104045  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.116503  7732 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:12.085671 11823 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:12.123680  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:12.151144  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:12.210608  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.150547  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.213675  7811 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:12.168359  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.215314  7657 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:12.201929  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.224309  7804 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:27:12.217530  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:12.194262  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.089608 12174 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:12.218793  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95213.3us
I20260709 12:27:12.093855 12081 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:12.225291  8412 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:27:12.079321 12348 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 12:27:12.220331  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.211104 12338 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:12.226197  7806 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:12.229794  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:12.269742  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 170610us
I20260709 12:27:12.279479  7811 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.271251  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:12.280611  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.281492  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.281970  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.118450 12350 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:12.118163 12369 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:12.271270  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120125us
I20260709 12:27:12.211367 12227 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:12.225443  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.276903  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:12.104921 12242 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.182260 12255 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:12.221825 11841 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.115718 12339 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.161s
I20260709 12:27:12.106074 11995 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 12:27:12.148612 12229 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:12.342021  7660 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:12.284907 12338 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.181763 12297 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:27:12.318797  8408 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:27:12.310452 12320 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:12.328657  8407 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:27:12.329892  7702 connection.cc:256] RPC call timeout handler was delayed by -1.06552861199998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 12:27:12.363049  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.363816  8407 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:12.329763  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.364593  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:12.365475  7730 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:12.366014  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:12.333801  7812 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:12.366770  7806 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.367776  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:12.372210  7737 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:27:12.374116  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.375159  7735 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:12.279438 11932 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:12.376878 11932 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.288774 12211 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:12.380074 12211 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.380681 12211 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:12.344748  7811 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:12.312851 12073 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.384475  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.384222 12073 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.385543  7811 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:12.315140  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.323585 12348 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.363848  7731 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:27:12.387698 12348 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.276129 12349 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.393760  7730 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:27:12.394394  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.394650  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:12.395747  7734 raft_consensus.cc:2368] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
W20260709 12:27:12.388008  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72988.7us
I20260709 12:27:12.395404  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.400108  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.401036  7728 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:12.401432  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.387048  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.402733  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.404124  7736 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 7.
I20260709 12:27:12.406602  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.408480  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.408934  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.410522 12255 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:12.415971  7660 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:27:12.426255  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.417940  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.418860  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.431582  7811 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:12.431849  7729 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 6.
I20260709 12:27:12.420033  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.433559  8407 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:12.420290  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1056) took 1311 ms. Trace:
I20260709 12:27:12.356070 12338 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:12.438110  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.436019  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:12.439747  7731 raft_consensus.cc:2368] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:27:12.440201  8408 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 6.
I20260709 12:27:12.434809  8413 rpcz_store.cc:276] 0709 12:27:11.108800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.109044 (+   244us) service_pool.cc:224] Handling call
0709 12:27:12.420011 (+1310967us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540013fbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:12.420207 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359332096}
I20260709 12:27:12.452139  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.453204  7661 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:27:12.453332  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.454397  7660 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:12.455843  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:12.455961  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:12.339529 11841 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.399119 12351 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.407178 12350 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.463480  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.479038  7661 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:12.337347 12242 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.480515  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.465131  8407 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:27:12.485316  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.486382  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.487198  7729 raft_consensus.cc:2368] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
W20260709 12:27:12.493290 12390 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:27:12.496806  7661 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:12.496922  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:12.498234  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.442214 12073 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.458570 12176 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.398795 12320 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.349097 12081 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.419811 12195 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.521291  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:12.524744 12389 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:27:12.525928 12346 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.542188  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:12.544314  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.544848  7734 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 12.
I20260709 12:27:12.544770  8411 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:12.520095 12073 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.554932  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:12.557298  7661 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:12.548713  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:12.406721 12369 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.495939 12351 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.562896  8414 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.10-8.10]
W20260709 12:27:12.579726 12380 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:27:12.580118  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.580430 12351 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:12.581455 12178 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:12.582404 12320 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:12.582664 12397 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:12.552511 12346 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.583647 12346 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:12.585222 12391 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:27:12.586333 12346 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.586572 11840 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:12.543464 12081 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:12.568979 12337 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.572933 12195 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:12.574196 12392 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:27:12.575795 12369 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.576571 12357 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.576992 11841 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.552772 12314 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.578223 12242 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:12.587421 12393 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:27:12.496127 12350 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.624869  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.628465 12348 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.567868  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:12.599150  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 143386us
I20260709 12:27:12.599395  7804 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:12.605409  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.592406 11840 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:12.608726  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.529778 12176 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.632506  7661 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:12.637307  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.621162  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:12.640164  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.641034  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:12.618912  7659 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.004s
I20260709 12:27:12.642020  7659 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:12.591681  7806 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:12.641054  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:12.619879 11995 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.620074 12384 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.638415  7661 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:12.642288  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.641467  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.628748 12174 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.672606  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.603969 12383 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:27:12.642510  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.699029  7731 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:12.628764 12242 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.656383  7732 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:12.657107 12229 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:12.658116 11840 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.652581 12299 raft_consensus.cc:2764] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:27:12.669761 12211 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:12.670068  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 64809.4us
I20260709 12:27:12.734113  8415 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:27:12.734714  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.669800 12176 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:12.649739  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.680866  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.738610  7654 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:12.643183 11995 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.685159  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.688565  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:12.742491  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:12.695518  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.696309  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.697576  7661 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:12.626116 12314 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.733323  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:12.747690  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59242.6us
I20260709 12:27:12.748602  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:12.740684  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84185.9us
I20260709 12:27:12.750016  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.736435  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:12.752537  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.655453 12348 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:12.740125  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1053) took 2339 ms. Trace:
I20260709 12:27:12.682685  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.755925  7656 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:12.694936  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.737797  7659 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:12.746656  7654 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:12.754158  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:12.741271 11975 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.001s
I20260709 12:27:12.768412  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.772377  7660 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:12.776587  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:12.777998  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:12.778772  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.785208  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:12.786655  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:12.673403 12384 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.795828  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.798105  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.798599  7731 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:12.763967 12357 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:12.720770 12242 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.766508 12351 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.802268  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.755187  8406 rpcz_store.cc:276] 0709 12:27:10.394275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.394917 (+   642us) service_pool.cc:224] Handling call
0709 12:27:10.395313 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.445059 (+ 49746us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:10.466080 (+ 21021us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.844999 (+1378919us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b1800332440. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:27:12.333625 (+488626us) log.cc:844] Serialized 14 byte log entry
0709 12:27:12.733490 (+399865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:12.733551 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:12.733604 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.733671 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.733805 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46652,"prepare.queue_time_us":633487,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":125,"raft.queue_time_us":79122,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":16156,"spinlock_wait_cycles":18027264,"thread_start_us":1447013,"threads_started":3,"wal-append.queue_time_us":844838,"wal-append.run_cpu_time_us":1521,"wal-append.run_wall_time_us":1001614}
I20260709 12:27:12.763912 12112 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.837828  8411 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:12.805482  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.770251 11840 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:12.767400 12176 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:12.847420 12213 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.000s
I20260709 12:27:12.801347  7730 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:27:12.827311  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.733099 12229 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.792218 12311 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:12.849733 12311 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.850207 12311 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.850427 12229 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.827378  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.770954 12297 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:12.851449  7661 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:12.769539 12348 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:12.852216 12348 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:12.852723  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.853442  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 5.
I20260709 12:27:12.776376 11841 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:12.770571 12402 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:12.855211 12402 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.855130 11841 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:12.855784 12402 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.856751 11841 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.824203 12357 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.860338 12357 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:12.791347 12346 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:12.848102 12213 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:12.854368  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.866267  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.867815  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.868487  7804 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:27:12.869405  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:12.870009  7804 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:12.850449 11995 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:12.791361 12285 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:27:12.872085 12285 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:12.872565 12285 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:12.804016 12073 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:12.803961 12399 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:12.875486 12073 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.875877 12399 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.876065 12073 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.876430 12399 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.844918 12211 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:12.881350 12384 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.882863 12350 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.791757 12320 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:27:12.885890  7804 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:12.889014  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:12.895430  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.906260  7661 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:12.906548  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:11.202077  7733 rpcz_store.cc:276] 0709 12:27:09.965314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.965527 (+   213us) service_pool.cc:224] Handling call
0709 12:27:10.062549 (+ 97022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.062795 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:10.809250 (+746455us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.201069 (+391819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.201157 (+    88us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:11.201229 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.201337 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.201489 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31381,"prepare.queue_time_us":648041,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":1124,"thread_start_us":1106024,"threads_started":2}
I20260709 12:27:12.899412  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:12.803812 12388 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:12.803872 12376 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:27:12.903720  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:12.917624  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:12.908839  7658 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:12.919615  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.930316  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.932148  7728 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:12.951905  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.944837  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:12.953650  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:12.954473 12344 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:12.954902 12344 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:12.955240 12178 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:12.955309 12344 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.955914 12178 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.956352 12178 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.935180 12383 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:27:12.935415 12211 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:12.959645 12335 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:12.959718 12383 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.961485 12383 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:12.960158 12211 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:12.963582 12211 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:12.964584 11904 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:12.965377 12344 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:12.927724 12227 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.005s
I20260709 12:27:12.909554 12242 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:12.912281 12388 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.916873 12376 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.984316  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:12.988583  7653 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.977710  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:12.840497 12351 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.976133  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:12.885779 12112 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:12.997716  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.002251 12402 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.006108  7661 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:12.978381  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 176011us
I20260709 12:27:12.987887 12388 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.988461 12227 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:12.992659 12376 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:13.012728  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:12.974553 12073 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.039321  7808 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:27:13.040678  7653 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:13.013317 12344 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.042699  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:13.017939 12174 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:13.044823  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.045984  7653 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.051504  7808 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:27:13.050189  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.047694 12337 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:13.063941 12406 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:27:13.064612 12409 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:27:13.064985  7661 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:13.037320 12285 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.070976  7728 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:13.072021 12314 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.075314 12213 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.089517  7653 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
W20260709 12:27:13.031598 12255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.021s	sys 0.016s
I20260709 12:27:13.094156  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.064045 12337 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:13.096109 12255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.022s	sys 0.017s
W20260709 12:27:13.095728 12349 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.707s	user 0.015s	sys 0.000s
I20260709 12:27:13.113637  7654 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:13.113914 12349 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:13.115444  7804 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:13.116674 12081 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 12:27:13.117640 12351 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.117816  7658 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:13.121282  7656 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:27:13.114190  7654 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:13.134694  7654 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.136056  7654 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:13.140334 12178 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.142817 12178 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:13.144304 12338 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.006s
I20260709 12:27:13.145172 12338 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.146375  7662 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:13.147189  7811 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:13.149507 12402 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.155519 12178 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:13.156455 12178 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.159634 12337 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.160939 12073 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:13.163182 12369 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.003s
I20260709 12:27:13.168244 12112 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.159555 12178 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.158128 12399 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.181423  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2721) took 4222 ms. Trace:
I20260709 12:27:13.181918  7810 rpcz_store.cc:276] 0709 12:27:08.959523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.099834 (+140311us) service_pool.cc:224] Handling call
0709 12:27:09.100152 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.481017 (+2380865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.181100 (+1700083us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b54003f4b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.181344 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5246415104}
I20260709 12:27:13.185336  7661 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:13.183434  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:13.190460  7810 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:13.103833 12255 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:13.197237  7810 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:13.198271  7653 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: [4.3-8.5]
I20260709 12:27:13.130488 12285 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.166859 12311 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.219575 12399 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.220767 12255 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.213584  7804 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:13.206660 12348 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.272120 12338 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.277657  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1057) took 1843 ms. Trace:
I20260709 12:27:13.279345 12311 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.278187  8409 rpcz_store.cc:276] 0709 12:27:11.434360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.434778 (+   418us) service_pool.cc:224] Handling call
0709 12:27:11.435277 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.770189 (+334912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.770236 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:11.770314 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:11.770363 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.277397 (+1507034us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54006310d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.277453 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.277584 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14068,"raft.queue_time_us":162385,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":411363072,"thread_start_us":141333,"threads_started":1}
I20260709 12:27:13.282397  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2310) took 1212 ms. Trace:
I20260709 12:27:13.283452  7811 rpcz_store.cc:276] 0709 12:27:12.069702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.432748 (+363046us) service_pool.cc:224] Handling call
0709 12:27:12.433261 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.147877 (+714616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.281957 (+134080us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.282290 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641198464}
I20260709 12:27:13.302780  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50788 (request call id 23) took 2919 ms. Trace:
I20260709 12:27:13.303639  7807 rpcz_store.cc:276] 0709 12:27:10.440910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.508124 (+ 67214us) service_pool.cc:224] Handling call
0709 12:27:13.297914 (+2789790us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54003c46d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:13.298071 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068450560}
I20260709 12:27:13.252900 12211 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.325860  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 764) took 1044 ms. Trace:
I20260709 12:27:13.326380  8414 rpcz_store.cc:276] 0709 12:27:12.280937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.281080 (+   143us) service_pool.cc:224] Handling call
0709 12:27:12.281478 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.578981 (+297503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.325448 (+746467us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.325772 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623706880}
I20260709 12:27:13.327306 12211 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.347086 12348 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.356678 12108 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.324220 12338 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:13.368170  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2755) took 1183 ms. Trace:
I20260709 12:27:13.368873  7812 rpcz_store.cc:276] 0709 12:27:12.184698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.184878 (+   180us) service_pool.cc:224] Handling call
0709 12:27:13.367898 (+1183020us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b5400332054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:13.368074 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104893312}
W20260709 12:27:13.370702 12428 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:13.375804  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:13.377624 12211 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:13.378196 12211 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:13.379142 12211 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.323823 12305 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.383963 12305 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:13.389513 12425 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:27:13.259006 12357 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.391686  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.393523  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.396476 12357 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.397373 12357 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.397955 12357 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.400151 12399 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:13.400810 12399 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.401154 12399 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:13.401692 12337 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:13.402423 12337 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.404450  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2530) took 1136 ms. Trace:
I20260709 12:27:13.404281 12383 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.405570  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.406088  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 875) took 1491 ms. Trace:
I20260709 12:27:13.406414  8409 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:13.406458  8412 rpcz_store.cc:276] 0709 12:27:11.914236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.914434 (+   198us) service_pool.cc:224] Handling call
0709 12:27:11.996125 (+ 81691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.275147 (+279022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.405833 (+1130686us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400780354. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.406025 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526461312}
I20260709 12:27:13.404978  7657 rpcz_store.cc:276] 0709 12:27:12.268640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.272078 (+  3438us) service_pool.cc:224] Handling call
0709 12:27:12.272470 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.531766 (+259296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.531801 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:12.558260 (+ 26459us) log.cc:844] Serialized 26 byte log entry
0709 12:27:12.726743 (+168483us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:12.726796 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.404004 (+677208us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:13.404057 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.404110 (+    53us) raft_consensus.cc:1685] finished
0709 12:27:13.404126 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.404382 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14737,"raft.queue_time_us":14874,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":135,"spinlock_wait_cycles":561092992,"thread_start_us":157597,"threads_started":1,"wal-append.queue_time_us":234701}
I20260709 12:27:13.368220 12338 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.405011 12337 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.410523 12338 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:13.412511  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.413329 12205 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:13.414076 12205 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.417014 12205 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.377660  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:13.420115 12178 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:13.420944 12178 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:13.427975  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.430014 12375 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.430835 12375 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.433895 12375 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:13.435002  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 459034us
W20260709 12:27:13.438133  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.439661  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 426017us
I20260709 12:27:13.440196 12320 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.440809 12320 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.441186 12412 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:13.442708  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:13.443538 12081 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:13.442188 12412 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.445261  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.445385 12351 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:13.446491 12351 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:13.446682 12412 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.448618 12311 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:13.447471 12285 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:27:13.447181 12073 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:13.450676 12285 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:13.451074 12285 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:13.451297  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2631) took 1058 ms. Trace:
W20260709 12:27:13.451560  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.452189 12285 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.453397 12404 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.448640  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:13.448966 12085 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:13.455317 12085 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:13.455600 12073 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:13.444417 12081 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:13.456192 12073 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.456440 12085 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.457152 12073 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.457914  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:13.459641 12174 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:13.460119 12174 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.460511 12174 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:13.449697  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.471416  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.466910  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:13.474215 12415 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.123s	user 0.002s	sys 0.000s
I20260709 12:27:13.472231  7655 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:13.451844  7737 rpcz_store.cc:276] 0709 12:27:12.392539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.400494 (+  7955us) service_pool.cc:224] Handling call
0709 12:27:12.556389 (+155895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.569880 (+ 13491us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:12.570242 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:27:12.789615 (+219373us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.001227 (+211612us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.002905 (+  1678us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.004154 (+  1249us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.004315 (+   161us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:13.004508 (+   193us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.353424 (+348916us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b18003e2480. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:13.426582 (+ 73158us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b18003e2480. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fc005df658f 00007fbffee35571 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd
0709 12:27:13.439652 (+ 13070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.439772 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.439833 (+    61us) raft_consensus.cc:1685] finished
0709 12:27:13.439848 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.451059 (+ 11211us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001c43d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:13.451201 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1032222,"prepare.queue_time_us":704988,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":175,"raft.queue_time_us":505396,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":41395,"spinlock_wait_cycles":404892160,"thread_start_us":417149,"threads_started":2,"wal-append.queue_time_us":342316}
I20260709 12:27:13.382253 12376 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.480103  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:13.481091 12351 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:13.481721 12351 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:13.482257 12376 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.454046 12404 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.482805 12351 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.457134 12081 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.485157 12404 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.469974 12320 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.386994 12305 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.422221 12178 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.395058  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:13.512658  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 117693us
I20260709 12:27:13.513705  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:13.469316  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.488073  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.516923  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:13.517664  7811 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:13.490598  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.519804  8411 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 7.
I20260709 12:27:13.482383  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.491823  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.492131  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.523659  7657 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.492807  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.493685  7659 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:13.493800  7655 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:27:13.390970  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.501070  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.504393  7656 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:13.508291  7804 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:13.510027  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.528311  7627 leader_election.cc:400] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:13.528913  7805 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:13.529354  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.528782  7627 leader_election.cc:403] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:13.528645  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.528735  7662 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:27:13.531150  7806 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:13.511376  7660 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:13.510808  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.532401  7812 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:27:13.532444  7732 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:13.533560  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.534487  7737 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:13.545955  8414 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:13.546208  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.545994  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.547348  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.547603  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:13.547871  7809 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:13.548892  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.549917  8412 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:13.552083  7657 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:13.552716  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:13.553164  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:13.549933  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.554399  8414 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:13.554307  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.556015  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:13.557917  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.563637  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:13.565023  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:13.566363  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.567417  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:13.487910 12338 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:13.569165  7737 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:27:13.572690  7737 pending_rounds.cc:85] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:27:13.573078  7737 pending_rounds.cc:107] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:27:13.573424  7737 raft_consensus.cc:2889] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:13.570181  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.574793  7653 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:13.574952  7811 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:13.563921  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.571312  7808 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:13.571432  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:13.511420 12299 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.580997  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.581470  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.582867  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:13.519161 12383 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:13.584060  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:13.518628 12213 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:13.588502  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.588860  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.589391  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.529709 12350 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:13.592317  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.592484  7808 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:27:13.594017  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.596900  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.596791  7732 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:27:13.598868  7662 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:13.595760 12285 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:13.603106 12395 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:13.606094 12337 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:13.606328  7653 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:13.511868 12320 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:27:13.607815 12386 raft_consensus.cc:2764] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:27:13.608477 12320 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:13.592669 12299 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.610081 12357 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.596822 12348 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:27:13.570605 11975 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.570422 12376 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:13.574666 12335 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:13.621865 12357 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.554268 12305 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:27:13.643402  7804 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:27:13.645313  7729 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:13.573523 12311 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:27:13.569545 12314 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:13.621151  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:13.623136  7730 raft_consensus.cc:2368] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11.
I20260709 12:27:13.511112 12242 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:27:13.600296 12350 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.676393  7657 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:13.602375  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.626044  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.629643  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.584734 12232 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.618904  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.590245 12343 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:13.633560  7812 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:13.598132 11841 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.619441  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:13.608952 12320 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:13.601864 12402 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.655819  7662 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:27:13.599886 12213 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.602391  7808 pending_rounds.cc:85] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 6
I20260709 12:27:13.608858 12386 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:13.606840 12337 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.662577  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.666457  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:13.631698 12335 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.678620  7804 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [8.6-10.7]
W20260709 12:27:13.679878  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50344.7us
I20260709 12:27:13.682206  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.684340  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.689260  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.600513 12338 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:27:13.655938 12305 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:27:13.694571 12343 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:13.813784 12343 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:13.813385 12402 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.699681  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.819082 11841 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.820060 12320 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.824151 12299 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:13.825133  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.913033  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2630) took 1520 ms. Trace:
I20260709 12:27:13.849285 12335 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.829476  7809 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:13.831429  7657 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:13.832775  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:13.833535  7808 pending_rounds.cc:107] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:27:13.926375  7808 raft_consensus.cc:2889] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:13.926810  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 94127.5us
I20260709 12:27:13.641645 11975 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.681187 12350 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.850872 12402 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:13.879707  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:13.993153  7653 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:13.847796 12305 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:27:13.693845 12232 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.647374 12357 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:13.890829 12343 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:13.900068  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.901623  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.063686  7732 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:14.064177  7729 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:13.902796  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.904728  7728 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:13.903707  8414 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:13.804450  7662 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
W20260709 12:27:13.828262  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 280735us
I20260709 12:27:14.074198  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:13.923396  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:13.917464  7735 rpcz_store.cc:276] 0709 12:27:12.392185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.399529 (+  7344us) service_pool.cc:224] Handling call
0709 12:27:12.399873 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.445073 (+ 45200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.445110 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:12.445942 (+   832us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.299395 (+853453us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b18002e0e40. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:13.494413 (+195018us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:13.567324 (+ 72911us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.572340 (+  5016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.572421 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.827824 (+255403us) raft_consensus.cc:1685] finished
0709 12:27:13.912655 (+ 84831us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.912928 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112360,"prepare.queue_time_us":77965,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":62387,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":23049,"spinlock_wait_cycles":37265024,"thread_start_us":194791,"threads_started":1,"wal-append.queue_time_us":1193485}
I20260709 12:27:13.848824 12337 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:13.930680  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:14.079361  7660 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:13.928745  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:14.016753  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 488585us
W20260709 12:27:13.915138 12211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.013s
I20260709 12:27:13.830114  7810 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:14.070173  7734 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:27:14.087275  7660 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:14.075495  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:14.089684  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:13.697057 11932 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.376s
I20260709 12:27:14.096508  7627 leader_election.cc:400] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:27:14.111896  7627 leader_election.cc:403] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:14.112721  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:14.007150 12305 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:14.059456 12388 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.040s	user 0.017s	sys 0.000s
W20260709 12:27:14.091524 12211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.012s	sys 0.014s
W20260709 12:27:14.067946 12374 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 12:27:14.063567 12085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.010s	sys 0.021s
I20260709 12:27:14.136056 12242 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11.
I20260709 12:27:14.137128 12242 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:14.137606 12242 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:14.138031 12242 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.110164 12320 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:14.169618 12205 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:14.170100 12205 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:14.133302  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:14.171411  7660 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:14.093380 12338 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.110404 12386 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:14.142403  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:14.112020 12402 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:14.156322  7811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.017s
I20260709 12:27:14.115449 12112 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:14.093901 12299 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:14.180830  7811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.010s	sys 0.017s
W20260709 12:27:14.181408  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50683.5us
I20260709 12:27:14.183683 12112 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.184345 12112 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:14.182636  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:14.159412  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.020s	sys 0.023s
W20260709 12:27:14.189006  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.021s	sys 0.024s
W20260709 12:27:14.151153 12085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.010s	sys 0.022s
W20260709 12:27:14.095691  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 167036us
W20260709 12:27:14.156719  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60431.1us
I20260709 12:27:14.198309  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:14.180363  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.025s
W20260709 12:27:14.199278  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.011s	sys 0.026s
I20260709 12:27:14.199522  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:14.200313  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:14.200582  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:14.201303  7809 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:14.201489  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:14.177709 12314 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:14.204713  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.202211  7808 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:14.192315 11840 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:14.211526  7737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.016s	sys 0.024s
W20260709 12:27:14.216310  7737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.017s	sys 0.024s
I20260709 12:27:14.157913 12388 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.187420 12299 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:14.219893  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.222007  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:14.227370  8414 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:27:14.214843 12073 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.755s	user 0.013s	sys 0.001s
I20260709 12:27:14.238682  7656 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:14.169008 12305 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.231477  7660 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:27:14.242687  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:14.243659  7806 raft_consensus.cc:606] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.244120  7806 raft_consensus.cc:614] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:14.240477  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:14.231072  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:14.240113  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.014s	sys 0.037s
I20260709 12:27:14.238876  7659 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:27:14.247411  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.015s	sys 0.037s
I20260709 12:27:14.252710  7809 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:14.254981  7810 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:14.224553 12299 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:14.258183  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.017s	sys 0.009s
W20260709 12:27:14.236685  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.032s	sys 0.024s
W20260709 12:27:14.259749  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.032s	sys 0.024s
W20260709 12:27:14.259598  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.017s	sys 0.009s
I20260709 12:27:14.264946  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:14.256521  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.027s	sys 0.032s
W20260709 12:27:14.266395  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.028s	sys 0.033s
W20260709 12:27:14.265101  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 614166us
W20260709 12:27:14.266801  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 714117us
I20260709 12:27:14.231853 12402 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.235988 12213 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.273911  7737 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:14.294416  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:14.237185 12081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.011s	sys 0.027s
I20260709 12:27:14.295845  7734 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:27:14.236469 12073 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.258898 12351 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:27:14.239357 12412 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:27:14.302383  8378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.032s	sys 0.043s
W20260709 12:27:14.352219  8378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.033s	sys 0.043s
I20260709 12:27:14.295154  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:14.273403 12376 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:14.295665 12081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.011s	sys 0.028s
I20260709 12:27:14.302479  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.341005  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:14.323547  8380 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.303706  8411 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:14.259109 12444 raft_consensus.cc:2764] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:27:14.363634  8414 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:14.258481 12441 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:14.309931 12285 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.001s
I20260709 12:27:14.364300  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.273670 12375 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:14.375782  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:14.376145  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:14.377123  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.401240  8411 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:14.380664  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.009s	sys 0.032s
W20260709 12:27:14.401996  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.010s	sys 0.032s
I20260709 12:27:14.309731 11997 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:14.337620 12108 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:27:14.309206 11841 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.309000 12305 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:27:14.406946  7803 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:14.309005 11975 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.300246 12402 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.323098 12232 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:14.339442 12343 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:14.439013  8412 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:14.370747 12022 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:14.326216 12369 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 12:27:14.409997  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:14.410390  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.405347 11997 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:14.357612 12357 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.395081 12441 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.444293 12444 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:14.358969 12376 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.445509  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:14.483744 11997 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:14.471585  8411 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:14.426599  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.492259 12022 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:14.474895  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 276667us
I20260709 12:27:14.539858  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:14.475771  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:14.542325  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66907.2us
W20260709 12:27:14.476807  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 283529us
I20260709 12:27:14.477538  8415 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:14.478361  8407 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:14.461992 11841 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.523208 11840 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:14.547900  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.525604 12357 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.402690 12445 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 12:27:14.346055 12314 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.440537 12213 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:14.450388 12447 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.355661  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60585.3us
W20260709 12:27:14.530689  7777 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.656s (SENT) [suppressed 16 similar messages]
W20260709 12:27:14.445150 12429 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 12:27:14.512247 12441 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:14.409978 12375 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.526043 12439 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:14.534967 12376 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:14.538164 12450 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:14.555814 11840 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.546607 12395 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:14.529322 12453 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:14.570024 12436 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:27:14.538903 12444 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:14.569326 12446 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:27:14.634866 12399 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.233s	user 0.014s	sys 0.003s
I20260709 12:27:14.641978  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:14.636782 12174 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.107s	user 0.011s	sys 0.003s
W20260709 12:27:14.650149  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 326693us
I20260709 12:27:14.644928  7778 leader_election.cc:400] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11.
I20260709 12:27:14.570806 12445 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:14.568564 12388 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.675498  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:14.544572 12451 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 12:27:14.551462 12112 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.559026 11841 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.629726 12213 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:14.570443 12402 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.676496 12174 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.680013 12073 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.728168  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2332) took 1155 ms. Trace:
I20260709 12:27:14.728698  7811 rpcz_store.cc:276] 0709 12:27:13.572339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.576028 (+  3689us) service_pool.cc:224] Handling call
0709 12:27:13.576464 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.181382 (+604918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.181419 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.181978 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.529111 (+347133us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:14.529151 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.529239 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.727837 (+198598us) raft_consensus.cc:1685] finished
0709 12:27:14.727865 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.728094 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3702,"raft.queue_time_us":35782,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9534,"thread_start_us":863698,"threads_started":2,"wal-append.queue_time_us":490762}
I20260709 12:27:14.638592 12348 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:14.557178 12386 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:14.721457 12320 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.000s
I20260709 12:27:14.731109 12348 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.731379  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2214) took 2098 ms. Trace:
I20260709 12:27:14.739907  7658 rpcz_store.cc:276] 0709 12:27:12.633367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.118842 (+485475us) service_pool.cc:224] Handling call
0709 12:27:13.119255 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.119759 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:13.225558 (+105799us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.668917 (+443359us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.686346 (+ 17429us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.686510 (+   164us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:13.686762 (+   252us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.720772 (+ 34010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.720897 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.731011 (+1010114us) raft_consensus.cc:1685] finished
0709 12:27:14.731040 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.731282 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54472,"prepare.queue_time_us":103806,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":146,"raft.queue_time_us":182118,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":123554,"thread_start_us":534875,"threads_started":2,"wal-append.queue_time_us":848155}
I20260709 12:27:14.744030  7807 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:14.745319 12174 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.732420 12386 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.564579 12357 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.747156 12456 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:27:14.636198 12351 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:27:14.748744 12351 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:14.749302 12351 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:14.749521 12357 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.750387 12348 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.751339 12349 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:14.750540 12351 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.694552 12458 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:14.752720 12458 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:14.753023 12351 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:14.681020 12085 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:14.638513 12447 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.753823 12285 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.753974 12085 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.754528 12085 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:14.706102 12213 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.712189 12439 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:14.713994 12375 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:14.687130 12395 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:27:14.705832 12343 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:27:14.761878 12343 consensus_queue.cc:1243] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:14.754498 12285 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.762898 12343 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:14.763361 12343 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:14.674124 12399 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.648695 12412 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:27:14.684381 12022 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.764714  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2326) took 1250 ms. Trace:
I20260709 12:27:14.765378  7807 rpcz_store.cc:276] 0709 12:27:13.514563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.514732 (+   169us) service_pool.cc:224] Handling call
0709 12:27:14.764413 (+1249681us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003ea3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:14.764632 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696014592}
I20260709 12:27:14.766335 12399 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.719751 11842 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:14.767709  7808 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:14.720139 12444 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:14.720449 12454 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:27:14.701989 12413 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:14.721354 12112 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:14.722434 12335 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 12:27:14.690393 12445 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:14.725342 11841 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.638381 11975 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:27:14.751796 12349 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:14.753249 12458 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:14.686707 12242 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.003s
W20260709 12:27:14.690029  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45360.9us
I20260709 12:27:14.732602 12311 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.124s
I20260709 12:27:14.759500 12447 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:14.759456 12178 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:14.873646  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.841250 12375 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.716652 12337 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.003s
I20260709 12:27:14.827255 12460 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.828924  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 460301us
I20260709 12:27:14.764626 12285 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:14.834853  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2327) took 1319 ms (client timeout 1546 ms). Trace:
W20260709 12:27:14.937889  7803 rpcz_store.cc:269] 0709 12:27:13.515623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.515792 (+   169us) service_pool.cc:224] Handling call
0709 12:27:14.834585 (+1318793us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54001347d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:14.834780 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950238976}
W20260709 12:27:14.828254  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 473438us
I20260709 12:27:14.857072  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:14.757519  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48469.8us
I20260709 12:27:14.771768 12299 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.775751 12413 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:14.947970  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91053.3us
W20260709 12:27:14.948557  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 393904us
I20260709 12:27:14.948909  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:14.881366 11841 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:14.831019 12412 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:27:14.851100 12350 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.788s	user 0.002s	sys 0.013s
I20260709 12:27:14.761796 12450 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:14.735124 11997 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.916213  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:14.919909  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.925252  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:14.956585  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 263620us
I20260709 12:27:14.956744  7734 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:14.927167  7737 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:14.929123  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.931360  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.823755 12441 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.964129  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:14.935325 12375 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.708534 12205 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.940634  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.979859  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:14.882957  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 335132us
W20260709 12:27:14.883867  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 353211us
I20260709 12:27:14.826174 11842 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.952503 12467 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:14.840202 12335 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.954355  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.956758  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.769809 12343 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:14.934406  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:14.997861  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63607.1us
I20260709 12:27:14.987890  7732 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:14.999533  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:14.964762  7555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 304) took 25638 ms (client timeout 15000 ms). Trace:
I20260709 12:27:15.006109  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.934363 12447 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:15.013562  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:14.957074  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:14.970836  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:14.981606  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.019716  7728 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:14.883165 12242 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:14.966122 12450 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.951356 12413 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.021065  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:15.038813  7627 leader_election.cc:400] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:27:15.040313  7627 leader_election.cc:400] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:27:15.021876  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:15.024823  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:15.033363  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:15.046216  7735 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 7: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:15.048337  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 91320us
I20260709 12:27:15.051271  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:14.940583 12337 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:15.055199  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 77835.9us
I20260709 12:27:14.996135 12232 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:14.947718 12465 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:27:14.978099 12466 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:14.996585 12344 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:27:15.065577  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:15.053826 12205 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:15.000759  7555 rpcz_store.cc:269] 0709 12:26:49.333076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.336103 (+  3027us) service_pool.cc:224] Handling call
0709 12:26:50.365767 (+1029664us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400234bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001f2af3a 00007fc007ce0e9c 00007fc007ddf04e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c 00007fc00025c8c6
0709 12:27:14.964661 (+24598894us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:07.388687 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:07.565038 (+176351us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:07.565125 (+    87us) write_op.cc:435] Acquired schema lock
0709 12:27:07.565148 (+    23us) tablet.cc:662] Decoding operations
0709 12:27:07.567004 (+  1856us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:07.567137 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:07.567170 (+    33us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:27:07.694815 (+127645us) tablet.cc:701] Row locks acquired
0709 12:27:07.694881 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:27:07.695080 (+   199us) write_op.cc:270] Start()
0709 12:27:07.695278 (+   198us) write_op.cc:276] Timestamp: P: 1783600027695005 usec, L: 0
0709 12:27:07.695321 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:27:07.802489 (+107168us) log.cc:844] Serialized 6588 byte log entry
0709 12:27:08.814120 (+1011631us) op_driver.cc:464] REPLICATION: finished
0709 12:27:09.035796 (+221676us) write_op.cc:301] APPLY: starting
0709 12:27:09.035945 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:09.340912 (+304967us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:09.341099 (+   187us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:10.065394 (+724295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:10.065816 (+   422us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:27:10.065894 (+    78us) write_op.cc:312] APPLY: finished
0709 12:27:10.066433 (+   539us) log.cc:844] Serialized 148 byte log entry
0709 12:27:10.792901 (+726468us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:10.881558 (+ 88657us) write_op.cc:454] Released schema lock
0709 12:27:10.894693 (+ 13135us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:11.537702 (+643009us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800474d60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:11.537909 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:11.537953 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5653186,"reported_tablets":49,"spinlock_wait_cycles":246640000,"child_traces":[["op",{"apply.queue_time_us":199374,"apply.run_cpu_time_us":60722,"apply.run_wall_time_us":1893260,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71007,"num_ops":16,"prepare.queue_time_us":113966,"prepare.run_cpu_time_us":23079,"prepare.run_wall_time_us":704073,"raft.queue_time_us":236800,"raft.run_cpu_time_us":17337,"raft.run_wall_time_us":186488,"replication_time_us":1092453,"spinlock_wait_cycles":27506944,"thread_start_us":1111719,"threads_started":5,"wal-append.queue_time_us":1166200,"wal-append.run_cpu_time_us":134973,"wal-append.run_wall_time_us":2972272}]]}
I20260709 12:27:15.056039  7702 leader_election.cc:400] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:15.102329  7702 leader_election.cc:403] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.056800 12073 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:15.021219 12174 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.971895 12383 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:27:14.966382 12350 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.069572  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:15.120596  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51166.1us
I20260709 12:27:15.121351  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:15.122219 12350 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:15.072135  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:15.011188 12369 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:15.074031  7737 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:15.149609  7737 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:15.052610 12338 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.004s
I20260709 12:27:15.077909  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:15.022650 12348 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.059743  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:14.963205 11841 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.080942  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:15.066337  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:14.949249 12460 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.084463  8414 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:15.081548  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:15.159983  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98762.5us
I20260709 12:27:15.023169 12357 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:15.056943 12314 raft_consensus.cc:2764] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:27:15.105896  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:15.169221  7803 raft_consensus.cc:606] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.169878  7803 raft_consensus.cc:614] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.061697 12178 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:15.022369 12444 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:27:15.056684 12402 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:15.124519 12439 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:15.125012 12112 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:15.123667  7805 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:15.060802 12386 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:27:15.128199 12350 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:15.132537 12477 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:27:15.139832 11997 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:14.961688 12349 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:15.141091  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 173654us
I20260709 12:27:15.123019  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:15.124086 12459 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:27:15.072237 12343 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.062363 12471 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:15.177222 12469 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:15.093297  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.163416  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103841us
I20260709 12:27:15.164489  7734 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:15.242688  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.162251  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.217157 12388 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:27:15.167903  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 86490.7us
I20260709 12:27:15.118360 12073 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.166383 12357 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:15.152048 12376 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.006s
I20260709 12:27:15.152024  7556 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:15.189496  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:15.181771 12439 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.197693  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2652) took 1563 ms. Trace:
W20260709 12:27:15.200001  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2330) took 1669 ms (client timeout 1694 ms). Trace:
I20260709 12:27:15.182040 12112 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.162667 12338 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.165171 12285 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:15.204026 12375 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.204604  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:15.205219 12441 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.206413 12395 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:15.209026 12262 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:15.159380 12369 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.211180 12320 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:15.214792 11841 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.149068 12466 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:15.188344 12402 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.173664 11863 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.217036 12178 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:15.224400  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 101476us
I20260709 12:27:14.949437 12445 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.148505 12174 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.246528  7808 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:15.246294  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:15.150473  7737 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:15.223367  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.004s
W20260709 12:27:15.247516  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.402657  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:15.384559  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2546) took 1148 ms. Trace:
I20260709 12:27:15.405154  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.317709  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:15.407076  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:15.407878  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.008s	sys 0.013s
I20260709 12:27:15.408414  7658 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.409265  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 784) took 1007 ms. Trace:
W20260709 12:27:15.289456  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 127291us
I20260709 12:27:15.406440  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:15.288447  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:15.421552  7810 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:15.416894  7737 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:15.425113  7805 rpcz_store.cc:269] 0709 12:27:13.530158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.553966 (+ 23808us) service_pool.cc:224] Handling call
0709 12:27:15.199781 (+1645815us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003f4654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:15.199928 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3603211136}
I20260709 12:27:15.427598  7729 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:15.417505  7736 rpcz_store.cc:276] 0709 12:27:13.634647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.634834 (+   187us) service_pool.cc:224] Handling call
0709 12:27:13.635361 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.128994 (+493633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.129026 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.605628 (+476602us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b18002112a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:27:14.606963 (+  1335us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:14.633418 (+ 26455us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.937246 (+303828us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:14.965309 (+ 28063us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.965592 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.066269 (+100677us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:15.066325 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.196118 (+129793us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b18002112a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1
0709 12:27:15.197409 (+  1291us) raft_consensus.cc:1685] finished
0709 12:27:15.197442 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.197620 (+   178us) inbound_call.cc:177] Queueing success response
0709 12:27:15.197666 (+    46us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b18002112a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
Metrics: {"mutex_wait_us":92831,"prepare.queue_time_us":33432,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":65821,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":33530,"spinlock_wait_cycles":1071872384,"thread_start_us":259509,"threads_started":1,"wal-append.queue_time_us":429384}
I20260709 12:27:15.224936 12337 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.443539  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.251389  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:15.409960  8406 rpcz_store.cc:276] 0709 12:27:14.402759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.402978 (+   219us) service_pool.cc:224] Handling call
0709 12:27:14.403536 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:14.403842 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:14.685458 (+281616us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.408693 (+723235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.408792 (+    99us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:15.408877 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.408997 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.409196 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":506717,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":140,"thread_start_us":942994,"threads_started":2,"wal-append.queue_time_us":678270}
I20260709 12:27:15.248382 12451 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:15.388772 12351 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
W20260709 12:27:15.361269 12453 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.003s
I20260709 12:27:15.271286  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:15.474457  7807 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:27:15.449257  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:15.451268  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 261825us
W20260709 12:27:15.361624 12482 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:27:15.359315 12469 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.494036  7661 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:15.317564 12467 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:27:15.478382  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.499650  7657 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.483739  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.484455  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.484987  7655 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:15.483083  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.259660 12242 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:15.364042 11840 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.714s	user 0.006s	sys 0.007s
I20260709 12:27:15.263252 12335 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:15.412077 12357 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:15.488417 12344 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:15.248170 12412 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:15.411902 12073 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.494617  7808 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
W20260709 12:27:15.388203 12305 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.910s	user 0.012s	sys 0.001s
I20260709 12:27:15.287881 12299 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.476912  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.419615 12386 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.261620 12464 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:15.223019 12350 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.406071 12388 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:15.391963 12349 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.499336  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.481086  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.501730  7660 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.230013 12314 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:15.564769  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:15.418512 12022 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.404951  7653 rpcz_store.cc:276] 0709 12:27:14.236237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.236379 (+   142us) service_pool.cc:224] Handling call
0709 12:27:14.236749 (+   370us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:14.630666 (+393917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.630719 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:14.640966 (+ 10247us) log.cc:844] Serialized 78 byte log entry
0709 12:27:14.954941 (+313975us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:14.954983 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.955057 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.384094 (+429037us) raft_consensus.cc:1685] finished
0709 12:27:15.384132 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.384458 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22177,"raft.queue_time_us":436774,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":2712,"thread_start_us":630114,"threads_started":2,"wal-append.queue_time_us":256353}
I20260709 12:27:15.576102  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.417068 12395 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:15.415318 12262 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:15.598067  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:15.598551  7662 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.598812  7810 raft_consensus.cc:606] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:15.489928 12213 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.002s
I20260709 12:27:15.599189  7810 raft_consensus.cc:614] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.443089 12320 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:15.245595 11997 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.249s
I20260709 12:27:15.561585 12386 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.442941 12112 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.248246 12465 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:15.442301 12439 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.570570 12314 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.535478 11840 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.493464 11863 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:15.505734 12404 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.493503 12453 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:15.511005 12375 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:15.443531 12285 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.526736 12467 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:15.490370 12451 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.596992  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:15.418762 12376 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.597975 12449 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.505159 12469 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.618266 11997 consensus_queue.cc:1243] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:15.613821 12386 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.562135 12412 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.546491 12413 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.643287  8415 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.620962 12314 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:15.636246 12205 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.001s
I20260709 12:27:15.625212 12255 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:15.596992 12022 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:15.653117 12498 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:15.613130 12500 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:27:15.657918 12451 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:15.688766 12255 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.596727 12466 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.641888 12395 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.692413 12395 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.660081 12350 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.614483 12178 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.661175 12402 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.662212 12445 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.663107 12285 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.631349 12404 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.665424 11842 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:15.627635 12465 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.670499 12413 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.636873 12375 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.670774 11840 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.606801 12213 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.680327 12369 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.681187 12081 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:15.709368 12081 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:15.709080 12467 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.709947 12174 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:15.643694 12262 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.710518 12412 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.710400 12081 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:15.711710 12085 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.006s
I20260709 12:27:15.712172 12081 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.713130 12213 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.634343 11863 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:15.715328 12375 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.684154 12022 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.635684 12388 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.730768 12357 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.733016 12404 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.706607 12460 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:15.746579 12458 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.863s	user 0.013s	sys 0.001s
I20260709 12:27:15.744349 12469 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.757462 12458 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:15.517930  7556 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:15.814050  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.026s	sys 0.050s
W20260709 12:27:15.956235  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.026s	sys 0.050s
W20260709 12:27:15.823657  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.016s	sys 0.001s
I20260709 12:27:15.962543 12458 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:15.963070  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.016s	sys 0.002s
W20260709 12:27:15.964771  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 722100us
W20260709 12:27:15.965662  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.966423  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.966960  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.594203  7658 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:15.825477  7653 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.826093  7656 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:15.828272  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.836225  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.842038  7660 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:15.852694  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.853119  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.566371  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 324879us
I20260709 12:27:15.595407  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:16.015560  7662 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:15.568965  7728 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.002s
I20260709 12:27:16.017154  7728 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:16.018253  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2515) took 1067 ms. Trace:
I20260709 12:27:15.601761  7729 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:15.857483  7730 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:15.857890  7731 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:27:15.860601  7735 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:15.637307  7808 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:15.637542  7804 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:15.882862  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.024621  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.024804  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:15.698695  8415 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:15.887213 12376 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.015638  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.029414  7660 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:16.030594  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:15.886859  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:15.716224 12255 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:16.033918  7660 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:15.716192 11863 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.036602 11932 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:16.036887 12478 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:16.018806  7728 rpcz_store.cc:276] 0709 12:27:14.951007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.951161 (+   154us) service_pool.cc:224] Handling call
0709 12:27:16.017911 (+1066750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.895888 12232 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:16.040841  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:15.745411 12388 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:16.044638  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.046197  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:15.712184 12085 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.050009 12085 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:15.725648 12262 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:16.052305  7737 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.629s	user 0.015s	sys 0.004s
W20260709 12:27:16.052701  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 457252us
I20260709 12:27:16.053362  7737 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:16.054333  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 924) took 1023 ms. Trace:
I20260709 12:27:15.904241  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:15.906177 12357 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.909197  8409 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:15.912997  8405 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: [8.10-8.10]
W20260709 12:27:15.754557 12450 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.000s
I20260709 12:27:15.916359 12439 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:15.740689 12465 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:15.918804 12455 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.514464  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:15.966730  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:15.967842 12469 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:15.968487  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.055454  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:16.001724  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.002828  7661 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:16.005963  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.002928  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:16.014633  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.064034  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.063911  7655 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:15.958540  7556 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.059201 12469 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.059679 12455 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.061411  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.063464 12450 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.082387  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.061321 12460 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:16.054801  7737 rpcz_store.cc:276] 0709 12:27:15.031349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.073538 (+ 42189us) service_pool.cc:224] Handling call
0709 12:27:15.073895 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.053903 (+980008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.054260 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:16.072072  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.055904  7812 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:27:16.076749  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.077400  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.078616  8405 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:16.077582  7556 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 7, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:16.091853  7556 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 2 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.093863 12112 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.080914 12455 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.080736 12439 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:16.081981  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.082499  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 116045us
I20260709 12:27:16.096498  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.100593  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:16.094013  7556 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 9, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.094429  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.081261  7659 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:16.100754  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.067412  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.100378 12112 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.113333  7654 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:16.114307  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.119287  7660 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:16.095199  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.098460 12450 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.134363  7661 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:16.126098  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70975.5us
I20260709 12:27:16.103022 12469 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.139003  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:16.144104  7804 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 8: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:16.147076  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.108155 12085 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:27:16.151798  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.140561 12450 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.146539  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:16.159123  7657 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.019s
I20260709 12:27:16.159575  8409 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 7: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:16.159878  7657 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:16.161254  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.162295  7657 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:16.165925 12073 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.012s
I20260709 12:27:16.143112 12376 raft_consensus.cc:2764] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:16.169368 12314 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.172585  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.173761  8406 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:16.174338  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:16.176009  7655 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:16.160280  7556 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 6, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 6 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:16.181939  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.175143  8412 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:16.159150 12450 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.189949  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.189471  7556 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 6, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 6 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.190527  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2509) took 1275 ms. Trace:
W20260709 12:27:16.191156  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 88446.8us
I20260709 12:27:16.205209  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.206228  7655 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:16.206776 12466 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:16.207105 12467 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:16.207674  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.208689 12314 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.209187 12450 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.209470 12466 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:16.209913 12466 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:16.166659 12073 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.191915  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.221778  7653 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:16.191393  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.210920  7556 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:16.193396  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.191092  7731 rpcz_store.cc:276] 0709 12:27:14.915278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.915804 (+   526us) service_pool.cc:224] Handling call
0709 12:27:16.190263 (+1274459us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:16.190459 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071302528}
I20260709 12:27:16.233887 12357 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:16.234660 12463 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:27:16.227560 11932 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.237439 11932 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:16.243492 12496 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.589s	user 0.003s	sys 0.003s
I20260709 12:27:16.246593  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.247943  7662 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:16.249125  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.249886  7662 raft_consensus.cc:2376] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9. Re-sending same reply.
I20260709 12:27:16.250339  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2521) took 1061 ms. Trace:
W20260709 12:27:16.252938  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76303.7us
I20260709 12:27:16.255640  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.255837 12500 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:16.223148 12458 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:16.257814  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 87977us
I20260709 12:27:16.259542  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:16.260753 12463 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:16.261296 12463 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:16.261790 12463 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:16.262332  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72642.8us
I20260709 12:27:16.263655  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:16.264703  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.251077  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.184336 12085 consensus_queue.cc:1243] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:16.274420  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.281216  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.284116  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.284931  7660 raft_consensus.cc:2376] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10. Re-sending same reply.
I20260709 12:27:16.254985 12073 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:16.287451  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.250763  7734 rpcz_store.cc:276] 0709 12:27:15.188997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.404570 (+215573us) service_pool.cc:224] Handling call
0709 12:27:15.404966 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.857702 (+452736us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:15.858583 (+   881us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.953061 (+ 94478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.953161 (+   100us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:15.953248 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.249777 (+296529us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54001c57d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:16.249835 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.250129 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70147,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":101,"raft.queue_time_us":4858,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":362,"spinlock_wait_cycles":1038806912,"thread_start_us":84562,"threads_started":1,"wal-append.queue_time_us":173604}
I20260709 12:27:16.293175  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:16.194306 12460 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.293329  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.295775 11863 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:16.298454  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:16.296608 12513 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:16.305809  7805 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:27:16.211122 12466 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.313277  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.303642  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:16.187858  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:16.315706  7804 raft_consensus.cc:606] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.316203  7804 raft_consensus.cc:614] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:16.295388  7662 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:16.297207  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.159654 12320 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.307618 12450 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:16.307420 12081 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:27:16.321676  7654 raft_consensus.cc:2376] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11. Re-sending same reply.
I20260709 12:27:16.302804  7659 raft_consensus.cc:2376] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10. Re-sending same reply.
I20260709 12:27:16.260025 12395 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:27:16.305359 11863 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:16.297710 11932 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.294090 12357 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.336773 12357 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.335381 12255 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:16.337441  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.337982 12255 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:16.338331  7805 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:16.338569 12255 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:16.339672  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 927) took 1307 ms. Trace:
I20260709 12:27:16.340183  7735 rpcz_store.cc:276] 0709 12:27:15.032218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.065227 (+ 33009us) service_pool.cc:224] Handling call
0709 12:27:16.339427 (+1274200us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400348854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:16.339601 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785673600}
I20260709 12:27:16.323077  8412 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:16.338613  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2358) took 1013 ms. Trace:
I20260709 12:27:16.344131  7660 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:27:16.345409  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2348) took 1040 ms. Trace:
I20260709 12:27:16.336814  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.347609  7659 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:16.335970  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.347517  7654 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:16.336943  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:16.343086  7807 rpcz_store.cc:276] 0709 12:27:15.324989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.600379 (+275390us) service_pool.cc:224] Handling call
0709 12:27:15.600726 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.876056 (+275330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.876154 (+    98us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.876265 (+   111us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:15.876329 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.338180 (+461851us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54003ea8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:16.338241 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.338437 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631613696}
I20260709 12:27:16.355031  7658 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:16.359534  8408 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:27:16.359009  8415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 887) took 1393 ms (client timeout 1725 ms). Trace:
I20260709 12:27:16.342166 11863 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:16.363513  7657 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:27:16.362309  8415 rpcz_store.cc:269] 0709 12:27:14.948656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.948929 (+   273us) service_pool.cc:224] Handling call
0709 12:27:16.341699 (+1392770us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:16.341886 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559848320}
I20260709 12:27:16.369481  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 922) took 1338 ms. Trace:
I20260709 12:27:16.369999  7730 rpcz_store.cc:276] 0709 12:27:15.030657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.032602 (+  1945us) service_pool.cc:224] Handling call
0709 12:27:15.032950 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.021172 (+988222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.366338 (+345166us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b5400283054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:16.369368 (+  3030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894268160}
I20260709 12:27:16.372380  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.333987  7556 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 5, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 5 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:16.381207  7804 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:16.384845  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.388182  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.389112  7807 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:16.346040  7806 rpcz_store.cc:276] 0709 12:27:15.243320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.245069 (+  1749us) service_pool.cc:224] Handling call
0709 12:27:15.262944 (+ 17875us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:15.263454 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.263488 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:15.264167 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 12:27:15.533980 (+269813us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:15.534024 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.534094 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.242324 (+708230us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:16.264397 (+ 22073us) raft_consensus.cc:1685] finished
0709 12:27:16.264420 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.283186 (+ 18766us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24662,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":39294,"spinlock_wait_cycles":7469824,"thread_start_us":269385,"threads_started":1,"wal-append.queue_time_us":370492}
W20260709 12:27:16.392795 12509 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
W20260709 12:27:16.393405 12515 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.129s	user 0.001s	sys 0.002s
I20260709 12:27:16.394155  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2359) took 1039 ms. Trace:
I20260709 12:27:16.394605  7808 rpcz_store.cc:276] 0709 12:27:15.354365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.602877 (+248512us) service_pool.cc:224] Handling call
0709 12:27:15.603214 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.023013 (+419799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.393640 (+370627us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:16.393939 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77861120}
I20260709 12:27:16.396162  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.398056  7656 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:16.401576  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:16.408941  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 111904us
I20260709 12:27:16.409909  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:16.412370  7812 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:16.413300  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.419097  7657 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:16.413295  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.414983  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:16.416425  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 146495us
I20260709 12:27:16.353446 12478 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:27:16.427372  7805 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:16.428035  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.431469  7662 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:16.431970  7804 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:27:16.408663 12357 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.438603  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.438086  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:16.444940  7658 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11.
I20260709 12:27:16.428519 12320 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:16.452139  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.442698  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:16.457361  7808 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:16.459196  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:16.463595  7660 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:27:16.447108 12386 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.649s	user 0.016s	sys 0.011s
I20260709 12:27:16.459544 12357 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.470647  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:16.474472  7660 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:16.484238  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.484773  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 219095us
W20260709 12:27:16.489646  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.480432 12073 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:16.509459  7658 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:16.492831  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 79866.1us
W20260709 12:27:16.501191  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88077.3us
W20260709 12:27:16.511654  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.515021  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.516274  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.495932 12467 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:16.526173  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:16.527034  7657 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:27:16.528371  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 216449us
I20260709 12:27:16.497581  7556 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 8 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:16.542066  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:16.545636 12337 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:16.546676  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:16.526235 12357 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.547734 12314 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:16.548720  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.549602 12404 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:16.550143  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:16.551656 12314 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:16.552284 12514 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 12:27:16.552639  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.553560  7736 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:16.555480 12314 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:27:16.558328  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:16.558851  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:16.559710 12450 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:16.560186 12450 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.550246  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.560637 12450 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:16.562134  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.562978  7737 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:16.565542  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.534901 12085 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.567135  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:16.568141 12085 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.569087  7729 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:16.570708 12085 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.514683 12073 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:16.571282  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:16.480125 12386 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:16.583940  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.586066  7729 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:16.545662  7556 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 3 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:16.581296 12073 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:16.603248  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 147458us
I20260709 12:27:16.550302 12404 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:16.608350  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2349) took 1359 ms. Trace:
W20260709 12:27:16.614774  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.615622  7736 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:27:16.615914 12388 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.616611 12388 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.618953 12386 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:16.599038 12460 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.615772  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:16.642123  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116211us
W20260709 12:27:16.654213  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57432.5us
I20260709 12:27:16.661055  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:16.662958  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.665243  7729 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:16.667073 12463 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.670365  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:16.670089  7803 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:16.631495 12511 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.419s	user 0.004s	sys 0.000s
I20260709 12:27:16.671774  7556 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:27:16.677013  7661 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 12:27:16.678321  7661 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:16.679774  7659 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:16.678865  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:16.619150 12388 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.684963 12467 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:16.686043  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71885.4us
I20260709 12:27:16.688256  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:16.650499 12450 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:16.669940 12460 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.692019  7655 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:16.692669  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2236) took 1206 ms. Trace:
I20260709 12:27:16.693727 12386 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:16.619385 12213 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:27:16.695961 12213 consensus_queue.cc:1243] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:16.696740 12213 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:16.697172 12213 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:16.701923  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 751) took 1139 ms. Trace:
I20260709 12:27:16.692991  7655 rpcz_store.cc:276] 0709 12:27:15.485847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.204716 (+718869us) service_pool.cc:224] Handling call
0709 12:27:16.692599 (+487883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:16.683468  7556 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 8, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.702491  7810 rpcz_store.cc:276] 0709 12:27:15.562622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.612092 (+ 49470us) service_pool.cc:224] Handling call
0709 12:27:15.612569 (+   477us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:15.615629 (+  3060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.615684 (+    55us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:15.626725 (+ 11041us) log.cc:844] Serialized 78 byte log entry
0709 12:27:16.135302 (+508577us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:16.171691 (+ 36389us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.172353 (+   662us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.172595 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.172753 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.488176 (+315423us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:16.701470 (+213294us) raft_consensus.cc:1685] finished
0709 12:27:16.701503 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.701710 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14846,"prepare.queue_time_us":16060,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":5603,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":7914,"spinlock_wait_cycles":6849408,"thread_start_us":126925,"threads_started":1,"wal-append.queue_time_us":567939}
I20260709 12:27:16.710739  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:16.713001  7807 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:16.717523  7808 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:16.612258  7811 rpcz_store.cc:276] 0709 12:27:15.243611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.245171 (+  1560us) service_pool.cc:224] Handling call
0709 12:27:15.245566 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.474278 (+228712us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:15.530813 (+ 56535us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.120397 (+589584us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800399400. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:27:16.283169 (+162772us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.366357 (+ 83188us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.393467 (+ 27110us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:16.393865 (+   398us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.394151 (+   286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.394896 (+   745us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:16.394946 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.602703 (+207757us) raft_consensus.cc:1685] finished
0709 12:27:16.602731 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.602923 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62479,"prepare.queue_time_us":63933,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":93,"raft.queue_time_us":105874,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":15140,"spinlock_wait_cycles":403396096,"thread_start_us":88633,"threads_started":1,"wal-append.queue_time_us":836239}
I20260709 12:27:16.694828 12460 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:16.750772  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 62623.3us
W20260709 12:27:16.752368  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 144645us
W20260709 12:27:16.759464  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 95210.5us
I20260709 12:27:16.750824 12337 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:16.698689 12213 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:16.771715  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:16.768560  7556 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.786242  7655 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:16.771510  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:16.799382  7661 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:16.779597 12463 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:16.827978 12465 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:16.789372 12386 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:16.789000 12085 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:27:16.771097 12376 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:27:16.856117  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 176634us
I20260709 12:27:16.847131  7556 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:16.846578 12337 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:16.812000 12112 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:16.871248 12213 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:17.014868  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:17.016287  7812 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:27:17.019817  7812 pending_rounds.cc:85] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 8
W20260709 12:27:17.010874  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.022704  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.037514  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.040427  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 225223us
I20260709 12:27:16.874965 12386 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:16.809057 12404 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.020200  7812 pending_rounds.cc:107] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:27:17.076313  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:17.086427  7812 raft_consensus.cc:2889] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:17.072501  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196670us
I20260709 12:27:17.067139  7805 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:27:17.055557  7807 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:17.068387  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.033819 12112 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.082552 12255 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.001s
I20260709 12:27:16.905793 12073 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:17.107033  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.105161  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.035434 12450 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.060940 12466 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:17.114329 12533 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.147545 12533 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.109977  7556 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:17.115839  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.151311  7807 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:17.153316  7807 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:27:17.154256 11863 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.736s	user 0.007s	sys 0.007s
I20260709 12:27:17.122521 12357 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:17.124738 12526 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.295s	user 0.009s	sys 0.002s
I20260709 12:27:17.125416 12314 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:17.061803 12460 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:17.057750 12478 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.046643 11840 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.157749 12478 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:17.128679  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60390.8us
I20260709 12:27:17.159169 12073 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.127763  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.042903 12337 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.165541 12337 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.135776 12255 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.166172  7807 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:17.157868 11840 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:17.165668  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.139840 12386 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.067504 12530 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:17.168061  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:17.161717  7556 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 9, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:17.045635 12525 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.164999 12478 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.173125  7809 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:27:17.162886  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.129328 12112 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.049696 12529 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:17.191568  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 844) took 1119 ms. Trace:
I20260709 12:27:17.199465  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:17.202543  7812 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:17.186255  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.202355  7656 rpcz_store.cc:276] 0709 12:27:16.061486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.416327 (+354841us) service_pool.cc:224] Handling call
0709 12:27:16.436896 (+ 20569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.443814 (+  6918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.443848 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.471549 (+ 27701us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.767872 (+296323us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:16.767925 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.768008 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.123719 (+355711us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:17.169006 (+ 45287us) raft_consensus.cc:1685] finished
0709 12:27:17.169045 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.180672 (+ 11627us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":15347,"spinlock_wait_cycles":42997504,"thread_start_us":258815,"threads_started":1,"wal-append.queue_time_us":336937}
I20260709 12:27:17.205900  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2232) took 1721 ms. Trace:
I20260709 12:27:17.206847  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:17.208010  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:17.208405 12357 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:17.206413  7653 rpcz_store.cc:276] 0709 12:27:15.484646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.306610 (+821964us) service_pool.cc:224] Handling call
0709 12:27:16.365294 (+ 58684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.412629 (+ 47335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.412669 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.442058 (+ 29389us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.781614 (+339556us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:16.781698 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.781815 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.205635 (+423820us) raft_consensus.cc:1685] finished
0709 12:27:17.205658 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.205832 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":262629,"threads_started":1,"wal-append.queue_time_us":629499}
I20260709 12:27:17.209560  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:17.210429  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.211715 12530 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.214803 12530 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.214996 11840 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.211746  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:17.219787  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2553) took 1260 ms. Trace:
I20260709 12:27:17.224289  7659 rpcz_store.cc:276] 0709 12:27:15.958977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.366717 (+407740us) service_pool.cc:224] Handling call
0709 12:27:16.679577 (+312860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.680163 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.219484 (+539321us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400116050. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:17.219695 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647100160}
I20260709 12:27:17.225651 12525 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.226974  7806 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:17.229835 12525 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.212347  7807 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:17.235419  7805 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.240236  7812 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:17.154805 11863 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:17.187057 12320 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.003s
I20260709 12:27:17.243466  7806 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:17.246088 12465 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:17.246353 12357 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:17.256596 12320 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.257225 12320 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.247157 12255 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.247771  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:17.241287 12073 raft_consensus.cc:2764] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:27:17.251402 12488 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.260218 12488 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.262740 12488 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.252450  7811 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:17.254441 11863 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:17.255419 12505 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.264439 12505 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.265161  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:17.266469 12314 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:17.267464 12314 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:17.267277 12505 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.268719 12314 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.271610 12505 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:17.273310 12112 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:17.273851 12488 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:17.274518 12488 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:17.276216 12488 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.279807 12232 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:17.281991 12337 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:17.283751 12337 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:17.284292 12337 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:17.285979 12337 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.286849 12488 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:17.287856 12337 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:17.292804  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 131221us
I20260709 12:27:17.294015  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.192864 12533 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.295292 12450 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:17.296612 12450 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:17.299734 12465 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:17.301035 12465 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:17.303717  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56137.7us
I20260709 12:27:17.304423  8415 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:27:17.305374  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.305691 12529 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:17.306496 12529 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:17.310086  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.309886 12460 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:17.311470  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2768) took 1282 ms. Trace:
I20260709 12:27:17.307890 12529 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.312115  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:17.312868 11863 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:17.310981 12460 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.313493 11863 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:17.313795 12460 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:17.314844 12460 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.316211 12460 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.320032 12469 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:17.320333  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 756) took 1002 ms. Trace:
I20260709 12:27:17.311892  7803 rpcz_store.cc:276] 0709 12:27:16.028663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.184939 (+156276us) service_pool.cc:224] Handling call
0709 12:27:16.268491 (+ 83552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.673646 (+405155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.311174 (+637528us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54003728d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:17.311400 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923026176}
I20260709 12:27:17.314616 11863 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.321519  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.323484 12525 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:17.324108 12458 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.324656 12458 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:17.324335  7556 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 9, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:17.324115 12525 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:17.327901 12525 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.324923 12469 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:17.334654 12469 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.335080  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.338654 12073 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:17.339756 12073 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:17.340749  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55119.2us
I20260709 12:27:17.340965 12073 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.343854  7556 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:17.346467  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:17.315738  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:17.350498  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 285381us
I20260709 12:27:17.348273  7806 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:17.345885  7556 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 7 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:17.321161  7808 rpcz_store.cc:276] 0709 12:27:16.318281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.476893 (+158612us) service_pool.cc:224] Handling call
0709 12:27:16.713793 (+236900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.739538 (+ 25745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.319999 (+580461us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400331b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:17.320221 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523803904}
I20260709 12:27:17.358772  7805 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:17.360834  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:17.364773  7556 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 11, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 11 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:17.362160  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.368251  7804 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:17.339006  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:17.375779  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.376608  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:17.377693  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.379123  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:17.382053  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.386961  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93084.9us
I20260709 12:27:17.390980  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.389212  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:17.395957  7803 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:17.388247  7556 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 6, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 6 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:17.399425  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:17.400993  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:17.406196  7653 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:17.398716  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.413380  7805 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:17.325794 12458 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.390738 12255 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:17.418989  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 793686us
I20260709 12:27:17.410053  7661 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:17.420825  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.421732  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.424815  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:17.375528 12232 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:17.414489 11840 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:17.433493  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.445358 11840 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:17.423146  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.445894 11840 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.436508  7662 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:17.448320  7662 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [4.3-9.6]
I20260709 12:27:17.450439  7662 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:17.389873 12530 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:17.451579 12530 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.452154  7659 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:27:17.454044 12395 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.454592 12530 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.445781  7653 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 8: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:17.457015 12500 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.457563 12500 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:17.438681  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.441480  7661 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:27:17.444415  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.464530  7660 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:17.454738 12395 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.466979  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:17.471007  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.472304  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:17.439061  7734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.008s	sys 0.015s
I20260709 12:27:17.415189 12463 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.473614  7734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.009s	sys 0.016s
I20260709 12:27:17.420610 12178 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.414968 12478 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:17.432629 12320 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.435405 12255 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:17.491938  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67263.1us
I20260709 12:27:17.501646  7655 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:17.507663  7657 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:27:17.509927 12357 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:17.510932 12357 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.419112 12112 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:27:17.490906 12536 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:17.516922  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:17.522303  7777 leader_election.cc:400] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11.
I20260709 12:27:17.524053  7777 leader_election.cc:403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:17.506590 12255 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.516052 12458 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:17.536232  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 182859us
I20260709 12:27:17.515985 12530 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:17.541646  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122168us
I20260709 12:27:17.495973 12463 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.458847 12500 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:17.547488  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.560236  7728 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:17.577502  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 166948us
I20260709 12:27:17.523509 12213 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:27:17.540306 12536 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:17.589564  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:17.507238 12178 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.576193 12525 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11.
I20260709 12:27:17.612933  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:17.614604  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.614789  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.612624  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.590679 12530 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:17.624359  7805 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:17.641077  7806 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:27:17.641687  7806 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 8
I20260709 12:27:17.641990  7806 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 8.9
W20260709 12:27:17.642066  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.642328  7806 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:17.642756  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.644564  7657 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:17.585677 12504 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.230s
I20260709 12:27:17.650646  7655 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:27:17.609084 12529 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:17.652889  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92865.2us
I20260709 12:27:17.651868  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.588266 12478 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:27:17.623255 12357 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:17.643455 12450 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:17.670564  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 191680us
I20260709 12:27:17.639835 12500 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.646086 12112 raft_consensus.cc:2764] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:27:17.622478 12533 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:27:17.603611 12399 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.259s
W20260709 12:27:17.707324 12534 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:17.712201  7662 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260709 12:27:17.712719  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 210108us
W20260709 12:27:17.721534 12538 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:27:17.725136  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2523) took 1184 ms. Trace:
I20260709 12:27:17.725625  7728 rpcz_store.cc:276] 0709 12:27:16.540692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.545645 (+  4953us) service_pool.cc:224] Handling call
0709 12:27:16.546158 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.584513 (+1038355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.724839 (+140326us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400121e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:17.725044 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312382848}
I20260709 12:27:17.726203 12469 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.726938 12469 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.727141 11932 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:17.727638 12178 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.728003 11932 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:17.728214 12178 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.728525 11932 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.728876 11932 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:17.728750 12178 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.721606 12395 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.729794 11932 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.681299 12529 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.733560 12529 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.734086 12545 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:17.734754 12545 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.740693 12545 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.741147 12530 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.742553 12537 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:17.742986  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:17.666190 12085 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:27:17.744418  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.745452 12085 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:17.746210 12085 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:17.746727  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:17.747982 12399 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.748462 12399 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:17.749609 12545 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:17.749375 12536 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.751199  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:17.749629 12399 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:17.665504 12544 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:27:17.762930 12399 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.764101 12478 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.768285 12466 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:17.768829 12466 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:17.769657 12548 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 12:27:17.769840 12466 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.771404 12404 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:17.771857 12404 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:17.772922 12404 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.774323 12469 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.774437 12404 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:17.752774  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:17.776386 12463 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:17.776727 12320 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:17.743134 12537 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:17.791811  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 209839us
I20260709 12:27:17.804487  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.805542  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.806267  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.806903  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.807989  7729 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:27:17.809247  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.809962  7732 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:17.810088  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.810679  7728 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 6.
I20260709 12:27:17.730975 12178 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:17.819774  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:17.731123 11932 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.779366 12466 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:17.717242 12213 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:17.814862 12544 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:17.822170  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.793018  7803 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:17.836726  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.842103  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:17.791791 12320 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:17.855628  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102921us
I20260709 12:27:17.856547  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:17.857574  7656 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:27:17.859630  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:17.867637  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.868857  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.866822  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:17.866808  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:17.874439  8413 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:17.876995  7737 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:17.880460  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:17.881462  8408 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:17.881949  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:17.883774  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:17.890508  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.893618  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:17.894225  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.895560  7732 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:17.866982  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.900167  7730 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:17.901942  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:17.904611  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:17.779002 12085 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:17.905340  7729 raft_consensus.cc:2376] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8. Re-sending same reply.
I20260709 12:27:17.858382 12320 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.904204  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:17.934278  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:17.939296  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:17.944269  7729 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:27:17.946592  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.948025  7737 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:17.948770  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.951912 12543 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:17.953830 12560 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:17.954545 12560 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.955531 12560 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:17.959685 12178 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:17.952711 12543 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.962532 12543 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:17.969933  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.971120  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:17.911772 12357 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:17.856712 12213 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:17.976428  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71315.1us
W20260709 12:27:17.977603  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 268103us
W20260709 12:27:18.047756  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.062130  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.115828  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:18.117393  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:18.119386  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.141816  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 248260us
W20260709 12:27:18.150180  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 328125us
I20260709 12:27:18.055112 12395 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:18.056910 12357 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:18.181857  7803 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:18.053345 12536 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:18.190712  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:18.192226  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:18.193352  7805 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:18.197530  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 293433us
W20260709 12:27:18.148325 11840 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
W20260709 12:27:18.179152 12255 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.620s	user 0.020s	sys 0.002s
I20260709 12:27:18.225654  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:18.232411  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.234139  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 340076us
W20260709 12:27:18.235833  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.237759  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:18.227808  7806 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:18.050132 12557 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.233s
I20260709 12:27:18.181591 12450 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:18.241778  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 114380us
I20260709 12:27:18.244349  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:18.217861 12540 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:18.269517  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.249091  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70525.7us
W20260709 12:27:18.292887  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44936.8us
I20260709 12:27:18.318037  8415 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:27:18.318200  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80553.8us
I20260709 12:27:18.232764 12469 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.181813 12554 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.460s
I20260709 12:27:17.985215 12537 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.337520  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:18.337040  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.340981  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:18.343652  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.343524  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:18.344811  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144209us
I20260709 12:27:18.240763 12550 raft_consensus.cc:2764] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:18.244427 12558 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:18.355712 12399 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.003s
I20260709 12:27:18.356824 12579 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:27:18.233402 11840 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:18.353485  7731 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:18.239147 12255 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.245450 12536 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:18.416328  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.415777  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.425747  7657 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:18.379428  7700 leader_election.cc:400] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 7. Current term is 8.
I20260709 12:27:18.327498 12553 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.321979 12467 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:18.422052  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2567) took 1005 ms. Trace:
W20260709 12:27:18.447666  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 106792us
W20260709 12:27:18.444515  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.446411  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:18.362643 12556 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:18.362767 12460 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:18.417569 12458 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.005s
W20260709 12:27:18.406396 12563 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:27:18.475059  7659 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:18.457310  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:18.420048 12552 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
W20260709 12:27:18.413941 12576 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:18.457463  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:18.419103 12337 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.131s	user 0.012s	sys 0.003s
I20260709 12:27:18.407011 12213 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:18.420866 12558 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:18.479147  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57569.1us
I20260709 12:27:18.488718  7728 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:18.496206  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:18.410024 12587 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:18.459132 12112 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:18.485582 12466 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.008s
I20260709 12:27:18.453150 12557 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.489549  7804 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:18.492746  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:18.353547 12469 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:18.524690  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.339125 12540 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:18.520744  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 150997us
I20260709 12:27:18.532472  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:18.533195  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:18.522109  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 177413us
W20260709 12:27:18.531039  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44978.3us
I20260709 12:27:18.463965 12537 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:18.523083  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.537518  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.364183 12450 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.551321  7805 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:18.376559 12357 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:18.549309 12399 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.592077 12399 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:18.593271 12469 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:18.594533 12399 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:18.596913 12572 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 12:27:18.598330  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2571) took 1158 ms. Trace:
I20260709 12:27:18.598788  7658 rpcz_store.cc:276] 0709 12:27:17.440063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.440198 (+   135us) service_pool.cc:224] Handling call
0709 12:27:17.453530 (+ 13332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.656258 (+202728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.656291 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.704961 (+ 48670us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.972178 (+267217us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:17.972385 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.972474 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.597991 (+625517us) raft_consensus.cc:1685] finished
0709 12:27:18.598015 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.598215 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21007,"raft.queue_time_us":56703,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":319426,"threads_started":2,"wal-append.queue_time_us":340389}
I20260709 12:27:18.600551 12501 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.600811  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.601780  7658 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:18.601328 12501 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.602464  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:18.603566  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.603991  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:18.604149  7808 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:18.604677  7662 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:18.604921  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.605202 12586 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:18.605813  7658 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:18.604169  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:18.608350 12465 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:18.550189 12467 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:18.608793 12465 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:18.609486 12465 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:18.603624 12562 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.486s	user 0.000s	sys 0.002s
I20260709 12:27:18.609385 12467 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:18.611119 12467 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:18.606923  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.616515  7658 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:18.617247 12540 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:18.617518  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:18.618324 12478 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.854s	user 0.013s	sys 0.000s
I20260709 12:27:18.618821 12478 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:18.551571 12587 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.623893  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:18.624516 12536 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:18.625451  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:18.625972 12555 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:18.626400 12463 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:18.627576 12555 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:18.628360 12450 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:18.618813  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.629976  7658 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:18.648497  7662 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:18.474303 12460 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.604379  7660 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 8.
I20260709 12:27:18.654636  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:18.659143  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:18.500188 12558 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:18.664058  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115703us
W20260709 12:27:18.589092 12500 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.894s	user 0.018s	sys 0.000s
W20260709 12:27:18.665956  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121339us
I20260709 12:27:18.537688 12556 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:18.646487  7655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.019s	sys 0.016s
I20260709 12:27:18.549197 12525 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:18.680579  7656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.001s	sys 0.027s
W20260709 12:27:18.702370  7656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.002s	sys 0.027s
I20260709 12:27:18.687745  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:18.663640 12573 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:27:18.690083  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:18.705855  7662 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:18.684048 12560 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.728s	user 0.016s	sys 0.000s
W20260709 12:27:18.690634  7655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.020s	sys 0.016s
I20260709 12:27:18.648481 12337 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.478683 12553 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:18.702952  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:18.697623  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.694160  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:18.709852  7658 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:18.710541  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:18.710736  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:18.722734  8412 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
W20260709 12:27:18.726522  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178811us
I20260709 12:27:18.717643  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:18.727730  7660 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:18.727888  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:18.726943  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:18.702250 12478 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:27:18.729671  8406 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
W20260709 12:27:18.709581 12543 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
I20260709 12:27:18.709357 12536 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:18.752884  7662 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:18.734689  8406 pending_rounds.cc:85] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 8
I20260709 12:27:18.753851  8406 pending_rounds.cc:107] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:27:18.754246  8406 raft_consensus.cc:2889] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:18.730438  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.537914 12557 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.731352  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:18.711504 12556 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:18.701638 12587 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:18.768467  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:18.764531  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:18.777714  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:18.729458 12337 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.780984  7811 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:18.786419  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:18.745808 12555 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:18.709435 12525 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.790109  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:18.796150  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:18.752137 12536 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.799749  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:18.722107 12560 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.849823  7808 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:18.774560 12530 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:18.775178 12469 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:18.781121 12501 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:18.858691  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61147.1us
I20260709 12:27:18.790990 12555 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.839207 12550 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.795317 11863 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:18.845518 12463 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:18.795846 12399 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:27:18.847306 12536 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:18.922109  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 157652us
I20260709 12:27:18.754540 12543 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.922133 12555 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:18.920163 12571 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 12:27:18.857218 11840 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:18.861611 12530 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:19.007507 12530 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:19.008790 12543 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:19.009847 12404 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.235s	user 0.014s	sys 0.000s
I20260709 12:27:18.932829  7662 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:18.987093  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:19.005065 12463 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:19.011318 12592 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:19.011605 12553 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.011832 12590 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:18.987980 12465 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.014123 12525 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.015658 12460 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:19.016971 12569 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:27:19.066026 12575 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:19.066979 12575 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.067737 12458 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.067548 12553 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:19.068425 12458 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:19.069535  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.952740 12588 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.314s
I20260709 12:27:19.070830 12458 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:18.921934 12320 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.976s	user 0.014s	sys 0.000s
I20260709 12:27:19.071684 12320 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.027397  7662 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:27:18.948033 12500 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.074651  7732 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:19.019714 12585 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:27:19.075158 12525 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:19.075762 12525 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:19.011020 12598 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:27:18.942932 11863 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:19.077333 11863 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:19.077425  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 863) took 1199 ms. Trace:
I20260709 12:27:19.077009 12525 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.077839  7653 rpcz_store.cc:276] 0709 12:27:17.877731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.878006 (+   275us) service_pool.cc:224] Handling call
0709 12:27:17.911996 (+ 33990us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.934856 (+ 22860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.934888 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.946253 (+ 11365us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.205322 (+259069us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:18.205508 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.205640 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.077171 (+871531us) raft_consensus.cc:1685] finished
0709 12:27:19.077196 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.077356 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99177,"raft.queue_time_us":23955,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":30095,"thread_start_us":83881,"threads_started":1,"wal-append.queue_time_us":716643}
I20260709 12:27:19.080916  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 859) took 1236 ms. Trace:
I20260709 12:27:19.081467  7655 rpcz_store.cc:276] 0709 12:27:17.844195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.844452 (+   257us) service_pool.cc:224] Handling call
0709 12:27:17.870955 (+ 26503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.935046 (+ 64091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.935095 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.935824 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.718358 (+782534us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:18.718423 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.718492 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.080504 (+362012us) raft_consensus.cc:1685] finished
0709 12:27:19.080535 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.080834 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39535,"raft.queue_time_us":45375,"spinlock_wait_cycles":21506432,"thread_start_us":777178,"threads_started":1,"wal-append.queue_time_us":784811}
I20260709 12:27:19.083678  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 958) took 1276 ms. Trace:
W20260709 12:27:19.084549 12580 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.311s	user 0.002s	sys 0.001s
I20260709 12:27:19.084290  7736 rpcz_store.cc:276] 0709 12:27:17.807121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.807330 (+   209us) service_pool.cc:224] Handling call
0709 12:27:17.807714 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.808163 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.808193 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.808715 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.974851 (+166136us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:17.974896 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.974974 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.066960 (+1091986us) raft_consensus.cc:1685] finished
0709 12:27:19.067001 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.083585 (+ 16584us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":165369,"threads_started":1,"wal-append.queue_time_us":936338}
I20260709 12:27:19.087671 12465 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:18.952447 12550 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.957958 12504 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.091780 12504 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:18.448014  7654 rpcz_store.cc:276] 0709 12:27:17.416690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.416831 (+   141us) service_pool.cc:224] Handling call
0709 12:27:17.423906 (+  7075us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:17.638924 (+215018us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:17.913462 (+274538us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.327405 (+413943us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:18.377865 (+ 50460us) log.cc:844] Serialized 52 byte log entry
0709 12:27:18.378269 (+   404us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:18.379487 (+  1218us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.379845 (+   358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.379956 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.421789 (+ 41833us) raft_consensus.cc:1685] finished
0709 12:27:18.421810 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.421977 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":304267,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":125,"raft.queue_time_us":295080,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":15612,"thread_start_us":714447,"threads_started":4,"wal-append.queue_time_us":274809}
I20260709 12:27:19.074193 12500 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.097802 12550 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:19.098167  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.099833 12586 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.100402 12586 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:19.103734 12611 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:19.104516 12314 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:19.105450  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.108045  7654 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:27:19.112922 12178 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:19.113479 12178 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:18.996856 12399 consensus_queue.cc:1243] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:19.116165 12504 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:19.014551 12467 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:19.117281  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 767) took 1504 ms (client timeout 1783 ms). Trace:
W20260709 12:27:19.117704  7804 rpcz_store.cc:269] 0709 12:27:17.612523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.612687 (+   164us) service_pool.cc:224] Handling call
0709 12:27:19.117018 (+1504331us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:19.117175 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013748992}
I20260709 12:27:19.118852 12592 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.119427 12467 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:19.121935 12592 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:18.998606 12469 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.102934  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:19.125416 12469 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.126194 12469 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:19.070077  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.127408 12463 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:19.128679 12463 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.132381 12560 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:18.830763 12556 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.134114 12320 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.059684  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.070401 12590 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:19.073343 12599 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:27:19.142593  8408 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:19.144940  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89859.5us
W20260709 12:27:19.176738  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 377048us
I20260709 12:27:19.178411  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:19.114671 12178 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.168596 12314 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.183377  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.182703 12610 raft_consensus.cc:2764] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:19.194011 12600 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:27:19.195278 12601 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:19.195742 12601 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:19.196810 12601 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.198273 12337 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.190713 12320 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:27:19.190922 12404 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:27:19.216317  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 861) took 1352 ms. Trace:
I20260709 12:27:19.217027  7656 rpcz_store.cc:276] 0709 12:27:17.863812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.863984 (+   172us) service_pool.cc:224] Handling call
0709 12:27:17.864552 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.899827 (+ 35275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.899884 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.946894 (+ 47010us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.703035 (+756141us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:18.703087 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.703177 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.215931 (+512754us) raft_consensus.cc:1685] finished
0709 12:27:19.215983 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.216219 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106158,"raft.queue_time_us":45935,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6103,"thread_start_us":652760,"threads_started":1,"wal-append.queue_time_us":810066}
I20260709 12:27:19.187309 12469 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:27:19.191995 12314 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.225623 12557 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.227460 12314 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:19.221835  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.227867  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.228245 12611 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:19.200654 12610 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:19.228638 12611 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:19.222434 12590 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.228668  8406 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:19.229827 12611 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.231420 12611 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.234875  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:19.235385  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.236243  8406 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:19.224645 12575 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.227823  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:19.248293  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:19.250600  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:19.271116  7734 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:19.268505  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:19.281582  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 98252.9us
I20260709 12:27:19.247622 12601 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.242166 12590 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.257778 12314 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:19.287247  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.259087 12386 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:19.265573 12404 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:19.316581  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.317354  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.315735  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.322335  7805 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
W20260709 12:27:19.303016  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.327404  7661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.014s	sys 0.038s
W20260709 12:27:19.328145  7661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.014s	sys 0.038s
I20260709 12:27:19.330722  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.337946  7804 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:19.342438  7734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2547) took 1291 ms (client timeout 1631 ms). Trace:
W20260709 12:27:19.343170  7734 rpcz_store.cc:269] 0709 12:27:18.050706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.054276 (+  3570us) service_pool.cc:224] Handling call
0709 12:27:19.342136 (+1287860us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:19.342345 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156155264}
I20260709 12:27:19.292153 12395 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:19.350134  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:19.354970  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.297392 12085 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:27:19.360240  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77373.9us
I20260709 12:27:19.362798  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.300477 12314 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:19.337764 12386 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.283488 12320 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:27:19.374100  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:19.392755  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.317198 12590 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.402472  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.403198  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.373198 12314 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.291798 12550 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.405795  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.406715 12615 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.407248 12556 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:19.408370 12610 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.409420  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.409826  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.410851  7807 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:19.451694  7655 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:19.402277 12463 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:19.402017 12085 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:19.419442 12590 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.425441  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2261) took 1012 ms. Trace:
I20260709 12:27:19.425522  7808 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:19.465358  7659 rpcz_store.cc:276] 0709 12:27:18.412867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.416008 (+  3141us) service_pool.cc:224] Handling call
0709 12:27:19.402401 (+986393us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b5400218ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:19.425368 (+ 22967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114619264}
I20260709 12:27:19.388516 12232 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:19.409428  7654 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:27:19.349222 12536 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.375648 12386 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.440099  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.442677  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:19.470917  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:19.444108 12617 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:27:19.452490  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49396.2us
I20260709 12:27:19.388249 12530 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.466327  7662 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:19.481678  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.469161  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.443454 12619 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:19.460211 12558 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.004s
I20260709 12:27:19.476257  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:19.495524  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.496620  7655 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:19.476402  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:19.486773  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51407.4us
I20260709 12:27:19.468585 12357 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:19.495249  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.510656  7653 raft_consensus.cc:2368] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:27:19.460345 12525 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.515563 12582 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.516338 12582 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.461620 12615 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.516244 12558 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.528038 11840 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:19.530230 11840 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:27:19.531654  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.532430  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.480861 11932 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.626s	user 0.013s	sys 0.009s
I20260709 12:27:19.549718  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:19.550216  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.550819 12500 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:19.511541 12537 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.962s	user 0.006s	sys 0.008s
I20260709 12:27:19.531656  7803 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:19.553597  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.555394 12530 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:19.555544 12536 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.479028 12618 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.556891 12615 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.524138  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.559779 12545 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.560214  7810 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:19.527707  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.561430 12550 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.560808 12545 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.562424  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:19.563171  7737 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 7.
I20260709 12:27:19.564435  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:19.565092  7730 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.538008 12073 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:19.567521 12621 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:19.568439 12621 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:19.569723 12621 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.484602 12540 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.556890 12178 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:19.573575 12540 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:19.536764  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60413.5us
I20260709 12:27:19.543678  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:19.545284  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.539183 12582 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.547439 12255 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.588754  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:19.548694  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.589423  8415 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.550406  7728 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.588816 12399 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:19.589512 12085 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.509718 12619 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.593734 12619 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.595281 12525 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.550892 12587 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.600734  7657 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:19.601508  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:19.515602  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.562598  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.603292  7660 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:19.603657  7808 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:19.605522  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.532430  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.606349  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:19.538506  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2550) took 1319 ms (client timeout 1753 ms). Trace:
I20260709 12:27:19.546573  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.607659  7812 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:19.530665 12558 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.575737  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:19.576424  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.554863  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.561316  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:19.613667  7810 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.485198 12395 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.621076 11840 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:19.621688 11840 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:19.622498 11840 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:19.586071  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.587383 12320 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:19.469555 12610 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.548785  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:19.595883  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.589563  7654 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:19.547211  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:19.597466  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.609372  7808 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:19.606611  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:19.602353  7735 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
W20260709 12:27:19.607514  7732 rpcz_store.cc:269] 0709 12:27:18.219115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.219369 (+   254us) service_pool.cc:224] Handling call
0709 12:27:19.538278 (+1318909us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54001c43d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:19.538441 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721449344}
I20260709 12:27:19.611754 12533 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:19.618815  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:19.619609  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:19.651582  7731 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:27:19.620761  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:19.652706  7803 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:19.628073 12536 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.656944 12536 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.657630  7811 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:19.662185 12610 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:19.665498 12386 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:19.667704  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89759.7us
I20260709 12:27:19.668700  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:19.635006 12537 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:27:19.672165 12533 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:19.672945  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.676024 12399 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:19.676687 12399 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:19.681648  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.628145  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.472368 12556 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.632407  7733 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:19.685349  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:19.634269 12395 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.686287  7807 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.643963  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:19.687582  7656 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:19.649483  7812 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:19.685527  7811 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:19.701887  7728 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:19.624174  7730 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
W20260709 12:27:19.721808  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.729251  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.730613  7660 raft_consensus.cc:2368] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:19.737629 12540 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.739315  8380 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:19.740300  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.740795  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:19.637691 12320 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.742025  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:19.742949 12466 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:19.597147 11863 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.613361 12587 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.743443 12466 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.744063 12466 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:19.744632 12533 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.744992 11863 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.745117 12603 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:19.745888 12603 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:19.747576  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:19.749305  7729 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:19.745249 12533 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:27:19.751482  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 132755us
I20260709 12:27:19.751677  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:19.758502 12575 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:19.759117 12575 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.759559 12575 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.759989 12395 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.750444 12073 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.762100  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.762725 12595 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:19.761782 12073 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.763415 12595 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:19.750598 12590 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.764446 12595 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.765269 12590 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.765894 12073 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.767848  7808 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:19.758533 12530 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:19.769631  7808 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:27:19.769809 12530 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:19.770412 12612 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:19.771103 12530 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.772986  7812 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 12:27:19.759618 12611 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.006s
I20260709 12:27:19.781994  7662 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:19.597004 12586 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.783264  7803 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:19.781502 12213 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:19.788682 12213 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:19.789646 12213 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.790521  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:19.797284  7807 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:19.743146 12320 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.806180  7807 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:19.751520 12610 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:19.787464 12555 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.004s
I20260709 12:27:19.762755 12357 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:19.752287 12533 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.833734 12478 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.834882 12478 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:19.848649  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.771165 12612 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:19.883306  7804 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:19.886994  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:19.811383 12610 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:19.816766 12320 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.838670 12478 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.898967  7730 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:27:19.895248  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:19.826958 12533 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:19.882272 12592 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:19.891026 12569 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:27:19.827114 12555 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.924229  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:19.897581 12610 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:19.902510 12629 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:19.926676  8379 leader_election.cc:400] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:19.932865  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:19.928812  7778 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.753s (SENT)
I20260709 12:27:19.937008  8379 leader_election.cc:403] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:19.937268  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:19.902734 12590 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:19.945044  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.945863  8406 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:19.946058  7656 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:19.947685  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:19.948177  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:19.917482 12232 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:19.951938  7809 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:19.804849 12603 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:19.953030  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:19.953907  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:19.939538 12386 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:19.903612 12320 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:19.961308  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.066s (SENT)
I20260709 12:27:19.962860  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:19.962703  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:19.987360  7775 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:19.949035 12465 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:19.983741  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.967044  8415 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:19.993868  7654 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:20.000154  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.001381  7728 raft_consensus.cc:2410] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:20.009508  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.010598  7733 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:19.960119 12537 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:19.967751 12558 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.939829 12592 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.056974  7731 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
W20260709 12:27:20.025661  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64401.7us
I20260709 12:27:20.036515 12612 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.038736  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:20.081986  8415 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:19.960889 12255 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.984337 12590 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:19.985590 12603 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:27:20.026962 12634 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 12:27:19.969921 12583 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:20.058892  7730 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:20.086251  7654 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:20.001959 12463 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:20.084501 12404 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.004s
W20260709 12:27:20.102147  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.092715  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:20.103276  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.095791  7656 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:19.983620 12611 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:27:20.113875  7654 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:20.121837  7731 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:20.079819 12556 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:20.199864  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.536s (SENT) [suppressed 20 similar messages]
W20260709 12:27:20.109692 12598 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:27:20.213878  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 287433us
W20260709 12:27:20.116405 12631 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 12:27:20.080974 12537 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:20.222743  7808 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:20.155562  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.088222 12638 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:20.126492 12586 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:20.198258 12404 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:20.105433 12255 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:20.230616  7810 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.001s
I20260709 12:27:20.235643  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.236274  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:20.237797  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:20.239950 12635 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:27:20.239655 12537 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:20.241402 12636 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:27:20.242846  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2262) took 1829 ms. Trace:
I20260709 12:27:20.243611  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 776) took 1475 ms. Trace:
I20260709 12:27:20.244122  7806 rpcz_store.cc:276] 0709 12:27:18.767978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.781511 (+ 13533us) service_pool.cc:224] Handling call
0709 12:27:18.844426 (+ 62915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.296482 (+452056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.296524 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.297157 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.719111 (+421954us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:19.719184 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.719343 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.243333 (+523990us) raft_consensus.cc:1685] finished
0709 12:27:20.243358 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.243537 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2495,"raft.queue_time_us":230209,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"thread_start_us":768626,"threads_started":2,"wal-append.queue_time_us":533490}
I20260709 12:27:20.246294  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:20.247167  7806 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.121732 12463 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:20.228689 11840 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.603s	user 0.015s	sys 0.000s
I20260709 12:27:20.243330  7661 rpcz_store.cc:276] 0709 12:27:18.413294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.695504 (+282210us) service_pool.cc:224] Handling call
0709 12:27:18.714951 (+ 19447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.335615 (+620664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.335678 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.369880 (+ 34202us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.781857 (+411977us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:19.949572 (+167715us) log.cc:844] Serialized 14 byte log entry
0709 12:27:19.963269 (+ 13697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.967615 (+  4346us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.242566 (+274951us) raft_consensus.cc:1685] finished
0709 12:27:20.242594 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.242768 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50980,"prepare.queue_time_us":28576,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":102953,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":48874,"thread_start_us":1028860,"threads_started":3,"wal-append.queue_time_us":485196}
I20260709 12:27:20.260987  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:20.261942  7810 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:20.255107 12466 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
W20260709 12:27:20.281240 12642 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:27:20.234906 12638 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.296994  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.123165 12583 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:20.262718 12463 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.305490  7806 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
W20260709 12:27:20.306385  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2267) took 1884 ms (client timeout 1548 ms). Trace:
W20260709 12:27:20.308336  7657 rpcz_store.cc:269] 0709 12:27:18.415139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.468219 (+ 53080us) service_pool.cc:224] Handling call
0709 12:27:20.298994 (+1830775us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:20.299115 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397569280}
I20260709 12:27:20.309980  7657 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:20.307780  8406 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:20.307406  7733 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:20.309890  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:20.316496  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:20.317332  7806 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:20.317864  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:20.318820  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:20.320447  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:20.321545  7810 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:20.321153  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98976.9us
W20260709 12:27:20.314095 12575 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
W20260709 12:27:20.308439 12314 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.859s	user 0.015s	sys 0.000s
I20260709 12:27:20.324721  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.327441  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:20.329421  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.304018 12555 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:20.268235 11840 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:20.275697 12466 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.336499  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:20.347756 12638 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:20.329247 12640 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:27:20.334970 12545 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.355460 12488 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:20.382925  7733 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:20.377504  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.377519  7732 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:20.335740 12320 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:20.365545 12178 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.004s
I20260709 12:27:20.335772 12314 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.333678 12575 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:20.391777  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 191786us
I20260709 12:27:20.389902 12594 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:20.407601 12255 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:20.407831  7730 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:20.409132  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.410035  7736 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:20.412300  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.414616  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.424612  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.428289  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2280) took 1826 ms. Trace:
I20260709 12:27:20.432022  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.433148  7731 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.407790 12594 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:20.429145  7658 rpcz_store.cc:276] 0709 12:27:18.601883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.074087 (+472204us) service_pool.cc:224] Handling call
0709 12:27:19.074559 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.140023 (+ 65464us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:19.343839 (+203816us) log.cc:844] Serialized 14 byte log entry
0709 12:27:19.629586 (+285747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:19.629658 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:19.629710 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.427903 (+798193us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b540002fb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:20.427967 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.428195 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50859,"prepare.queue_time_us":340284,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":17999,"raft.queue_time_us":34733,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":123,"spinlock_wait_cycles":149684608,"thread_start_us":449085,"threads_started":2,"wal-append.queue_time_us":461443}
I20260709 12:27:20.397389 12610 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:20.441128  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 75698.2us
I20260709 12:27:20.446441  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.437892 12592 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.454082  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:20.439627 12533 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.518s	user 0.019s	sys 0.008s
I20260709 12:27:20.458202  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:20.463379  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.460500 12533 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.471553  8413 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:20.480400 12314 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.481889 12466 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:20.484002 12466 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.485605 12594 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.489904 12575 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:20.495548 12533 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:20.509308 11840 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:20.513696 12618 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.957s	user 0.009s	sys 0.006s
I20260709 12:27:20.514302 12618 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:20.515143  7805 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:20.542529  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.543367  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.543707  7733 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:20.544533  7730 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:27:20.543346  7736 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:20.549479 12610 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.550065 12466 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:20.565745  7736 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:20.711007  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 143905us
I20260709 12:27:20.720665  7731 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:20.723315  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.724395  7731 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:20.741285  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.742725  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
W20260709 12:27:20.720860 12399 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.044s	user 0.014s	sys 0.000s
I20260709 12:27:20.771977  7736 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:27:20.772753  7736 pending_rounds.cc:85] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:27:20.773063  7736 pending_rounds.cc:107] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:27:20.773386  7736 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.500825 12463 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.790771  7731 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:20.791478  7731 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:20.794003  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 339906us
I20260709 12:27:20.792518  7731 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:20.753634 12399 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:20.808298  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.796051  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:20.834137  7731 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:20.836023  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.837152  7806 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:20.835577 12663 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:27:20.833060 12558 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.756s	user 0.013s	sys 0.002s
I20260709 12:27:20.839447  7732 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:20.842149  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.844981  7730 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:20.848554  7811 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:20.849924  7731 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:27:20.850535  7731 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:20.850852  7731 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:20.851182  7731 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.851990  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:20.851140  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:20.859977  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:20.860661  7810 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:20.861053  7810 raft_consensus.cc:614] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:20.864207  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.864616 12533 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:20.865260  7731 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:27:20.868440 12603 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.759s	user 0.011s	sys 0.003s
I20260709 12:27:20.885677  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.886059  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.861439 12558 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:20.886756  7730 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:20.886749  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.887859  7733 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:20.887795  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.888976  7736 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:20.888195  7735 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:20.860213 12663 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:20.891814  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.894781 12587 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:20.892961  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2284) took 1487 ms. Trace:
I20260709 12:27:20.895845  7653 rpcz_store.cc:276] 0709 12:27:19.405594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.405770 (+   176us) service_pool.cc:224] Handling call
0709 12:27:20.892739 (+1486969us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b54003994d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:20.892888 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135770496}
I20260709 12:27:20.894589  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.899469  7733 raft_consensus.cc:2376] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12. Re-sending same reply.
I20260709 12:27:20.900100  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:20.899482  7732 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:20.901127  7731 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:20.905557 12399 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:20.917375  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2380) took 1496 ms. Trace:
I20260709 12:27:20.917927  7809 rpcz_store.cc:276] 0709 12:27:19.420946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.551297 (+130351us) service_pool.cc:224] Handling call
0709 12:27:19.551640 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.954378 (+402738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.917023 (+962645us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540042a754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:20.917287 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920106752}
I20260709 12:27:20.934900  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:20.935674  7733 raft_consensus.cc:2376] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13. Re-sending same reply.
I20260709 12:27:20.941469  8406 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:20.943445  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:20.940424 12463 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.945677 12663 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:20.947990 12587 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:20.946805  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:20.950045 12587 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:20.950515 12587 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:20.950911 12587 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.957500  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2388) took 1534 ms. Trace:
I20260709 12:27:20.958027  7804 rpcz_store.cc:276] 0709 12:27:19.423214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.761052 (+337838us) service_pool.cc:224] Handling call
0709 12:27:19.761425 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.891414 (+129989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.957168 (+1065754us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:20.957426 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266978560}
I20260709 12:27:20.963270  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:20.973719 12525 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:20.975437 12320 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:20.971895 12603 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:20.977156 12603 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.977650 12603 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:20.972103  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:20.977839 12583 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:20.971525 12073 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:20.992512  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.995157  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1095) took 1038 ms. Trace:
I20260709 12:27:20.996011  8413 rpcz_store.cc:276] 0709 12:27:19.956414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.956735 (+   321us) service_pool.cc:224] Handling call
0709 12:27:19.980115 (+ 23380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.472161 (+492046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.994927 (+522766us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:20.995092 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129321216}
W20260709 12:27:20.999761  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.005926  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.007628  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.010609  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.012739  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.013856  8412 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:20.991524 12590 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:21.020895  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 876) took 1303 ms. Trace:
I20260709 12:27:21.027004  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:21.027257  8413 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:21.028384  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.025192  7659 rpcz_store.cc:276] 0709 12:27:19.717524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.214700 (+497176us) service_pool.cc:224] Handling call
0709 12:27:20.233946 (+ 19246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.234201 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:20.306798 (+ 72597us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.020320 (+713522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:21.020409 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:21.020498 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.020615 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.020764 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":608871,"prepare.queue_time_us":74845,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"thread_start_us":122191,"threads_started":1,"wal-append.queue_time_us":184882}
I20260709 12:27:21.033283  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 877) took 1109 ms. Trace:
I20260709 12:27:21.033782  7657 rpcz_store.cc:276] 0709 12:27:19.924396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.312082 (+387686us) service_pool.cc:224] Handling call
0709 12:27:20.329996 (+ 17914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.330216 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:20.363148 (+ 32932us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.032935 (+669787us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:21.032992 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:21.033034 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.033095 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.033223 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":569092,"prepare.queue_time_us":33099,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":3622,"thread_start_us":93677,"threads_started":1,"wal-append.queue_time_us":376503}
I20260709 12:27:21.011970 12213 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:21.036463  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2548) took 2923 ms (client timeout 2066 ms). Trace:
I20260709 12:27:21.038409  7730 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:21.026538 12320 consensus_queue.cc:1243] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:21.037081 12213 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:21.038499  7729 rpcz_store.cc:269] 0709 12:27:18.113322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.115466 (+  2144us) service_pool.cc:224] Handling call
0709 12:27:21.036221 (+2920755us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54002490d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:21.036357 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3749154688}
I20260709 12:27:21.018055 12604 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.016899 12656 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.064309  7730 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:20.998588 12587 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.066382  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.076952  7729 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:21.071761  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.090242  7735 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.073477  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.106379  7729 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:21.110648  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2288) took 1416 ms. Trace:
I20260709 12:27:21.112250  7656 rpcz_store.cc:276] 0709 12:27:19.694496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.127445 (+432949us) service_pool.cc:224] Handling call
0709 12:27:20.198916 (+ 71471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.000527 (+801611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.000582 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.010502 (+  9920us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.109908 (+ 99406us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:21.109982 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.110208 (+   226us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.110291 (+    83us) raft_consensus.cc:1685] finished
0709 12:27:21.110311 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.110533 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":800936,"raft.queue_time_us":819701,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12260,"thread_start_us":83375,"threads_started":1,"wal-append.queue_time_us":57942}
I20260709 12:27:21.116216  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2386) took 1693 ms. Trace:
I20260709 12:27:21.116911  7805 rpcz_store.cc:276] 0709 12:27:19.422625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.737310 (+314685us) service_pool.cc:224] Handling call
0709 12:27:19.737693 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.515702 (+778009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.115794 (+600092us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:21.116117 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787306368}
I20260709 12:27:21.117677  7735 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:21.119339  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 779) took 1564 ms. Trace:
I20260709 12:27:21.119956  7808 rpcz_store.cc:276] 0709 12:27:19.554820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.828952 (+274132us) service_pool.cc:224] Handling call
0709 12:27:19.882975 (+ 54023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.227162 (+344187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.118922 (+891760us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b5400295754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:21.119223 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772245376}
I20260709 12:27:21.129523  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.136116  8406 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:27:21.131448  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 124003us
I20260709 12:27:21.144765  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2805) took 1671 ms. Trace:
I20260709 12:27:21.145359  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:21.135924  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 163924us
I20260709 12:27:21.147866  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:21.135473 12073 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.149647  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:21.149103 12073 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.038043 12659 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:21.151544 12659 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.136700 12656 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.153095 12073 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:21.153703 12659 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.123996 12604 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.145419  7812 rpcz_store.cc:276] 0709 12:27:19.473815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.779589 (+305774us) service_pool.cc:224] Handling call
0709 12:27:19.802511 (+ 22922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.802877 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.802906 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:20.937834 (+1134928us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:20.939713 (+  1879us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.969915 (+ 30202us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:20.970084 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.970187 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.144351 (+174164us) raft_consensus.cc:1685] finished
0709 12:27:21.144382 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.144680 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23435,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":35211,"spinlock_wait_cycles":37666688,"thread_start_us":27486,"threads_started":1,"wal-append.queue_time_us":67647}
I20260709 12:27:21.141846  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.094779 12536 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:21.072144 12320 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:21.138644 12545 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:21.160660  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.162814  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.161396  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:21.076121 12672 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.208312  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:21.216153  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53443us
I20260709 12:27:21.123579 12550 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:21.159152 12604 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:21.206574 12666 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
I20260709 12:27:21.218720  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:21.220808  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.221138  7736 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:21.231083  8408 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:21.172443 12673 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:21.172271 12533 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:27:21.227453 12450 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.221035 12469 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:21.227685 12545 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:21.281186  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 148595us
I20260709 12:27:21.256296  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2291) took 1535 ms. Trace:
I20260709 12:27:21.258423  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:21.242045  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:21.261492  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:21.279492  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70922.9us
W20260709 12:27:21.246467 12630 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.158s	user 0.002s	sys 0.001s
I20260709 12:27:21.260196  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.241686  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:21.284963  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 68412.9us
I20260709 12:27:21.291074  8406 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:21.286180  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68116.9us
I20260709 12:27:21.264628 12673 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:21.208365  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61231.9us
I20260709 12:27:21.273208 12671 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.282449  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.295310  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.295631  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:21.296046  7655 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:21.248845 12550 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.258626 12320 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.296908  7662 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:21.291324  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:21.297392 12320 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.280251 12469 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.299422 12469 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.253947 12656 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.294530  7657 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:27:21.300038  7729 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:27:21.300437  7729 pending_rounds.cc:85] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:21.282222  7661 rpcz_store.cc:276] 0709 12:27:19.720820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.261319 (+540499us) service_pool.cc:224] Handling call
0709 12:27:20.293876 (+ 32557us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:21.088068 (+794192us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:27:21.120086 (+ 32018us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.162294 (+ 42208us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:21.166793 (+  4499us) log.cc:844] Serialized 52 byte log entry
0709 12:27:21.167263 (+   470us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:21.167766 (+   503us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.250183 (+ 82417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.250264 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.255993 (+  5729us) raft_consensus.cc:1685] finished
0709 12:27:21.256017 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.256221 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":896586,"prepare.queue_time_us":32189,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":96,"raft.queue_time_us":800642,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":22604,"spinlock_wait_cycles":100813184,"thread_start_us":101002,"threads_started":3,"wal-append.queue_time_us":1433}
I20260709 12:27:21.300652  7729 pending_rounds.cc:107] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:21.300911  7729 raft_consensus.cc:2889] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.294554 12673 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:21.302670  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:21.304546  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.305213  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:21.306097  7729 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:21.287603  7656 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:21.307399  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:21.256223 12592 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.295006 12671 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.308471  7730 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:21.238075 12213 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.332571  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:21.310109  7658 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:21.307906  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47774.5us
I20260709 12:27:21.320247  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:21.335129  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 257689us
I20260709 12:27:21.338827  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.342957  8405 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:21.345088 12619 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:21.345803  7732 raft_consensus.cc:1217] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[6.5-12.8]   Dedup: 6.5->[8.6-12.8]
I20260709 12:27:21.346397  7732 pending_rounds.cc:85] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:21.346645  7732 pending_rounds.cc:107] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:21.346905  7732 raft_consensus.cc:2889] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.335762 12536 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:27:21.348279 12536 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:21.348668 12536 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:21.349851 12536 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.311165 12073 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:21.351884 12073 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:21.352914 12073 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.345129  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.325824  8406 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:21.310837  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.328370  7730 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:21.365167  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.366212  7804 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:21.381181  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.382674  7656 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:21.309156 12530 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.368510  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.384915  7733 raft_consensus.cc:2368] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:27:21.259248 12590 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.310606 12592 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.392807  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:21.285318 12603 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.395073  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.334306 12671 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:21.270728 12450 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.351094 12536 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:21.327649 12674 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:21.408638  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:21.408882  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:21.426800  7812 raft_consensus.cc:606] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:21.427400  7812 raft_consensus.cc:614] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:21.414570  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.428758  7733 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:21.346830 12619 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:27:21.430415  7735 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:21.435688  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.335381 12556 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:27:21.354067 12525 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:27:21.399001 12320 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:21.407330 12583 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.439077  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:21.439682  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101888us
I20260709 12:27:21.468693  7653 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:21.406513 12592 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:21.487195  7777 leader_election.cc:400] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:27:21.487637  7777 leader_election.cc:403] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.380940 12399 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:27:21.420583 12595 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:21.428570 12681 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.417783 12232 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:27:21.397675 12530 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:21.514026  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 211440us
I20260709 12:27:21.489030 12178 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:21.478513 12583 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.515606  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.519829  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:21.532442  7729 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:21.491318 12320 raft_consensus.cc:2764] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:27:21.504882 12681 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.534421  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:21.536710 12583 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:21.481635 12684 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:27:21.537626 12178 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:21.521500 12587 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.422328 12545 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:21.593278  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59059.2us
I20260709 12:27:21.556844  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.510553 12558 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:21.531080  7653 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:21.515609 12595 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:21.619097  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 240323us
W20260709 12:27:21.621837  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.580727  7736 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
W20260709 12:27:21.553658 12677 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:27:21.626011 12685 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:27:21.588703  7729 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:21.538987 12399 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:21.630677  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691f0d885861432f91375d5f288af312"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:21.631930  7804 raft_consensus.cc:606] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:21.632388  7804 raft_consensus.cc:614] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:21.600872 12178 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.562685 12525 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:27:21.221966 12672 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.611059 12558 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.614243 12670 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:21.615530 12595 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.624615  7733 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:21.438210 12674 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.628070  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2808) took 2101 ms. Trace:
I20260709 12:27:21.540607 12681 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:21.588248 12463 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:21.628643 12619 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:21.675701  7803 rpcz_store.cc:276] 0709 12:27:19.527014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.787037 (+260023us) service_pool.cc:224] Handling call
0709 12:27:19.787491 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.802758 (+ 15267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.802789 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.099860 (+1297071us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b1800458d40. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:27:21.100764 (+   904us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:21.101525 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.160407 (+ 58882us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:21.160448 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.161293 (+   845us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.627839 (+466546us) raft_consensus.cc:1685] finished
0709 12:27:21.627862 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.628003 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13815,"raft.queue_time_us":14822,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9044,"spinlock_wait_cycles":43959552,"thread_start_us":58389,"threads_started":1,"wal-append.queue_time_us":210177}
W20260709 12:27:21.685992  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.698331  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217816us
I20260709 12:27:21.630632 12399 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:21.685734  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 205124us
I20260709 12:27:21.686936  7737 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:27:21.702472  7737 pending_rounds.cc:85] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:21.694737 12671 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.660223 12670 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:21.710723  7731 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:21.708833  7737 pending_rounds.cc:107] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:21.713004  7731 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:21.713173  7737 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.714006  7731 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.716394 12399 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.717875 12399 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:21.719758  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.722084 12674 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:21.724705 12558 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.671844 12699 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:21.728932 12699 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.729434 12699 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:21.687948 12587 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.717955 12672 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.696956 12320 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:21.705757 12463 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.760607  7731 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:21.712627 12232 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.749852  7728 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:21.716499 12686 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:27:21.711473 12603 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.740669 12587 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.775168  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.736969  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.705415 12619 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.752547 12320 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:21.776365 12530 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:21.799127 12671 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:21.806142  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.809805  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98157.4us
I20260709 12:27:21.784030 12587 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:21.827782  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 233602us
I20260709 12:27:21.830571  7731 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:27:21.832159  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.835688 12698 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:27:21.838308 12619 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:21.827956 12621 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:21.839377 12558 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:21.829838 12688 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:21.774490 12232 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.845571  7731 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:21.845234 12232 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.843423 12663 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.843573 12530 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:27:21.846117 12621 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.844604  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:21.867712  7657 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:21.853147  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:21.859998  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.864812  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.865052  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.869091  7656 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:21.869729  7731 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:21.871870  7731 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:21.872823  7731 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:21.873884  7731 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:21.839900 12320 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.877825 12320 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:21.881000  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:21.814520 12583 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:21.904392  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:21.907027  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.908164  7732 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:21.881563 12663 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.911672 12663 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.914077 12583 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.915577 12619 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:21.918761  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.927760  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:21.928226  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.932996  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.941051  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.882426 12621 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:21.951064  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98003.7us
I20260709 12:27:21.952481  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.953209  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:21.959787  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:21.924018 12698 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:21.843118 12688 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.936939 12663 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:21.965543  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.967511  7810 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:21.966883  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.966806  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.979648  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.980399  7806 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:21.981117  7803 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:21.948453 12657 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:21.981523  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.983832  7804 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:21.980569  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.984797  7809 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:21.974313  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:21.985651  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.975306  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:21.983924  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:21.986397  7806 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:21.987147  7807 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:21.960645 12687 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:21.961616  7734 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:21.989439  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:21.986572  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:21.992376  7810 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:21.990695  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:21.993762  7806 raft_consensus.cc:606] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:21.985600 12657 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.956575 12699 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.004873  7806 raft_consensus.cc:614] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:21.968688 12705 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.999095  8406 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:21.986135 12469 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.005s
I20260709 12:27:21.971992 12650 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:22.037802 12650 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.040714 12650 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:22.043556 12550 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.046226  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:21.948328 12680 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:27:22.009268  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.000802 12558 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:22.015477 12692 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:22.049413 12692 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:22.049921 12692 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:21.980422 11863 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:27:22.019696  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:22.020762 12657 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:22.052167  8413 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:27:22.047870  7732 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:22.053421 12450 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.055840  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:22.002578 12698 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:21.991393 12687 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.973588 12693 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:27:21.980497 12556 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:22.023583 12705 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.031853 12699 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:21.994187 12664 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:22.036128  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:22.022243 12681 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:21.988931 12663 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:22.054760 12469 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.007632  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:22.083189  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:22.102888  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 127677us
W20260709 12:27:22.053658  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2878295649998108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:27:22.083016 12687 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:22.096381 12710 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:22.128762 12681 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.096769 12705 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.097409  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:22.098866 12621 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:22.088809 12708 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.003s
I20260709 12:27:22.143787  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:22.145336 12673 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.843s	user 0.013s	sys 0.000s
I20260709 12:27:22.146095 12673 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:22.092336 12711 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.004s
I20260709 12:27:22.106590  8413 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:22.107187  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2707295649997832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 12:27:22.177376  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1546295649995955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
W20260709 12:27:22.177757  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0133295649998217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 12:27:22.095000 12690 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:22.097515 12592 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.597s	user 0.013s	sys 0.002s
I20260709 12:27:22.101779 12463 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:22.114459 12536 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.689s	user 0.008s	sys 0.006s
I20260709 12:27:22.050843 12692 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.100339 12698 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:22.121031 12688 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.203310  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 990) took 1137 ms. Trace:
I20260709 12:27:22.126492 12469 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.127409  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:22.207525  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 80197us
I20260709 12:27:22.208730 12664 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:22.209214 12664 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.209582 12664 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.134272  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:22.135438 12663 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.139386 12621 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:22.133513 12587 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.088027  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:22.149082 12687 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.152024  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.152987 12670 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.161479  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:22.163280  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.253842  7808 raft_consensus.cc:606] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.155665 12657 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.165393  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:22.253134  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.166291  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.167049  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.255141  7735 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:22.168016  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:22.256783  7658 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:27:22.173928  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.178746  7804 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:27:22.178746  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 479299us
I20260709 12:27:22.181483  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:22.184185  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.184381  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:22.264626  7806 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:22.184864  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:22.266315  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 81531.6us
I20260709 12:27:22.186990  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.188014  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:22.217291  7656 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:27:22.219751  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.274608  7661 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:27:22.277400  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.281276  7733 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:22.281481  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.190436  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.194612 12717 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 12:27:22.289901  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102957us
I20260709 12:27:22.290908  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.196527  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:22.200114 12698 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.220655  7659 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:22.223889  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.207944 12692 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:22.182022 12589 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.294662  7653 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:22.211933  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:22.204304  7736 rpcz_store.cc:276] 0709 12:27:21.066337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.073128 (+  6791us) service_pool.cc:224] Handling call
0709 12:27:22.203009 (+1129881us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:22.203188 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288077696}
I20260709 12:27:22.211031  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.283108  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.296027  8413 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:22.283887  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93539.9us
I20260709 12:27:22.189522  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:22.286019  7658 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:22.262569  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.252678 11863 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.255731  7730 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:22.301862  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.289181  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.297567  7810 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.302544  7657 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:22.305924  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:22.307153  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.297899  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.298661 12670 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:22.300081  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89173.6us
I20260709 12:27:22.300649  7729 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:22.311640  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.309379  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:22.308741  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.312733  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:22.316170  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.316573  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.317345  7656 raft_consensus.cc:2368] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15.
I20260709 12:27:22.319263  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2290) took 2597 ms. Trace:
I20260709 12:27:22.181028 12690 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:22.319936  8412 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:27:22.319705  7654 rpcz_store.cc:276] 0709 12:27:19.706633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.207885 (+501252us) service_pool.cc:224] Handling call
0709 12:27:20.208318 (+   433us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:27:21.125268 (+916950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.125318 (+    50us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:27:21.126428 (+  1110us) log.cc:844] Serialized 104 byte log entry
0709 12:27:21.280694 (+154266us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b1800390a00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:21.471032 (+190338us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:21.537592 (+ 66560us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.537980 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.116007 (+578027us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b1800390a00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:22.169152 (+ 53145us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800390a00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:22.169626 (+   474us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.303699 (+134073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.303782 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:22.303824 (+    42us) raft_consensus.cc:1685] finished
0709 12:27:22.303836 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.303993 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":948609,"prepare.queue_time_us":148184,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":3215,"raft.queue_time_us":1547673,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":5078,"spinlock_wait_cycles":121958528,"thread_start_us":323786,"threads_started":2,"wal-append.queue_time_us":398286}
I20260709 12:27:22.327733  8406 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:27:22.328460  7655 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:22.336416  7806 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
W20260709 12:27:22.342679  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.344151  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:22.345093  8408 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.346619  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:22.194020 12592 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.344151  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.300354 12619 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:22.394269  7653 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:22.348090  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:22.194806 12556 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.132s
I20260709 12:27:22.311512 12693 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:22.394013  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44707.1us
I20260709 12:27:22.310103 12681 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:22.349248  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:22.319754 12469 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:22.406476  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:22.326890 12688 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:22.397070  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:22.319212 12680 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:22.300748 12399 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:22.299592 12615 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:22.402555  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58483.3us
I20260709 12:27:22.410269  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.417325  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.412886  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.340073  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.326663 12656 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:22.425750  7662 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:22.427007  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.427862  8408 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:22.433650  7658 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:22.418109  7655 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:22.440076  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100052us
I20260709 12:27:22.318725 12621 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 12:27:22.456368  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.319061 11863 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.320441 12589 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.341462 12463 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:27:22.415558 12619 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:22.488241  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:22.396397 12556 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:22.411682 12705 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:22.319181 12697 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.153s
W20260709 12:27:22.507916  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63201.7us
I20260709 12:27:22.508169  7653 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:22.509363  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.475972 12399 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:27:22.501053 12710 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:27:22.491080 12693 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:22.425931 12680 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.500360  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.492332 12537 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:22.528548  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130285us
I20260709 12:27:22.531481  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:22.441300 12450 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 12:27:22.482553  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55723.3us
I20260709 12:27:22.538218  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.503147 12717 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:27:22.550259  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.556257  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.558025 12680 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:22.417665 12594 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:27:22.568784  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.569876  7658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:22.571283  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.572389  7655 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:22.493927 12530 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:27:22.575069  8408 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:22.524547 12611 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:22.521433 12621 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:22.437798 12615 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.580431  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:22.596223  7654 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:22.587677  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49731us
I20260709 12:27:22.597487  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.491560 12656 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.536670 12674 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:22.553900 12697 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:22.524066 12545 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:27:22.524350 12469 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 12:27:22.608340  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70328.9us
I20260709 12:27:22.553539 12723 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:22.608901  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.481863 12690 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.635485  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:22.593422 12621 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:22.640699  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.641275  7655 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:22.607936 12673 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.650348  7662 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:22.611734 12687 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.639076 12611 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:22.661458  7662 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:22.680200  7654 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:22.660640  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:22.650571 12386 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:22.620251 12723 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:22.685096  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73820.9us
I20260709 12:27:22.622023 12469 consensus_queue.cc:1243] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:22.686324  7655 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:27:22.647673 12590 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:22.694396  8413 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:22.638386 12705 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:27:22.665009 12178 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:22.729059  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81284.6us
I20260709 12:27:22.657907 12314 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:22.676975 12657 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.654930 12688 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:27:22.680413 12695 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:22.741701 12589 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.657765 12655 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
W20260709 12:27:22.732378 12726 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:27:22.742583 12705 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:27:22.746248  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85691.4us
I20260709 12:27:22.746949 12314 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.748157 12611 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:22.749671 12583 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:22.750759 11863 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:22.750905 12692 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:22.752339 12656 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.753022 12592 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:22.688742 12723 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:22.731344 12590 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.711843 12687 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.760419 12550 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:22.763615 12587 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:22.766947  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:22.769714 12450 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:22.643414 12693 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:27:22.658555 12697 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:27:22.652371 12619 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.808483  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:22.803252  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 87067.6us
I20260709 12:27:22.804967  7777 leader_election.cc:400] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15.
I20260709 12:27:22.809725  7777 leader_election.cc:403] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:27:22.811478  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:22.787588 12692 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:22.877382  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63442.8us
I20260709 12:27:22.801304 12550 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:22.806438 12697 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:22.799757 12583 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:22.810901 11863 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:22.810097 12705 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:22.735260 12621 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:22.912380  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.917984  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:22.919106  7661 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:22.821565 12530 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15.
I20260709 12:27:22.880043 12737 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:22.923888  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53253.1us
I20260709 12:27:22.803970 12687 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:22.928846 12733 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:27:22.800386 12739 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
W20260709 12:27:22.955267  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.725s (SENT)
I20260709 12:27:22.923100 12693 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:22.943114  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129910us
W20260709 12:27:22.942559  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 134128us
I20260709 12:27:22.997393  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:22.949887 11863 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:27:22.946044 12692 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:22.950225 12737 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:23.038456  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 124994us
I20260709 12:27:23.023826  7655 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:27:23.025159  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.041198  7736 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:23.041909  7810 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:23.042241  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.025509  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.043015  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.025859  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.026726  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.043977  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.026842  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.044691  7729 raft_consensus.cc:2368] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 8. Current term is 9.
I20260709 12:27:23.021112  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:23.028609  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89622.2us
I20260709 12:27:23.045974  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.046028  7806 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:27:22.911109 12705 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:23.032641  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.043056  7736 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.044545  7734 raft_consensus.cc:2376] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10. Re-sending same reply.
I20260709 12:27:22.948493 12615 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:23.051935  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.042148  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.069743  7732 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11. Re-sending same reply.
I20260709 12:27:23.070974  7736 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:23.049901  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:23.031131 12693 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.077905  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.073678  7803 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:23.078675  7735 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:23.030030 12530 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:23.079798  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:23.080381  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 125187us
I20260709 12:27:23.080986  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.076794  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:23.081873  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.727s (SENT)
I20260709 12:27:23.083205  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.096510  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:23.111893  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.114447  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.114596  7729 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:27:23.115186  7737 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:23.116384  7729 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:27:23.116954  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.114684  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.118978  7728 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.119282  7735 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:23.083678  7736 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:27:23.120837  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.745s (SENT)
I20260709 12:27:23.121592  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.090576  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:23.113708  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.002s	sys 0.015s
I20260709 12:27:23.126832  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:23.127591  7803 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:23.128720  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.002s	sys 0.015s
W20260709 12:27:23.129120  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 642822us
I20260709 12:27:23.129871  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.131065  7730 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:23.049866 12739 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:23.086169 12719 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:23.085901 12693 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:23.076426 12450 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.036767 12728 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 12:27:23.136714  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.005s	sys 0.012s
W20260709 12:27:23.156356  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.006s	sys 0.012s
I20260709 12:27:23.139487  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:23.139097  7804 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:27:23.157696  7804 pending_rounds.cc:85] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 7
I20260709 12:27:23.085101 12530 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:23.138322 12664 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.923s	user 0.010s	sys 0.005s
I20260709 12:27:23.158013  7804 pending_rounds.cc:107] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:27:23.164606  7804 raft_consensus.cc:2889] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.164386  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.114144 12692 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:23.165062  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115249us
W20260709 12:27:23.138432 12525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.030s	sys 0.083s
W20260709 12:27:23.247891 12525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.030s	sys 0.083s
I20260709 12:27:23.217855 12664 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.136404 12739 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:23.128585 12386 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:27:23.161603  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.809s (SENT)
I20260709 12:27:23.249552 12739 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.250711  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.252310 12386 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:23.224982  7730 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:27:23.253928  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92386.1us
W20260709 12:27:23.254952  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.873s (SENT)
W20260709 12:27:23.255263 12743 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:27:23.255551  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:23.257247  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.899s (SENT)
W20260709 12:27:23.257308 12738 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:27:23.136075 12746 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
I20260709 12:27:23.259532  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:23.259873 12740 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:27:23.265061 12469 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:23.266000 12469 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:23.267681  7627 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.904s (SENT)
I20260709 12:27:23.269070 12705 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.270663 12705 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:23.270969  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.947s (SENT)
I20260709 12:27:23.273029  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.275259 12674 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:23.276882  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.993s (SENT)
I20260709 12:27:23.277493 12737 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.207924 12697 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.034125 12550 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.279335  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.280907 12550 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:23.284312  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.147s (SENT)
I20260709 12:27:23.285004  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.250973 12739 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:23.300382  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.300673  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.301522  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.304687  7736 raft_consensus.cc:2376] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10. Re-sending same reply.
I20260709 12:27:23.306071  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.306742  7803 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:23.307544  7736 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:23.135751 12719 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:23.309067  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.310590  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.310655  7737 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.311548  7728 raft_consensus.cc:2368] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:23.312386 12719 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:23.315774  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.077224 12583 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.332445 12073 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:23.250564 12664 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.226240 12314 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.385826  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.292243 12545 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:23.225404 12530 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:23.292447 12525 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.403365  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.404887  7730 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:23.333544 12723 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.005s
I20260709 12:27:23.211201 12450 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.333520 12073 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:23.441855  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:23.448879  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 319057us
I20260709 12:27:23.454182  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.455164  7729 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 8.
I20260709 12:27:23.456336  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:23.457345  7728 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.462296  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.462999  7729 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:23.463095  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.463908  7732 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:23.466653  7737 pending_rounds.cc:85] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:23.467125  7737 pending_rounds.cc:107] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:27:23.467412  7737 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.467573  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.468638  7729 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.279498 12697 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.443115 12314 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:23.483174  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:23.439756 12762 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:23.453510 12073 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:23.525861  7732 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.526772 12728 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:23.487488 12759 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:23.534288  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83052.8us
I20260709 12:27:23.541321  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:23.413265 12525 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.425793 12723 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:23.548785  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 307723us
I20260709 12:27:23.557408 12752 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:23.480612 12687 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.569550  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 894) took 1355 ms. Trace:
I20260709 12:27:23.558362 12752 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.570204  7660 rpcz_store.cc:276] 0709 12:27:22.214416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.224476 (+ 10060us) service_pool.cc:224] Handling call
0709 12:27:22.225003 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.225520 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.225595 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.273088 (+ 47493us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.157082 (+883994us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:23.157139 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.157221 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.569190 (+411969us) raft_consensus.cc:1685] finished
0709 12:27:23.569220 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.569431 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27994,"raft.queue_time_us":27104,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6584,"thread_start_us":748173,"threads_started":1,"wal-append.queue_time_us":1123423}
I20260709 12:27:23.507994 12746 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:27:23.508782 12652 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
W20260709 12:27:23.605602  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 321331us
I20260709 12:27:23.610235  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:23.611141  7807 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.615895  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:23.577253 12758 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:23.626828 12754 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:27:23.627743 12757 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:27:23.643937  7807 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:23.646704 12604 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:23.648499 12590 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.649011 12664 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:23.652210  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.653241  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.576349 12557 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:27:23.655395  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:23.730032 12697 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:23.730028  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:23.577940 12687 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.735544 12687 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.658474  7803 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:23.738070 12687 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:23.738698 12687 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:23.739507  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2311) took 1341 ms. Trace:
I20260709 12:27:23.739945 12687 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:23.658417  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.619127 12680 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.060s	user 0.017s	sys 0.000s
I20260709 12:27:23.641314 12450 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.742360 12680 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.743044  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:23.744789 12680 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:23.664069  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 176998us
I20260709 12:27:23.673507  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1107) took 1523 ms. Trace:
I20260709 12:27:23.747835  8406 rpcz_store.cc:276] 0709 12:27:22.134183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.134309 (+   126us) service_pool.cc:224] Handling call
0709 12:27:22.134633 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.330449 (+195816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.635549 (+1305100us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:23.657778 (+ 22229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443417344}
I20260709 12:27:23.748593  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:23.673439  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.626s (SENT)
W20260709 12:27:23.653527 12748 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.528s	user 0.002s	sys 0.000s
I20260709 12:27:23.587960 12619 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 12:27:23.752038  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 78655.3us
I20260709 12:27:23.754452 12450 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:27:23.755326 12450 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:27:23.756428 12657 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:23.756960 12657 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.757504 12657 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:23.758677 12687 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:23.759644 12687 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:23.744815  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:23.760401 12687 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:23.761920 12687 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.764420  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:23.740132  7653 rpcz_store.cc:276] 0709 12:27:22.398707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.400586 (+  1879us) service_pool.cc:224] Handling call
0709 12:27:22.435017 (+ 34431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.508662 (+ 73645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.739132 (+1230470us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b540033d6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:23.739417 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167382912}
I20260709 12:27:23.765671 12681 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:23.767259 12681 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:23.754613  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:23.772851 12656 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:23.773524 12656 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:23.774562 12656 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.780066  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.781368 12450 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:23.783684  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2317) took 1281 ms. Trace:
I20260709 12:27:23.784162  7662 rpcz_store.cc:276] 0709 12:27:22.502519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.685818 (+183299us) service_pool.cc:224] Handling call
0709 12:27:22.686178 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.817008 (+130830us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:22.817348 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.606620 (+789272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:23.606721 (+   101us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:23.606799 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.783344 (+176545us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540028d054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:23.783412 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.783607 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":283262,"prepare.queue_time_us":51679,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":318426,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":10221,"spinlock_wait_cycles":333055616,"thread_start_us":497668,"threads_started":2,"wal-append.queue_time_us":645883}
I20260709 12:27:23.788259 12655 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:23.797081 12655 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:23.801575 12690 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:23.805754  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.806181  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2829) took 1623 ms. Trace:
I20260709 12:27:23.806748  7659 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:23.807981  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.809029  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:23.809269  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.803413  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.809482  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:23.810333  7660 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:27:23.810369  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:23.806619  7809 rpcz_store.cc:276] 0709 12:27:22.182736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.184448 (+  1712us) service_pool.cc:224] Handling call
0709 12:27:23.805951 (+1621503us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540042a754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:23.806112 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2921579904}
I20260709 12:27:23.816998  7657 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 12:27:23.807843  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:23.826820 12674 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:23.827456 12674 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:23.827504 12537 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:23.828130 12674 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:23.828433 12537 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:23.829898 12537 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.832192  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:23.781251 12386 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:23.838914 12386 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:23.839579 12386 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:23.825693  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.848138 12450 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:23.852440  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:23.853482  7627 leader_election.cc:400] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:23.859774  7627 leader_election.cc:403] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:23.860543 12590 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:23.863026  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:23.864460  8406 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:23.866487  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:23.863133  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.855530 12674 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:23.868738 12590 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:23.774222 12681 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:23.874639  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:23.876829 12674 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:23.876775  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:23.878757  8405 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:23.879071  8406 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:23.881158 12681 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:23.882843 12681 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:23.883097 12590 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:23.657951 12587 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:23.888558  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:23.621286 12758 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:23.661988 12683 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:27:23.725739 12670 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.399s	user 0.020s	sys 0.000s
W20260709 12:27:23.924808  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 277917us
W20260709 12:27:23.926266  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 176118us
W20260709 12:27:23.904992  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.016s	sys 0.016s
I20260709 12:27:23.674984 12604 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:23.592815 12765 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.143s
W20260709 12:27:23.942014  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53547.1us
I20260709 12:27:24.052337  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:23.675159 12583 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:23.989040  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:23.989645  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.016s	sys 0.022s
I20260709 12:27:23.990607  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:24.083761  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 94800.4us
I20260709 12:27:24.085316  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:24.088277  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97784us
I20260709 12:27:24.091001  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:24.092782  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:23.901353 12587 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:24.101431  7812 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:23.924743 12764 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:27:23.902992 12759 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:23.993106  7733 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:23.923338 12761 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:27:24.000037 12619 consensus_queue.cc:1243] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:24.001410 12463 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:24.005167  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.030423 12550 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.000s
I20260709 12:27:24.032371 12723 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.050103 12525 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.072278  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:24.078610  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:24.080356  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:24.096179  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:24.175923  7803 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:23.922538 12683 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:24.113649  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:24.199592  7809 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:24.115886  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2315) took 1633 ms. Trace:
I20260709 12:27:24.200615  7654 rpcz_store.cc:276] 0709 12:27:22.482436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.682378 (+199942us) service_pool.cc:224] Handling call
0709 12:27:22.683005 (+   627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.707492 (+ 24487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.707552 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.738024 (+ 30472us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.136909 (+398885us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:23.136958 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.137047 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.115537 (+978490us) raft_consensus.cc:1685] finished
0709 12:27:24.115570 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.115794 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":207,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":22295,"thread_start_us":339638,"threads_started":1,"wal-append.queue_time_us":656942}
I20260709 12:27:24.119738  7812 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [12.10-14.11]
W20260709 12:27:24.121441  7808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.013s	sys 0.039s
W20260709 12:27:24.203516  7808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.014s	sys 0.039s
I20260709 12:27:24.085394 12469 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:24.108750 12652 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:24.213920  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2835) took 1173 ms. Trace:
I20260709 12:27:24.094657 12765 consensus_queue.cc:1243] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:27:24.098955 11863 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.109470 12178 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:24.104331 12674 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:24.125873 12545 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:24.109134 12466 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:24.185736  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:24.233821  7807 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:27:24.188805  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 108561us
I20260709 12:27:24.237051  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:24.238982  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:24.191690  7806 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:24.193300  7811 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:27:24.204581  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:24.145432 12750 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.109783 12583 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:24.245244  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.040s	sys 0.179s
W20260709 12:27:24.247292  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.041s	sys 0.179s
W20260709 12:27:24.250741  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.221264  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.670s (SENT)
I20260709 12:27:24.189904  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:24.219800 12652 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:24.214601  7808 rpcz_store.cc:276] 0709 12:27:23.040845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.053119 (+ 12274us) service_pool.cc:224] Handling call
0709 12:27:23.107930 (+ 54811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.108426 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:23.485395 (+376969us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.213398 (+728003us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.213478 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:24.213538 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.213645 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.213832 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9293,"prepare.queue_time_us":479480,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"thread_start_us":1052160,"threads_started":2}
I20260709 12:27:24.216785  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:24.218140  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:24.254011  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.207306 12469 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:24.210960 12723 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:24.258461 12545 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.305045  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:24.305869  7731 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:24.134938 12758 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.317493 12652 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.282595 12589 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:24.321563 12545 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:24.328321 12773 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
W20260709 12:27:24.319053 12657 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.000s
I20260709 12:27:24.358669  7731 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:24.260255 12750 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.294775 12674 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:24.440810 12752 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.332617 12583 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:24.350349 12769 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 12:27:24.367623 12767 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 12:27:24.368816  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.281759 12469 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:24.445948  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2844) took 1314 ms. Trace:
W20260709 12:27:24.450577 12770 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.527s	user 0.001s	sys 0.000s
W20260709 12:27:24.460088 12781 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:27:24.246416 11863 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.474872 11863 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.446727  7810 rpcz_store.cc:276] 0709 12:27:23.131133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.131307 (+   174us) service_pool.cc:224] Handling call
0709 12:27:23.131697 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.210330 (+ 78633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.210362 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.229566 (+ 19204us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.578392 (+348826us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:23.578434 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.578513 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.445563 (+867050us) raft_consensus.cc:1685] finished
0709 12:27:24.445599 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.445842 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93160,"raft.queue_time_us":277054,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":30817,"thread_start_us":319197,"threads_started":1,"wal-append.queue_time_us":629695}
W20260709 12:27:24.483352 12739 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.142s	user 0.010s	sys 0.004s
I20260709 12:27:24.377460 12657 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.607128  8409 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:24.215267 12587 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.609930 12587 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:24.519954 12758 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:24.521853 12693 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.228s	user 0.004s	sys 0.007s
I20260709 12:27:24.490192 12469 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.632953 12469 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:24.524075 12583 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.634229 12583 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.638258 12583 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:24.527335 12752 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.639621 12752 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.643159 12752 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:24.643699 12693 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:24.650027  7735 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:24.654788 12657 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:24.531575 12705 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.260s	user 0.011s	sys 0.004s
I20260709 12:27:24.580940 12756 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:24.664603 12756 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.595053  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2735) took 1134 ms. Trace:
I20260709 12:27:24.667513  7728 rpcz_store.cc:276] 0709 12:27:23.460926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.461089 (+   163us) service_pool.cc:224] Handling call
0709 12:27:23.485439 (+ 24350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.620280 (+134841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.620325 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.640443 (+ 20118us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.132658 (+492215us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:24.150762 (+ 18104us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.151001 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.151099 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.594690 (+443591us) raft_consensus.cc:1685] finished
0709 12:27:24.594724 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.594964 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3357,"prepare.queue_time_us":17968,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":7204,"raft.queue_time_us":59017,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":29913,"thread_start_us":625880,"threads_started":2,"wal-append.queue_time_us":845571}
W20260709 12:27:24.358239 12073 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.000s
I20260709 12:27:24.672298 12073 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.675778 12073 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.678272 12756 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:24.696136 12525 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:24.697105 11863 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:24.697842 12752 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:24.702594 12590 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.819s	user 0.013s	sys 0.000s
W20260709 12:27:24.706739  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.725131 12692 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.477s	user 0.014s	sys 0.000s
I20260709 12:27:24.739559 12693 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:24.309610 12450 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:24.354209  8413 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:27:24.781471  7737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.051s	sys 0.037s
W20260709 12:27:24.830711  7737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.054s	sys 0.037s
W20260709 12:27:24.808940  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.811108  7701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.032s	sys 0.054s
W20260709 12:27:24.813345  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.252s	user 0.024s	sys 0.116s
W20260709 12:27:24.824512  7732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.030s	sys 0.064s
W20260709 12:27:24.833352  7701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.033s	sys 0.054s
W20260709 12:27:24.833537  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.024s	sys 0.116s
W20260709 12:27:24.833700  7732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.031s	sys 0.064s
W20260709 12:27:24.834142  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.834378  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.27292e+06us
W20260709 12:27:24.508769 12530 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.066s	user 0.005s	sys 0.009s
I20260709 12:27:24.829173  7730 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:24.836849  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:24.846565  7731 raft_consensus.cc:2368] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
W20260709 12:27:24.847115  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.007s	sys 0.005s
W20260709 12:27:24.847843  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.007s	sys 0.005s
W20260709 12:27:24.842646  7661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.189s	user 0.000s	sys 0.011s
I20260709 12:27:24.847712  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:24.837070  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:24.758916  7809 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.004s
I20260709 12:27:24.868158  7809 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
W20260709 12:27:24.873461  7778 connection.cc:256] RPC call timeout handler was delayed by -1.2742203209995751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 12:27:24.880818  7778 connection.cc:256] RPC call timeout handler was delayed by -1.0231203209996238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 12:27:24.884380  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:24.778535 12687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.039s	sys 0.087s
I20260709 12:27:24.885784  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:24.887298  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:24.961778  7728 raft_consensus.cc:2368] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
W20260709 12:27:24.928773  7661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.275s	user 0.000s	sys 0.012s
W20260709 12:27:24.848446  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.29287e+06us
I20260709 12:27:24.802773 12756 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.958978  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.888319 12719 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.563s	user 0.009s	sys 0.006s
W20260709 12:27:24.925863  8405 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.002s	sys 0.006s
W20260709 12:27:24.983151  8405 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.003s	sys 0.007s
I20260709 12:27:24.963500  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2602) took 2404 ms. Trace:
W20260709 12:27:24.972086  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 87761.4us
W20260709 12:27:24.973462  7656 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.159s	user 0.007s	sys 0.044s
W20260709 12:27:24.987079  7656 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.007s	sys 0.045s
W20260709 12:27:24.849602 12537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.002s	sys 0.001s
I20260709 12:27:24.985775  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:24.995489  7736 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:24.865835 12739 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.888742 12762 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.046s	sys 0.056s
I20260709 12:27:24.778671 12705 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:24.976852  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:24.990249  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:24.973255 12615 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:24.998358 12762 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.046s	sys 0.057s
I20260709 12:27:25.023908 12399 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.847854 12655 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.984479  7661 rpcz_store.cc:276] 0709 12:27:22.559560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.943800 (+384240us) service_pool.cc:224] Handling call
0709 12:27:22.944133 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.944342 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:23.020996 (+ 76654us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.631109 (+610113us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b1800231f40. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:27:24.962715 (+1331606us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.962781 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:24.963012 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.963172 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.963401 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":75100,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":7386,"raft.queue_time_us":97,"spinlock_wait_cycles":15855360,"thread_start_us":1308887,"threads_started":1}
W20260709 12:27:24.975373 12687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.042s	sys 0.094s
I20260709 12:27:24.849164 12590 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:24.994767 12537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.002s	sys 0.001s
I20260709 12:27:25.001307 12777 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:25.004163  7736 pending_rounds.cc:85] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:24.847280 12752 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.826670 11863 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.315970 12683 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:24.887821 12530 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:24.988751 12314 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.499s	user 0.016s	sys 0.000s
I20260709 12:27:25.019850  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:25.026496  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.027s	sys 0.039s
I20260709 12:27:25.031541 12762 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:24.984479 12693 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.024708  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47955.2us
I20260709 12:27:24.885526 12073 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.028411  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.000s	sys 0.008s
I20260709 12:27:24.843540 12525 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:25.031311  7734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.019s	sys 0.047s
W20260709 12:27:25.104027  7734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.020s	sys 0.047s
I20260709 12:27:25.036145 12655 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:25.026907 12723 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.002s
I20260709 12:27:24.994052 12719 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:25.035282 12652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.021s	sys 0.029s
I20260709 12:27:24.989078 12756 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.032868 12758 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.021695 12739 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.037432  7736 pending_rounds.cc:107] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:27:25.114490  7736 raft_consensus.cc:2889] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.072736 12621 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:25.064433  7733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2707) took 2039 ms (client timeout 1899 ms). Trace:
I20260709 12:27:25.036799  7731 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:25.062849  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 904) took 1275 ms. Trace:
I20260709 12:27:25.121169  7662 rpcz_store.cc:276] 0709 12:27:23.787555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.847372 (+ 59817us) service_pool.cc:224] Handling call
0709 12:27:23.847854 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.299836 (+451982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.299891 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.299960 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:24.300013 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.062496 (+762483us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002198d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:25.062561 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.062757 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030872320}
W20260709 12:27:25.111479 12652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.022s	sys 0.030s
W20260709 12:27:25.115183 12746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.000s	sys 0.023s
W20260709 12:27:25.125906 12746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.000s	sys 0.023s
I20260709 12:27:25.126652 12746 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:25.119110  7733 rpcz_store.cc:269] 0709 12:27:23.025428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.025569 (+   141us) service_pool.cc:224] Handling call
0709 12:27:25.063987 (+2038418us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:25.064289 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180908928}
I20260709 12:27:25.035380 12399 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.133488 12399 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.134436 12399 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:25.134884 12399 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.063272 12693 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.138319 12693 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:25.141721 12525 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:25.142766 12525 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:25.029095 12705 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:25.101053  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.000s	sys 0.013s
W20260709 12:27:25.149057  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 910114us
W20260709 12:27:25.027333 12749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.003s	sys 0.004s
I20260709 12:27:25.054543 12314 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:25.112272 12723 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:25.052874 12530 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:25.036366  7655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.007s	sys 0.006s
W20260709 12:27:25.154866  7655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.319s	user 0.009s	sys 0.008s
W20260709 12:27:25.068610 12663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.004s	sys 0.009s
I20260709 12:27:25.148865 12399 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:25.097630 12656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.030s	sys 0.051s
I20260709 12:27:25.069556 12683 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:25.055158  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.027s	sys 0.039s
W20260709 12:27:25.185241  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 976878us
I20260709 12:27:25.146252 12756 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:25.065658  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.304s	user 0.004s	sys 0.010s
I20260709 12:27:25.059491 12073 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.046063 12777 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.162315  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:25.163782  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:25.151332 12749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.003s	sys 0.005s
W20260709 12:27:25.181603  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.012s	sys 0.020s
I20260709 12:27:25.182837  7658 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:25.173097  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2733) took 1725 ms. Trace:
W20260709 12:27:25.190937  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.430s	user 0.004s	sys 0.011s
W20260709 12:27:25.220448  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.219465  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.47466e+06us
W20260709 12:27:25.166829 12690 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.362s	user 0.014s	sys 0.000s
W20260709 12:27:25.168201 12386 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.328s	user 0.014s	sys 0.001s
W20260709 12:27:25.210325  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.228088  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:25.228762  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.229436  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.189996  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:25.160120 12663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.005s	sys 0.009s
W20260709 12:27:25.181479 12656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.031s	sys 0.051s
I20260709 12:27:25.165984 12719 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:25.164467 12793 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:27:25.171878 12795 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:27:25.160770 12800 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:27:25.193847  7807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.050s	sys 0.055s
W20260709 12:27:25.250984  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 273669us
W20260709 12:27:25.219573  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.013s	sys 0.021s
I20260709 12:27:25.234632  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:25.212451 12587 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.602s	user 0.014s	sys 0.000s
I20260709 12:27:25.239010  7805 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:27:25.272063  7805 pending_rounds.cc:85] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 9
I20260709 12:27:25.272372  7805 pending_rounds.cc:107] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:27:25.272626  7805 raft_consensus.cc:2889] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:25.239141  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49306.4us
I20260709 12:27:25.152698 12537 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:27:25.169045 12771 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:25.219033  7737 rpcz_store.cc:276] 0709 12:27:23.447791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.448037 (+   246us) service_pool.cc:224] Handling call
0709 12:27:23.466372 (+ 18335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:23.468207 (+  1835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.468269 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:23.469066 (+   797us) log.cc:844] Serialized 52 byte log entry
0709 12:27:23.675750 (+206684us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:24.831325 (+1155575us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.890177 (+ 58852us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.890393 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.890467 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.054470 (+164003us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b180041ab80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:25.172846 (+118376us) raft_consensus.cc:1685] finished
0709 12:27:25.172871 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.173029 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67585,"prepare.queue_time_us":398652,"prepare.run_cpu_time_us":1084,"prepare.run_wall_time_us":6010,"raft.queue_time_us":129289,"raft.run_cpu_time_us":562,"raft.run_wall_time_us":91558,"spinlock_wait_cycles":17293312,"thread_start_us":1059273,"threads_started":2,"wal-append.queue_time_us":601842}
I20260709 12:27:25.236574 12756 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:27:25.251298  7807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.051s	sys 0.055s
W20260709 12:27:25.232674  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.124760 12762 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.262888  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:25.285516  7809 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:25.274339  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.243093 12752 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.277098  7728 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:25.246717 12590 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:25.290107  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:25.290373  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:25.243921 12656 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.303488  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 935) took 1422 ms. Trace:
I20260709 12:27:25.295040  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:25.308797  7809 raft_consensus.cc:606] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:25.237553 12386 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.295712  7810 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
W20260709 12:27:25.313199  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 399339us
I20260709 12:27:25.314246  7810 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:25.309691  7809 raft_consensus.cc:614] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:25.304046  8405 rpcz_store.cc:276] 0709 12:27:23.880710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.067949 (+187239us) service_pool.cc:224] Handling call
0709 12:27:24.068508 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.069037 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.069082 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.069763 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.983934 (+914171us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:24.983985 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.984106 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.303210 (+319104us) raft_consensus.cc:1685] finished
0709 12:27:25.303251 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.303422 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1781,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":132,"thread_start_us":913746,"threads_started":1,"wal-append.queue_time_us":988160}
I20260709 12:27:25.287825 11863 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.297001 12590 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:25.315522  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.714s (SENT)
I20260709 12:27:25.344099 12796 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:25.309139 12656 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.292058 12787 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:25.349102  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:25.382831  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1027) took 1101 ms. Trace:
I20260709 12:27:25.383541  7732 rpcz_store.cc:276] 0709 12:27:24.281315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.281590 (+   275us) service_pool.cc:224] Handling call
0709 12:27:24.282234 (+   644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.282619 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.282781 (+   162us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.283652 (+   871us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.834623 (+550971us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:24.834691 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.834808 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.382453 (+547645us) raft_consensus.cc:1685] finished
0709 12:27:25.382489 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.382727 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1308,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9060,"thread_start_us":531687,"threads_started":1,"wal-append.queue_time_us":731435}
I20260709 12:27:25.342275 12590 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:25.388937  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 288380us
I20260709 12:27:25.340750 12799 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:25.310395 12525 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:27:25.392455  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 405120us
I20260709 12:27:25.318506 12450 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:27:25.397852  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:25.380539 12587 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:25.381945 12787 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:25.441332  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.698s (SENT) [suppressed 38 similar messages]
I20260709 12:27:25.419144  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:25.450621  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 135161us
W20260709 12:27:25.454286  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 219739us
I20260709 12:27:25.395195 12799 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:25.455437  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:25.413399 12705 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.443015 12779 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:25.382711 12796 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.416570 12655 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:25.444000 12787 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:25.407490 12537 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:25.452028 12525 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:25.476577  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 164333us
I20260709 12:27:25.438835 11863 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:27:25.526371  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:25.526626  7654 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:25.527585  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:25.470723 12803 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.153s	user 0.001s	sys 0.001s
W20260709 12:27:25.539572  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 98368.6us
W20260709 12:27:25.540385  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.536593  7659 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:25.472699 12779 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:25.469517 12530 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:25.478986 12756 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:27:25.525030 12797 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:27:25.492501 12655 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.475811 12073 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:25.493837 12619 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:25.590592 12537 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:25.529718 12802 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.282s	user 0.002s	sys 0.000s
I20260709 12:27:25.453433 12807 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
W20260709 12:27:25.612145  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156940us
I20260709 12:27:25.495115 12705 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:25.567051 12779 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:25.623674  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:25.494356 12777 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:25.541966 12752 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:25.654066  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.638806  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:25.662451  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:25.597471 12450 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:25.606531 12756 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:25.613147 12619 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:25.531572 12525 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.664114  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:25.623826 11863 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:25.628016 12705 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.493981 12604 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.637516 12823 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.609956 12530 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:25.710805  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.725135  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45609.9us
W20260709 12:27:25.726975  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.644847 12821 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.669404 12655 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:25.734833  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.737818  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:25.698736 12805 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 12:27:25.636436 12537 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.689031 12705 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:25.665416 12752 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:25.753383  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.733894 12790 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.379s	user 0.005s	sys 0.000s
I20260709 12:27:25.710105 12656 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:25.730996 12688 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:25.680418 12450 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.790273 12690 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:25.794000  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:25.734098 12604 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:25.822394  7660 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:25.818547 12821 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:25.752372 12828 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:25.854911  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:25.831297  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:25.858816  7661 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:25.832507 12813 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:27:25.775439 11863 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:27:25.756373 12822 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:25.777587 12756 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.777791 12619 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.836429 12811 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:27:25.837415 12820 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:27:25.837826 12827 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.838379  7731 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:25.925058  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:25.831336 12829 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:27:25.839898 12815 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.840366 12804 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.336s	user 0.001s	sys 0.001s
I20260709 12:27:25.840266 12656 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.844298 12525 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.844825  7660 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:25.846340 12816 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:27:25.727085 12823 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:25.873374  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:25.875559  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:25.876173  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:25.872226  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:25.980206  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 906) took 2187 ms. Trace:
I20260709 12:27:25.980695  8406 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:25.871804  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:25.877750  8415 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:25.874982  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2604) took 2070 ms. Trace:
I20260709 12:27:25.982421  7658 rpcz_store.cc:276] 0709 12:27:23.804473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.808231 (+  3758us) service_pool.cc:224] Handling call
0709 12:27:23.808573 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.216439 (+1407866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.874597 (+658158us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540017b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:25.874898 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159736064}
I20260709 12:27:25.880770  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:25.876713  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:25.856849 12794 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.322s	user 0.000s	sys 0.002s
I20260709 12:27:25.757313 12655 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.925345  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2601) took 1084 ms. Trace:
W20260709 12:27:25.839089 12798 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
W20260709 12:27:25.932044  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.934351  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 270319us
I20260709 12:27:26.066820  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:26.062763  7731 rpcz_store.cc:276] 0709 12:27:24.840499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.837560 (+997061us) service_pool.cc:224] Handling call
0709 12:27:25.837969 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.924845 (+ 86876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.925231 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:25.829136 12807 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:25.872025 12821 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.971611  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:26.111598  7660 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:27:25.973948  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.874914 12530 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:25.876987 12537 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:25.977831  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2601) took 3419 ms. Trace:
I20260709 12:27:26.151743  7656 rpcz_store.cc:276] 0709 12:27:22.558905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.762939 (+204034us) service_pool.cc:224] Handling call
0709 12:27:22.763271 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.814061 (+1050790us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:24.987697 (+1173636us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.345073 (+357376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:25.345152 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:25.345208 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.977456 (+632248us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540021b6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:25.977529 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.977726 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8503,"prepare.queue_time_us":1354382,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":117668,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":23339,"spinlock_wait_cycles":2429499136,"thread_start_us":1530099,"threads_started":2,"wal-append.queue_time_us":576460}
I20260709 12:27:25.981657  8405 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:25.824483 12688 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.039317 12756 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:26.160609  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 296393us
I20260709 12:27:26.056044  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:25.916746 12811 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.093132 12810 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:25.972414 12815 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.094199 12837 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:26.095714 12824 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:26.101455 12829 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:27:26.102234  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 369358us
I20260709 12:27:25.933982 12796 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.094323  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:26.103354  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:25.866474 12828 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.980829  7655 rpcz_store.cc:276] 0709 12:27:23.792782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.795375 (+  2593us) service_pool.cc:224] Handling call
0709 12:27:23.795878 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.835019 (+ 39141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.835071 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.835727 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.156280 (+1320553us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:25.156321 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.156396 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.979915 (+823519us) raft_consensus.cc:1685] finished
0709 12:27:25.979949 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.980125 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38566,"raft.queue_time_us":40503,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":1319176,"threads_started":1,"wal-append.queue_time_us":1460357}
I20260709 12:27:26.162170  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2597) took 1323 ms. Trace:
I20260709 12:27:26.216894  7733 rpcz_store.cc:276] 0709 12:27:24.838850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.128642 (+289792us) service_pool.cc:224] Handling call
0709 12:27:25.129026 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.129403 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.129436 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.130334 (+   898us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.381574 (+251240us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:25.381641 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.381757 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.161766 (+780009us) raft_consensus.cc:1685] finished
0709 12:27:26.161805 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.162055 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9759,"thread_start_us":250637,"threads_started":1,"wal-append.queue_time_us":363803}
I20260709 12:27:26.162729  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:26.190026  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.004s	sys 0.013s
W20260709 12:27:26.148873 12787 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.002s
I20260709 12:27:26.202651  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:26.185298 12683 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.955s	user 0.017s	sys 0.000s
I20260709 12:27:25.918972 12822 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:25.978041 12656 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.222453  7661 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:26.224097  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.004s	sys 0.014s
W20260709 12:27:26.254660  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 840022us
I20260709 12:27:26.228546  7660 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:26.229708  7733 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:26.256927  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2603) took 1415 ms. Trace:
I20260709 12:27:26.257411  7733 rpcz_store.cc:276] 0709 12:27:24.841127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.225875 (+1384748us) service_pool.cc:224] Handling call
0709 12:27:26.226236 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.256452 (+ 30216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.256853 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:26.241981  7655 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:26.243522  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:26.260926  7658 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:26.262081  7733 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:27:26.214572 12698 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:26.127158 12807 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.272140  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2606) took 1399 ms. Trace:
I20260709 12:27:26.280387  7733 rpcz_store.cc:276] 0709 12:27:24.873279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.258299 (+1385020us) service_pool.cc:224] Handling call
0709 12:27:26.258946 (+   647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:26.265813 (+  6867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.272063 (+  6250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:26.279186  7778 leader_election.cc:400] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 9 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10.
I20260709 12:27:26.283268  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:26.147459 12821 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.252010  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.296793  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:26.299052  7778 leader_election.cc:400] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:26.300000  7778 leader_election.cc:403] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:26.246742 12787 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.313977 12787 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.314374  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2851) took 2120 ms. Trace:
I20260709 12:27:26.315285 12823 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.316042 12823 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:26.317123 12799 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.318497 12831 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:27:26.256003 12837 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:26.320541 12837 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.324517  7737 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:26.325829 12779 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 12:27:26.325956  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2599) took 1486 ms. Trace:
I20260709 12:27:26.254133 12777 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.326879  7737 rpcz_store.cc:276] 0709 12:27:24.839772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.287365 (+447593us) service_pool.cc:224] Handling call
0709 12:27:25.305785 (+ 18420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.325051 (+1019266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.325696 (+   645us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002aba54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:26.325893 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2342313344}
W20260709 12:27:26.332103  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 228893us
I20260709 12:27:26.318734 12799 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.338289  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:26.339161  7656 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:26.217991 12758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.003s	sys 0.016s
I20260709 12:27:26.314803  7807 rpcz_store.cc:276] 0709 12:27:24.194103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.234924 (+ 40821us) service_pool.cc:224] Handling call
0709 12:27:24.235478 (+   554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:24.472285 (+236807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.472343 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:24.473442 (+  1099us) log.cc:844] Serialized 52 byte log entry
0709 12:27:25.291063 (+817621us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:25.306752 (+ 15689us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.307103 (+   351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.313923 (+1006820us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54003f3754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:26.313971 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.314024 (+    53us) raft_consensus.cc:1685] finished
0709 12:27:26.314137 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.314311 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17447,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":4991,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":12840,"spinlock_wait_cycles":542937216,"thread_start_us":810732,"threads_started":1,"wal-append.queue_time_us":867966}
W20260709 12:27:26.345626  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.006s	sys 0.042s
W20260709 12:27:26.364495  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.265931 12830 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:26.365130  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.007s	sys 0.047s
W20260709 12:27:26.348259  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96365.1us
I20260709 12:27:26.270879 12698 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.270761 12683 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:26.278681 12838 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:26.380228  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.378507  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:26.294572 12752 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:26.347465 12825 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:27:26.330379 12073 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:26.336575 12530 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:26.307706 12525 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:26.349056 12758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.004s	sys 0.017s
I20260709 12:27:26.215869 12824 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.341337 12814 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:26.410218 12707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.001s
I20260709 12:27:26.346357 12851 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:26.362883 12831 consensus_queue.cc:1243] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:26.211184 12810 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.326813 12779 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.401751 12838 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.372596 12450 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.384027 12836 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:27:26.445890  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65777.9us
I20260709 12:27:26.346575 12777 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.349836 12844 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
W20260709 12:27:26.447373 12707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.008s	sys 0.001s
I20260709 12:27:26.427557 12837 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.288803 12811 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.462955  8414 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:26.411028 12683 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.453584 12814 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:26.400437 12830 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.454912 12758 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:26.456646 12656 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:26.468945 12399 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.308s	user 0.015s	sys 0.002s
I20260709 12:27:26.453941 12838 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:26.460722 12851 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:26.455615 12836 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:26.463917 12831 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:26.473619 12828 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.476678 12822 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:26.506052 12705 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.000s
I20260709 12:27:26.493191 12758 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.514300 12758 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:26.516274 12843 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.055s	user 0.000s	sys 0.002s
I20260709 12:27:26.517608 12688 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:26.495533 12846 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:27:26.497013 12834 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:27:26.497418 12841 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:26.498092 12779 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:26.498469 12807 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.519819 12841 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:26.499253 12821 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.496938 12810 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.521113 12841 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.493477 12831 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.522579 12705 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.523013 12831 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.506753 12815 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.507989 12836 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.483322 12814 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:26.512810 12830 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:26.512496 12838 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.503954 12851 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.528396 12838 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:26.591899  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.027s	sys 0.065s
I20260709 12:27:26.592233  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2323) took 1066 ms. Trace:
W20260709 12:27:26.592913  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.028s	sys 0.066s
W20260709 12:27:26.593581  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 657479us
W20260709 12:27:26.407588  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.042s	sys 0.051s
W20260709 12:27:26.595942  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.042s	sys 0.051s
W20260709 12:27:26.598323  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.602366  7656 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:27:26.593273  7662 rpcz_store.cc:276] 0709 12:27:25.525606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.525772 (+   166us) service_pool.cc:224] Handling call
0709 12:27:25.530860 (+  5088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.555105 (+ 24245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.555150 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.594650 (+ 39500us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.045989 (+451339us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:26.046037 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.046344 (+   307us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.590611 (+544267us) raft_consensus.cc:1685] finished
0709 12:27:26.590767 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.591860 (+  1093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29025,"raft.queue_time_us":146244,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":43395,"thread_start_us":382717,"threads_started":1,"wal-append.queue_time_us":593192}
I20260709 12:27:26.422636  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:26.626426  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 204966us
I20260709 12:27:26.628661  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:26.613504  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2607) took 1739 ms. Trace:
I20260709 12:27:26.631510  7728 rpcz_store.cc:276] 0709 12:27:24.873803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.983292 (+109489us) service_pool.cc:224] Handling call
0709 12:27:26.612534 (+1629242us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400211e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:26.613130 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510755584}
I20260709 12:27:26.612947  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:26.626506  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1810719550003341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:27:26.635118  7659 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:26.616001  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2732) took 3208 ms (client timeout 3193 ms). Trace:
W20260709 12:27:26.637146  7730 rpcz_store.cc:269] 0709 12:27:23.410200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.410813 (+   613us) service_pool.cc:224] Handling call
0709 12:27:26.615034 (+3204221us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400419454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:26.615627 (+   593us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3185558272}
I20260709 12:27:26.616349  7653 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:26.636639  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.623s (SENT)
I20260709 12:27:26.642617  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:26.607657  7701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.009s	sys 0.018s
W20260709 12:27:26.648360  7701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.010s	sys 0.019s
W20260709 12:27:26.650017  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.650264  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0244719550000809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 12:27:26.641264  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 338334us
I20260709 12:27:26.648314  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 809) took 3516 ms. Trace:
I20260709 12:27:26.651309  7805 rpcz_store.cc:276] 0709 12:27:23.131925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.152865 (+ 20940us) service_pool.cc:224] Handling call
0709 12:27:23.227786 (+ 74921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.645015 (+3417229us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b5400229aa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99b06 00007fc001f2514a 00007fc001f24fb7 00007fc001f25470 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91
0709 12:27:26.645565 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.647557 (+  1992us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:26.648028 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4634380416}
I20260709 12:27:26.642490  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.654011  7806 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:26.654170  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:26.656061  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.656893  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:26.658334  7807 raft_consensus.cc:2368] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:26.660988  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:26.662571  7808 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:26.658964  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.643982  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:27:26.619412  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2725) took 3318 ms (client timeout 1670 ms). Trace:
W20260709 12:27:26.502112 12621 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.352s	user 0.017s	sys 0.000s
W20260709 12:27:26.650467  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.617169  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2600) took 1776 ms. Trace:
I20260709 12:27:26.508260 12683 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:26.639716  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:26.473992 11863 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:26.672724  7804 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:26.723541  7804 raft_consensus.cc:614] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:26.673604 12587 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:26.724916  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85616.9us
I20260709 12:27:26.725353 12587 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:26.669358  7735 rpcz_store.cc:269] 0709 12:27:23.300285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.300427 (+   142us) service_pool.cc:224] Handling call
0709 12:27:26.618582 (+3318155us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002ab554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:26.619040 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3093747968}
W20260709 12:27:26.676446  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 69923.1us
I20260709 12:27:26.696491  7732 rpcz_store.cc:276] 0709 12:27:24.840132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.393579 (+553447us) service_pool.cc:224] Handling call
0709 12:27:25.394131 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.394460 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:25.411969 (+ 17509us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.756247 (+344278us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.232010 (+475763us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.615840 (+383830us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.616282 (+   442us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:26.616384 (+   102us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:26.616490 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.616624 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.616928 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1339093,"prepare.queue_time_us":77458,"prepare.run_cpu_time_us":185,"prepare.run_wall_time_us":5414,"raft.queue_time_us":719529,"raft.run_cpu_time_us":1446,"raft.run_wall_time_us":1380799,"thread_start_us":1189894,"threads_started":4,"wal-append.queue_time_us":423738}
I20260709 12:27:26.679183  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:26.729564 12839 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:26.681018  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:26.682170 12756 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:26.683189 12821 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.732893  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:26.733194  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:26.733343 12821 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.734424 12845 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:27:26.709576  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.703955  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45391.9us
I20260709 12:27:26.734362  7730 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:26.719110  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.737644  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.728366 12830 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:26.681354 12697 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:26.736575  7737 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:26.722837 11863 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.749694 12780 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.750154 12838 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:26.753397  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:26.756572 12864 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:26.756816  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:26.757850  7733 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:26.757421 12399 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.758432 12537 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.759411 12537 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.761482  7810 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:26.722927 12717 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.752329  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:26.781860 12842 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.783067 12842 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.780222  7812 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:26.757449 12864 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.779258 12839 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.808379  7735 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:26.809093  7731 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:26.810647 12807 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:26.811192 12807 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.811630 12807 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:26.811856  7653 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:26.815321 12799 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:26.815498 12073 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.816331 12799 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.817830  7732 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:26.819362 12847 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.819643 12314 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.820215 12847 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.820631 12314 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.823817 12815 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.825155 12815 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.827649 12853 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:27:26.829046 12853 consensus_queue.cc:1243] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:26.829929 12758 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:27:26.831120 12758 consensus_queue.cc:1243] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:26.839784 12828 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:26.849256 12830 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:26.831779 12849 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:26.835358 12851 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:26.803045  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:26.905853  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 127389us
I20260709 12:27:26.795858 12864 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:26.918320  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:26.816697 12073 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.923442 12857 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
W20260709 12:27:26.924517 12862 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:27:26.926855 12604 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.073s	user 0.016s	sys 0.000s
I20260709 12:27:26.751143 12780 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.928545  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:26.850453 12717 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:26.898929 12847 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.940022 12556 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:26.940790  8413 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:26.941176  7732 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:27:26.943876 12849 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:26.946632  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.027s (SENT)
I20260709 12:27:26.928053  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:26.912706  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:26.951583 12830 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:26.953406 12590 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.491s	user 0.010s	sys 0.007s
I20260709 12:27:26.953249 11863 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.955328 12705 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.953279 12683 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:26.959476  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 90863.8us
I20260709 12:27:26.960594 12698 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:26.961715 12815 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:26.963061  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55373.4us
I20260709 12:27:26.964128 12821 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.973577  7732 pending_rounds.cc:85] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:26.924551 12587 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:26.981196  7804 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:26.973133 12556 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:26.995446  7732 pending_rounds.cc:107] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:27.000942  7732 raft_consensus.cc:2889] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.002365  7733 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:27.011492  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 243529us
I20260709 12:27:27.015758 12386 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.015882  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.018395 12556 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.021270  7730 raft_consensus.cc:1217] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-9.5]   Dedup: 7.4->[9.5-9.5]
I20260709 12:27:27.022333  7730 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.4-9.5]
W20260709 12:27:27.024616  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 234154us
I20260709 12:27:27.021739 12821 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:27.028690 12821 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.029171 12821 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.030146 12771 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.030850 12771 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:27.021646 12749 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:26.939133 12828 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.032003 12683 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.050925  7735 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:27.054785  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.069434 12857 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.070066  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:27.041559 12698 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.034569 12314 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.036628 12749 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:27.077526 12683 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.075131  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:27.078208 12450 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.011592 12590 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.099901  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:27.109735  7656 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:27.116204  7730 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:27.117268  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 170743us
I20260709 12:27:27.079149 12857 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.121732  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.125165 12779 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:27.126434 12779 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.125972 12749 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.127511 12314 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:27.128594 12314 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.129215 12314 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:27.131376 12822 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:27.131585 12450 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.132248 12822 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.135370 12688 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:27.135923 12688 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.136370 12688 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.136618 12824 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.127939 12749 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:27.140183 12870 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:27.127048 12779 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.127591 12758 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.076211 12871 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:27:27.134963 12822 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.169937  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.163832  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.173365  7653 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.170856  7733 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:27.155287 12799 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.197753 12399 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.121666 12839 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.166179 11863 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:27.203627  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.203372 12758 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:27.204728 12814 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.679s	user 0.016s	sys 0.004s
I20260709 12:27:27.205466 12814 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:27.182454  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.177436  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:27.186066  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 120327us
W20260709 12:27:27.187580  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.193s (SENT)
I20260709 12:27:27.191584  7626 leader_election.cc:400] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:27.209388  7626 leader_election.cc:403] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:27.194481 12073 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.194988 12764 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.195166 12830 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:27.206084  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.209352  7812 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.194046 12537 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.186162 12739 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.215420  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:27.233139 12764 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:27.234838  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.242136  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.242890 12694 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.243242  7733 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:27:27.245055  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:27.243116  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 942) took 2081 ms (client timeout 1666 ms). Trace:
I20260709 12:27:27.252982 12525 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:27.194123 12849 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.256983 12590 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.257133 12759 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.259094 12857 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:27.265416 12809 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.620s	user 0.007s	sys 0.008s
W20260709 12:27:27.267201 12801 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.905s	user 0.005s	sys 0.010s
I20260709 12:27:27.304559 12759 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:27.268991 12831 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:27.306005 12655 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.250758 12717 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.306854 12587 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:27.273541  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.309378  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.309868  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.275113  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2602) took 2434 ms. Trace:
I20260709 12:27:27.303383  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.275911  7733 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:27:27.313707  7733 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:27:27.314083  7733 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:27.276757  8413 rpcz_store.cc:269] 0709 12:27:25.161349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.162018 (+   669us) service_pool.cc:224] Handling call
0709 12:27:27.242743 (+2080725us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:27.243009 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3968703744}
W20260709 12:27:27.287881  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136682us
W20260709 12:27:27.290903  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.277366 12807 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.303345 12386 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:27.304277  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.234253  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.310475  7731 rpcz_store.cc:276] 0709 12:27:24.840852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.147183 (+1306331us) service_pool.cc:224] Handling call
0709 12:27:26.206904 (+ 59721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.810218 (+603314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.217291 (+407073us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540037ee54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:27.275016 (+ 57725us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104211200}
I20260709 12:27:27.319399 12587 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:27.321102 12877 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:27:27.320689  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.322419 12587 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.324324  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.325884  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.326228  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.328186  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.329224  7731 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:27.330389  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:27.323035  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.333256 12842 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.334959 12807 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.327158  8411 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:27.331071  7731 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.341146  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.344027 12868 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:27.344547 12814 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.346426 12811 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:27.349226 12828 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:27.350509  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1119) took 1477 ms (client timeout 1955 ms). Trace:
W20260709 12:27:27.350972  8414 rpcz_store.cc:269] 0709 12:27:25.872970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.873088 (+   118us) service_pool.cc:224] Handling call
0709 12:27:27.350121 (+1477033us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b540046d7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:27.350318 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111871744}
I20260709 12:27:27.356329  8415 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:27:27.278738 12525 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:27.358839  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.358626 12525 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.360318 12525 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:27.362183 12806 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.903s	user 0.005s	sys 0.010s
I20260709 12:27:27.362466  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.306993 12655 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:27.364596  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1030) took 3042 ms. Trace:
I20260709 12:27:27.365800  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2865) took 1525 ms. Trace:
I20260709 12:27:27.365643  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.366514  7806 rpcz_store.cc:276] 0709 12:27:25.840051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.856019 (+ 15968us) service_pool.cc:224] Handling call
0709 12:27:25.856385 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.655028 (+798643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.365326 (+710298us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54003c55d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:27.365684 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832257792}
I20260709 12:27:27.367244  7809 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:27.368008  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.368490  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.368937 12556 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.369735  7806 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:27:27.372083  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68896.8us
W20260709 12:27:27.373407  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58216.9us
I20260709 12:27:27.370965  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.342796 12780 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.383198  7812 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:27.276563  7653 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:27.380468  7806 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:27.385052  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:27.344569 12868 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:27.365335  7734 rpcz_store.cc:276] 0709 12:27:24.322131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.322686 (+   555us) service_pool.cc:224] Handling call
0709 12:27:24.323329 (+   643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.323957 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.324003 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.324838 (+   835us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.104592 (+779754us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:25.104647 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.104738 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.262297 (+2157559us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:27.308958 (+ 46661us) raft_consensus.cc:1685] finished
0709 12:27:27.308983 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.364386 (+ 55403us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1820,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":155,"spinlock_wait_cycles":54557440,"thread_start_us":779203,"threads_started":1,"wal-append.queue_time_us":975568}
I20260709 12:27:27.395582 12764 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.396207 12587 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.395984 12779 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.398442 12750 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.398262 12314 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.390244  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.313760 12604 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:27.404275  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.299610 12864 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.409471  7735 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:27.387053 12694 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.427007  7735 pending_rounds.cc:85] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:27:27.436933  7735 pending_rounds.cc:107] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:27:27.437378  7735 raft_consensus.cc:2889] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.387657 12771 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.443770 12771 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.445663 12864 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.447543 12604 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.449685  8413 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:27.452404  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a5afed00934a1d9878dd900b236263"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:27:27.452634  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.459703  7804 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:27.461234  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:27.472849  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:27.461472  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 274031us
I20260709 12:27:27.461524 12587 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:27.462741  8379 connection.cc:256] RPC call timeout handler was delayed by -1.6311182140002529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 12:27:27.477232  8379 connection.cc:256] RPC call timeout handler was delayed by -1.6311182140002529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 12:27:27.477596  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5492182140001205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 12:27:27.477927  8379 connection.cc:256] RPC call timeout handler was delayed by -1.2819182140001431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 12:27:27.430747 12688 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.475744  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.424594 12590 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:27.466933  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2863) took 1729 ms. Trace:
I20260709 12:27:27.486528  7730 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:27.461392  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1032) took 3088 ms. Trace:
I20260709 12:27:27.410931 12868 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.485922  7810 rpcz_store.cc:276] 0709 12:27:25.737231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.737414 (+   183us) service_pool.cc:224] Handling call
0709 12:27:27.466665 (+1729251us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:27.466834 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328972032}
I20260709 12:27:27.451159 12604 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:27.455631 12530 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.012s
I20260709 12:27:27.499454  7808 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.460881 12657 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.503242  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.452911 12680 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.501868  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 128982us
I20260709 12:27:27.465075 12750 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.456903 12779 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.507151  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:27.463135 12694 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.459836 12758 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.485836 12590 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.514444  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.518513  7810 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:27.518664 12587 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.523383 12836 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.523526 12750 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:27.524885 12873 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.435s	user 0.003s	sys 0.000s
I20260709 12:27:27.524565 12836 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.525986  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.526769 12750 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.527611 12680 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.530753 12777 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.531968 12690 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:27.465787 12650 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.527817  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.461017 12821 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.489591  7729 rpcz_store.cc:276] 0709 12:27:24.329062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.329195 (+   133us) service_pool.cc:224] Handling call
0709 12:27:24.329631 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.330065 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.330099 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.330676 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.254141 (+923465us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:25.254205 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.254307 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.361161 (+2106854us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:27.392650 (+ 31489us) raft_consensus.cc:1685] finished
0709 12:27:27.392688 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.417385 (+ 24697us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9841,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":6836,"spinlock_wait_cycles":13541248,"thread_start_us":922958,"threads_started":1,"wal-append.queue_time_us":1023888}
I20260709 12:27:27.553278  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.532450 12530 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:27.536540 12590 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.530823 12680 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.563382  7731 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:27.564770  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.567399  7657 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:27.559392  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.543021 12690 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.579804  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:27.580355  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.581542  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.582293  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.586622  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.588307  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:27.590771  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48016.6us
I20260709 12:27:27.591383  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.592463  7659 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.593811  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.594923  7657 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:27.598165  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.544886 12749 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:27:27.548803 12851 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:27.605217  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.618422  7700 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.698s (SENT)
I20260709 12:27:27.568285 12399 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:27.603435  7805 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:27.556439 12694 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:27:27.627590  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.579200 12690 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.667886  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.557943 12657 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.670830  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.572933 12650 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:27.679665  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63975.2us
I20260709 12:27:27.684816  7657 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.689919  7812 raft_consensus.cc:606] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:27.690456  7812 raft_consensus.cc:614] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:27.634552 12851 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.671697 12822 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.697537  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:27.697876 12817 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.972s	user 0.001s	sys 0.017s
I20260709 12:27:27.698566 12688 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.696790 12745 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:27:27.704702  7653 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:27.707585  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.707988  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.739461 12745 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:27.739975  7654 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:27.740350  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.741364  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.742182  7654 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 7.
I20260709 12:27:27.709730 12650 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.711635 12657 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.718819 12759 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.745760 12759 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:27.747932  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.733207  7658 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.735519  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.751832  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.752317  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.752743  7658 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.753316  7655 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:27.753816  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.754740  7658 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.756672  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:27.708395  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.739454  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2595) took 2901 ms. Trace:
I20260709 12:27:27.742852  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.758311  7653 raft_consensus.cc:2368] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:27.741884  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.759827  7805 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:27:27.757843  7736 rpcz_store.cc:276] 0709 12:27:24.838215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.003339 (+165124us) service_pool.cc:224] Handling call
0709 12:27:25.003887 (+   548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:25.121443 (+117556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.121499 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:25.122445 (+   946us) log.cc:844] Serialized 52 byte log entry
0709 12:27:25.386132 (+263687us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:25.386182 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.386264 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.630126 (+2243862us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:27.738622 (+108496us) raft_consensus.cc:1685] finished
0709 12:27:27.738646 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.739209 (+   563us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1372439,"raft.queue_time_us":1860,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":1371202,"spinlock_wait_cycles":14452608,"thread_start_us":202323,"threads_started":1,"wal-append.queue_time_us":441282}
I20260709 12:27:27.690042 12779 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:27.760833  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.761181 12779 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.761726  7805 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.761819 12779 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.762452  7657 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:27.739992 12745 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:27.763059  7805 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 14. Ops: []
I20260709 12:27:27.763471  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.764751  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.764228 12745 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.766088 12745 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.766374  7805 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:27.750006  7660 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:27:27.737553  7659 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:27.671413 12314 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.766412  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.770700  7808 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:27.754616  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.775110  7807 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:27.783733  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.784004  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.783818  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.785663  7661 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:27:27.786803  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.787470 12657 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.788190 12657 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.788795  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.789857  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.790516  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.787755  7659 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.791464 12831 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:27.792251  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.784000 12752 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:27:27.776803 12749 raft_consensus.cc:2764] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:27:27.793793 12752 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:27.794770  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.789070  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.796386  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.797243  7808 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.798316  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.793519 12821 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.780946 12688 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:27.776561 12759 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:27.799203  7808 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:27.801327  8405 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:27:27.801666  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.838907  8406 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:27.802762  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.804620  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:27.841797  8415 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:27.804746  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:27.816228  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.823009  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:27.809279  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.811939  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.846796  7658 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.812305  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:27.802140  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:27.823971  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.852356  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.822000  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.859195  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2867) took 1209 ms. Trace:
I20260709 12:27:27.860119  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:27.861840  7660 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:27.859784  7809 rpcz_store.cc:276] 0709 12:27:26.649491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.653282 (+  3791us) service_pool.cc:224] Handling call
0709 12:27:27.858995 (+1205713us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b54003c50d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:27.859134 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636632832}
I20260709 12:27:27.887535  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:27.890946  7661 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:27.893731  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:27.895666  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.896282  7657 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.896656  7808 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:27.807632 12752 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:27:27.812500 12822 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.851779 12759 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:27.829722 12688 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.905064  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.897693  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:27.905421  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123547us
I20260709 12:27:27.915354  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.897442 12779 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.916610  7808 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:27.918011  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:27.918711 12779 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.919004  7808 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:27.899466 12821 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.920420  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:27.920169 12821 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.921603  7657 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:27.922143 12759 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:27.899587 12314 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:27.927855 12314 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.928442 12314 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:27.932673 12657 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.900396 12796 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:27:27.934691 12655 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:27.935322 12655 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:27.935981 12587 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:27.936386 12655 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.936774 12587 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:27.937600  7809 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:27.938274 12655 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:27.934777 12688 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:27.939921  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:27.940086 12587 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:27.940941 12587 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.941563 12587 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.943454 12739 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.944542 12779 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:27.944715 12587 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.945504 12779 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:27.946353  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:27.946823  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.947328  7733 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:27.948648  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.950248  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:27.951107  7730 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:27.952087  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:27.946703 12779 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.957984 12688 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:27.975452  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.979570 12863 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:27.980408 12863 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:27.981791 12863 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:27.899181  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:27.996212  8415 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:27:28.001849  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.003252  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 104176us
I20260709 12:27:28.007079  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.011659  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.012600  7736 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:27.997418  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.017761  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.018740  7733 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:27.903677 12822 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:27.801239 12690 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.022991  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.032521  7734 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:27:28.008455 12796 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:27.901389 12590 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.038992  7809 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:28.032907  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 358622us
I20260709 12:27:28.033111  7734 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:27:28.039566  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:28.049712 12690 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:28.041090  7734 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.054385  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.055166  7730 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:28.056535  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 412172us
I20260709 12:27:27.944574 12739 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.090970 12739 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:28.071470 12771 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:28.086138 12590 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:28.087510 12821 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:28.051000 12796 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:28.088596  7732 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:28.104526  7733 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:28.106247  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.106932  7733 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:28.114534  7626 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.623s (SENT)
W20260709 12:27:28.114480  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 439693us
I20260709 12:27:28.098393 12771 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.090961 12688 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:28.147917  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.128738  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:28.152287  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.159009 12650 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:27:28.143805 12771 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.172940 12650 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:28.173439 12650 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:28.170331  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:28.176229 12657 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:28.177174 12657 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:28.178664  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 819) took 1172 ms. Trace:
I20260709 12:27:28.178336 12657 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.179116  7803 rpcz_store.cc:276] 0709 12:27:27.006136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.015349 (+  9213us) service_pool.cc:224] Handling call
0709 12:27:27.353164 (+337815us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:27:27.353443 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:27.502093 (+148650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.502180 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:27.533688 (+ 31508us) log.cc:844] Serialized 26 byte log entry
0709 12:27:28.018289 (+484601us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:28.051438 (+ 33149us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.051706 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.051783 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:28.178157 (+126374us) raft_consensus.cc:1685] finished
0709 12:27:28.178286 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.178469 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158544,"prepare.queue_time_us":34594,"prepare.run_wall_time_us":105,"raft.queue_time_us":149069,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":18814,"spinlock_wait_cycles":77618304,"thread_start_us":442056,"threads_started":1,"wal-append.queue_time_us":587001}
I20260709 12:27:28.182006  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:28.183153  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.184187  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.184332  7805 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:28.190043 12752 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:28.192369 12752 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:28.196403  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 81977.8us
I20260709 12:27:28.205086  7805 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 10. Ops: []
I20260709 12:27:28.207284  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.174670 12650 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.195040 12752 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:28.221783  7733 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:28.224243  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.231561  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.245968  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.247030  7730 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:28.249464  7775 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:28.236357  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.246783  7805 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:28.239472  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:28.263942  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:28.231139 12888 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.487s	user 0.001s	sys 0.000s
I20260709 12:27:28.268342  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:28.269531  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64953.2us
W20260709 12:27:28.288038  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.285379  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:28.294003  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.254629 12752 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.297573  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.298393  7730 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:28.280671 12745 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.009s
I20260709 12:27:28.272583 12792 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:28.282720  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.310725  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.314350  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.317364  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.318992  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.325826  7736 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:28.258198 12314 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.274106 12759 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.333541  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.340562  7730 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:28.335172  7733 pending_rounds.cc:85] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 3
I20260709 12:27:28.342072  7733 pending_rounds.cc:107] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:27:28.338275  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.342465  7733 raft_consensus.cc:2889] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.259395 12796 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.316540 12896 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:28.310187 12792 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:28.316722 12811 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:28.357687  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 108387us
I20260709 12:27:28.301870 12752 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.362128  8406 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:27:28.362911  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:28.365514  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 214483us
W20260709 12:27:28.371788  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.623s (SENT)
I20260709 12:27:28.372607  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:28.379840 12904 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.380141  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.380550 12904 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.386987  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.408006  7736 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:28.357502 12792 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:28.407845  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 116548us
I20260709 12:27:28.357944 12688 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:28.383074 12314 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:28.417379  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.418376  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.421849  8415 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:28.392064 12904 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:28.383708 12896 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:28.426956  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.428102  7736 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:27:28.432010 12896 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:28.435415  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63664.3us
W20260709 12:27:28.436108 12902 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:27:28.439949  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:28.445019  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:28.445886  7653 raft_consensus.cc:2368] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14.
I20260709 12:27:28.445672  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:28.408268 12811 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.456545  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.460312  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.426100 12611 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:28.462409  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:28.464030 12836 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:28.465317 12450 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:28.468645 12901 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:27:28.473683 12811 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:28.475597  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50884.8us
I20260709 12:27:28.485222  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:28.492678  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:28.492015  7736 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:28.493633  7732 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:28.498960  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.500335  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.467276 12851 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:28.510052  7687 heartbeater.cc:646] Failed to heartbeat to 127.6.76.254:33941 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.76.254:33941 timed out after 15.000s (SENT)
I20260709 12:27:28.503089 12863 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:28.512401 12863 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:28.512971 12863 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:28.515465 12771 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.517976 12771 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:28.526265 12851 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:28.530211  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.531559  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.532078  7653 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:28.532506  7735 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:28.533504  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.533768  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 828) took 1014 ms. Trace:
I20260709 12:27:28.534320  7735 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:28.531134  7732 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:28.536614  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:28.548331  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.534255  7803 rpcz_store.cc:276] 0709 12:27:27.518914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.181070 (+662156us) service_pool.cc:224] Handling call
0709 12:27:28.181553 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.198771 (+ 17218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.198808 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.222539 (+ 23731us) log.cc:844] Serialized 26 byte log entry
0709 12:27:28.395723 (+173184us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:28.395769 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.395841 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:28.533525 (+137684us) raft_consensus.cc:1685] finished
0709 12:27:28.533548 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.533707 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8440,"raft.queue_time_us":16723,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5411,"thread_start_us":101778,"threads_started":1,"wal-append.queue_time_us":246904}
I20260709 12:27:28.554677  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.555583  7732 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:28.559139  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:28.565713  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.566473  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:28.567615  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.570350  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.571940  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:28.568518  7735 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:28.574035  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.575964  7659 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:28.578579  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.579838  7736 raft_consensus.cc:2368] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12.
I20260709 12:27:28.585481  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.581820  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.591391  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.565376 12690 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:28.597572  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61090.1us
I20260709 12:27:28.601073  7735 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:28.586933  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.605388  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:28.603528  7811 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:28.583379 12759 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:28.611363  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:28.612258  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.613286  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:28.612346  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 104153us
I20260709 12:27:28.614590  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:28.615370  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.626042  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.628115  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:28.629436  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.609491 12530 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:28.631126  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.631969  7736 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:28.630200  7732 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:28.641491  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.643702  7734 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:28.647612  7735 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:28.647774  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:28.649187  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.628837 12690 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:28.654130 12690 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:28.650840  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 951) took 1329 ms. Trace:
I20260709 12:27:28.652021  7735 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:28.655085  8413 rpcz_store.cc:276] 0709 12:27:27.321807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.340714 (+ 18907us) service_pool.cc:224] Handling call
0709 12:27:28.650628 (+1309914us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54007b3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:28.650765 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244470016}
I20260709 12:27:28.656428  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:28.652465  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.655385  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.653023  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:28.657979  7735 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:28.630963 12796 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:28.640743 12530 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:28.657783  7734 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:28.672514  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.661171 12450 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:28.651018 12611 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:27:28.661666 12690 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.662949  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:28.660741 12796 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.678161 12796 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:28.678352  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:28.663930 12759 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.663830  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:28.667929  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.679567 12759 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.680821  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.681967 12759 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:28.682090  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.682921  7733 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:28.683631 12894 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:28.652108  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:28.687642  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.677042 12611 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:28.677676 12619 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:28.683895  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:28.676294  7734 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 9: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:28.697530  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.694787 12450 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:28.701470 12450 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:28.703784  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.705056  7734 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:28.706655  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.692287 12611 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:28.673987 12530 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.715961  7662 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:28.736078  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.738224  7734 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
W20260709 12:27:28.739071  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.741106  7736 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:28.741870  7735 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
W20260709 12:27:28.752900  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.762967  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.788398  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.791184  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.828444  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.829370  7732 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:28.831071  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.831835  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:28.833190  7734 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:28.854453  7806 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:27:28.857300  7730 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:28.859688 12619 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.860567 12619 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.861088  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50788 (request call id 35) took 1448 ms. Trace:
I20260709 12:27:28.861713  7812 rpcz_store.cc:276] 0709 12:27:27.413060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.452221 (+ 39161us) service_pool.cc:224] Handling call
0709 12:27:28.860700 (+1408479us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:28.860852 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491026304}
I20260709 12:27:28.864270  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2877) took 1401 ms. Trace:
I20260709 12:27:28.869199  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.865368  7807 rpcz_store.cc:276] 0709 12:27:27.463061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.586182 (+123121us) service_pool.cc:224] Handling call
0709 12:27:28.863950 (+1277768us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54003c4bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:28.864166 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430095744}
I20260709 12:27:28.875545  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:28.876657  7735 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:28.878836  7734 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:27:28.890974  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.994266  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103828us
W20260709 12:27:28.995779  7627 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.514s (SENT)
I20260709 12:27:28.996428  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:29.009394 12792 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.006s
I20260709 12:27:29.010139 12792 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.012684 12619 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.016505 12399 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:29.028987 12822 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:29.066020  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.050235 12399 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:29.070480  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74751us
I20260709 12:27:29.028115 12796 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.027619 12530 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:28.993798 12611 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:29.103366  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.132035  7734 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:29.176707  7627 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.688s (SENT)
W20260709 12:27:29.182685  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.196868  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.359081  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.362051  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.124380 12924 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.373900  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 825) took 1858 ms. Trace:
I20260709 12:27:29.379042  7809 rpcz_store.cc:276] 0709 12:27:27.515922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.938596 (+422674us) service_pool.cc:224] Handling call
0709 12:27:27.938954 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.046377 (+107423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.373536 (+1327159us) spinlock_profiling.cc:243] Waited 99.8 ms on lock 0x7b54002a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:29.373816 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229566720}
I20260709 12:27:29.379707  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.389135  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.392350  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.392643  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.393550  7812 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:29.397280  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.397571  7803 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:29.400784  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.401051  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.401954  7807 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:29.401204  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.402318  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.403072  8408 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:29.405184  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:29.407275  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:29.409118  7656 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:29.402953  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:29.376241 12857 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.432828  7807 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:29.437532  7808 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:29.438505  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 822) took 2337 ms (client timeout 1878 ms). Trace:
I20260709 12:27:29.448196  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.449242  8408 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:27:29.449787  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.449615  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:29.447150  7804 rpcz_store.cc:269] 0709 12:27:27.091850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.244647 (+152797us) service_pool.cc:224] Handling call
0709 12:27:29.413294 (+2168647us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54001338d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:29.429255 (+ 15961us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420403968}
I20260709 12:27:29.458505  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.459646  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:29.462162 12694 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.394635 12924 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.464181  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.464566 12792 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.465222 12924 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:29.462824 12694 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:29.466580  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.6578e+06us
I20260709 12:27:29.468327 12694 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:29.470145  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.470566 12863 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.957s	user 0.013s	sys 0.000s
W20260709 12:27:29.471395  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 294734us
W20260709 12:27:29.473287  7627 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.728s (SENT)
W20260709 12:27:29.474905  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.865s (SENT)
I20260709 12:27:29.462230  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.477632  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:29.480260  7809 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:29.480589  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.485805  7624 connection.cc:256] RPC call timeout handler was delayed by -1.3132501539998884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 12:27:29.486428  7624 connection.cc:256] RPC call timeout handler was delayed by -1.231350153999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
W20260709 12:27:29.488710  7624 connection.cc:256] RPC call timeout handler was delayed by -1.036050153999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:27:29.489217  7624 connection.cc:256] RPC call timeout handler was delayed by -1.0252501539998775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
I20260709 12:27:29.490965  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.490965  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:29.494946  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.495033  7809 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:29.485522  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.496724  8405 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:27:29.496826  7808 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:29.494596 12792 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.439150 12857 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.479242 12450 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:29.518381  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.519164  8408 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:29.511988  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 134482us
I20260709 12:27:29.471014 12863 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.578814  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.584442  7805 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:29.520081  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:29.590526  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70547.8us
I20260709 12:27:29.577913 12814 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.600768 12814 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.519750 12792 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.594101  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2636) took 1588 ms. Trace:
I20260709 12:27:29.604465  7729 rpcz_store.cc:276] 0709 12:27:28.005575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.005743 (+   168us) service_pool.cc:224] Handling call
0709 12:27:28.022389 (+ 16646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.037217 (+ 14828us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:28.037266 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.053556 (+ 16290us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.359316 (+1305760us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18000a0cc0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:29.535985 (+176669us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:29.536046 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.536160 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.589681 (+ 53521us) raft_consensus.cc:1685] finished
0709 12:27:29.589708 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.594029 (+  4321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1286,"raft.queue_time_us":8575,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":27288,"spinlock_wait_cycles":42064256,"thread_start_us":176452,"threads_started":1,"wal-append.queue_time_us":1513834}
I20260709 12:27:29.594165 12530 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.585085 12796 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:29.595306  7656 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:29.607906 12796 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:29.608712  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:29.610141  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.610867  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:29.613862 12814 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:29.600490  7655 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:29.617973  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.618700  7655 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
W20260709 12:27:29.619536 12926 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.493s	user 0.001s	sys 0.000s
I20260709 12:27:29.597297  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:29.597554  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.619557 12868 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:29.625243  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 921) took 1174 ms. Trace:
I20260709 12:27:29.625746  7653 rpcz_store.cc:276] 0709 12:27:28.451360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.451511 (+   151us) service_pool.cc:224] Handling call
0709 12:27:28.464031 (+ 12520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.532575 (+ 68544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.624900 (+1092325us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b5400435dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:29.625149 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155995008}
I20260709 12:27:29.626993 12792 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.627790  7729 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:29.627954 12857 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:29.630395 12863 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.632077  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.632895  7729 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:29.525050 12759 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:27:29.634698  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.635552  7733 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:29.615489 12690 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:29.636529 12690 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.636981 12690 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:29.622829  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:29.639369  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.640727  7655 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:29.598254  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.642401  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.643119  7729 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:29.593330 12822 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.598747  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.599565  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.644353  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:29.643962 12822 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:29.645499  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46845.7us
I20260709 12:27:29.641574  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.647751  7733 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:29.647948  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.649309 12792 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:29.652524 12792 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.653132 12792 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:29.597784 12752 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:29.581290 12450 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.658299 12450 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:29.649149  7731 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:29.647089  7777 leader_election.cc:400] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:29.668789  7777 leader_election.cc:403] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:29.674512  7810 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:29.688508  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.659402 12450 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:29.667712 12822 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:29.693882  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46996.3us
I20260709 12:27:29.698225 12752 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:29.705802  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:29.701474 12863 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:29.701568 12923 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.701882 12868 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:29.667572 12399 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.710842 12923 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.695380 12450 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.715654  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:29.721630  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.723273  8411 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:29.709879 12935 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:29.715693 12450 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:29.731900  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.718653 12604 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.745369  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:29.747345  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.727250  8405 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:29.727451 12868 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:29.712062 12863 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:29.760061  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 285161us
I20260709 12:27:29.730412 12935 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.712787 12792 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:29.766268  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:29.772150  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.977s (SENT)
I20260709 12:27:29.798445  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.799865  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:29.784832  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 293930us
I20260709 12:27:29.766390 12814 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:29.798095  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.811671  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.759785 12857 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:29.773116 12935 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.817148  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:29.819875  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.820643  8405 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.817040  8407 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:29.760437 12604 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:29.821657  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50878us
I20260709 12:27:29.824347  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.805949 12680 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:29.833452 12680 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:29.833946 12680 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:29.829582  7658 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:29.825429 12927 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:29.806282 12868 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:27:29.811671 12814 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.832145 12776 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:29.832659  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:29.774251 12930 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 12:27:29.869370  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97279.4us
W20260709 12:27:29.850066  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2878) took 2386 ms (client timeout 2076 ms). Trace:
W20260709 12:27:29.870314  7806 rpcz_store.cc:269] 0709 12:27:27.463461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.581101 (+117640us) service_pool.cc:224] Handling call
0709 12:27:29.849756 (+2268655us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400295754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:29.849960 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2924465920}
I20260709 12:27:29.875082 12792 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:29.870985  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.179s (SENT)
I20260709 12:27:29.890906  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:29.891786  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67531.3us
I20260709 12:27:29.884146  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:29.818392 12936 raft_consensus.cc:2764] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:29.909489 12863 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:29.881345 12694 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:29.839802 12927 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.870078 12814 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.898658  8407 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:29.917073  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:29.895735  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:29.910872 12694 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:29.906124  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.911793  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:29.897868  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 86283.5us
I20260709 12:27:29.883929  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:29.888748 12949 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.057s	user 0.003s	sys 0.001s
I20260709 12:27:29.919126  8405 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:29.890296 12941 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:27:29.937625  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.815115 12929 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:27:29.930152  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:29.940467  7736 raft_consensus.cc:2376] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11. Re-sending same reply.
W20260709 12:27:29.931975  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.931847  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:29.934402  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:29.922776  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.938966  7735 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:29.905941 12792 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:29.946175 12792 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.906762 12930 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:29.951383 12930 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:29.951759 12694 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.952442 12930 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:29.953244 12694 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:29.953810 12930 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:29.955610  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:29.956384  7731 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:29.963331  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92288us
I20260709 12:27:29.963439 12943 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:29.942494  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.949355 12929 consensus_queue.cc:1243] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:29.949383  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:29.950202  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.950202  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:29.946838 12792 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:29.916920 12927 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.967805  7733 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:30.063164  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54239.2us
W20260709 12:27:30.060355  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96516.3us
W20260709 12:27:30.060778  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.42115e+06us
W20260709 12:27:30.052274  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.091861  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149469us
W20260709 12:27:30.094282  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144945us
I20260709 12:27:29.997759  7731 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:30.108620  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.109581  7657 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:30.113837  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:30.115141  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:30.104979  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:29.971259 12779 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:30.124338  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:30.127677  7735 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:27:30.128019  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2747) took 1701 ms. Trace:
I20260709 12:27:30.128543  7732 rpcz_store.cc:276] 0709 12:27:28.426593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.830136 (+403543us) service_pool.cc:224] Handling call
0709 12:27:28.830514 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.112663 (+1282149us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:30.112715 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.127632 (+ 14917us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:30.127703 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.127792 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.127951 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1281601,"raft.queue_time_us":1288019}
W20260709 12:27:30.120885  7627 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.552s (SENT)
I20260709 12:27:30.132212  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.134562  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:29.970472 12604 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:30.136758  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.137435  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.137848  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.139413  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.140498  7653 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:29.879148 12680 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:30.153959  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 311627us
I20260709 12:27:30.134405  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.151289  7736 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:30.155848  7655 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:30.151737  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.152022  7737 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:27:30.159986  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.118788 12739 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.179898  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 59067us
I20260709 12:27:30.135638 12779 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:30.185945  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81072.3us
I20260709 12:27:30.149900 12851 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:30.188030  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:30.188894  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64652.5us
I20260709 12:27:30.166958 12739 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.168951 12863 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:30.194051  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:30.174954 12857 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:30.190882 12587 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:30.206993  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.185809 12779 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:30.217381  7733 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:30.217689 12739 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:30.220131  7736 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:30.221345  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:30.222049  7736 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:30.222793  7737 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:30.223268  7737 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:30.223532  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:30.224126  7737 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.226496  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.227118  7803 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.194718 12943 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:30.229066  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:30.229871  7737 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.231304  7735 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:30.199626 12863 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.205807 12929 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:30.238409  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.250481 12777 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:30.214143 12587 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.254455 12587 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.257205 12954 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:30.257644 12399 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.257803 12954 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.258337 12954 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.250506 12857 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.250519  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.202893 12851 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.244130 12929 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.272861 12929 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:30.266443 12739 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:30.243099 12952 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:30.279867 12952 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:30.279981 12587 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:30.280997 12952 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.233549 12956 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:30.290865 12690 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.653s	user 0.016s	sys 0.000s
I20260709 12:27:30.291467 12957 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:30.240836 12863 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:30.298501  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 817) took 3298 ms (client timeout 3150 ms). Trace:
I20260709 12:27:30.301434 12923 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.303517  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "629e345c827545598250d02a9f39e2fb"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:27:30.341634  7811 rpcz_store.cc:269] 0709 12:27:27.005373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.009067 (+  3694us) service_pool.cc:224] Handling call
0709 12:27:30.298273 (+3289206us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:30.298404 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3618823936}
I20260709 12:27:30.343541  7804 raft_consensus.cc:606] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.308796 12851 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:30.328641  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.352131  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80300.1us
I20260709 12:27:30.353039  7737 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:30.355316  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 814) took 2163 ms. Trace:
I20260709 12:27:30.355818  8415 rpcz_store.cc:276] 0709 12:27:28.192033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.201808 (+  9775us) service_pool.cc:224] Handling call
0709 12:27:28.220803 (+ 18995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.422364 (+201561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.354966 (+1932602us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:30.355165 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461867264}
W20260709 12:27:30.365763 12935 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.003s
I20260709 12:27:30.366269 12935 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:30.368597 12450 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.002s
I20260709 12:27:30.369256 12935 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:30.369195 12450 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.319144 12814 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.372682  7809 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:30.373827 12814 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.341677 12690 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.377442  7804 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:30.377751 12690 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.378512  7809 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:30.379024  7804 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:30.380100  7809 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:30.397126  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:30.397861  7729 raft_consensus.cc:2376] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12. Re-sending same reply.
I20260709 12:27:30.400192 12927 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.400879  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:30.403081  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49909.5us
I20260709 12:27:30.403530  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.404379  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:30.362504 12792 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.406185  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 837) took 1036 ms. Trace:
I20260709 12:27:30.406675  7805 rpcz_store.cc:276] 0709 12:27:29.369615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.388734 (+ 19119us) service_pool.cc:224] Handling call
0709 12:27:30.405950 (+1017216us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400331b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:30.406112 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444763008}
I20260709 12:27:30.381913 12779 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.383750  7812 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:30.384130 12957 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:30.385699  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.387742  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 503894us
W20260709 12:27:30.389111  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.389724 12604 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:30.391078  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 163983us
I20260709 12:27:30.392037 12680 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:30.411033 12450 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:30.407748  7805 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:30.463209  7805 raft_consensus.cc:606] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.465723  7805 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:30.407840  7811 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:30.467075  7803 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:30.468822 12927 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.417812 12944 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:27:30.470427 12944 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:30.420933  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:30.471913  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51097us
I20260709 12:27:30.474025 12930 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:30.474504  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:30.417812 12776 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:27:30.475087 12930 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:30.475946 12925 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:30.477717 12925 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:30.478726 12925 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:30.478832 12930 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:30.479324 12776 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:30.479493 12930 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:30.480003 12776 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:30.411716 12957 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.481092 12930 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.481302 12776 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.483002 12930 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:30.483366 12745 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:30.470839 12944 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:30.408191 12851 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:30.482920 12776 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:30.487309 12851 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:30.488070 12851 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:30.487953 12944 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.489689  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:30.489782  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:30.490779  8413 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.427101 12694 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.498432 12936 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:30.428153  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:30.500010  8415 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:30.490851 12925 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.428872 12928 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.430183 12611 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.430629  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.502441  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:30.431527  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.506279  7658 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:27:30.506628  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 312637us
I20260709 12:27:30.432273  8379 leader_election.cc:400] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14.
I20260709 12:27:30.407176  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:30.508201  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76068.5us
I20260709 12:27:30.432350  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:30.509572  8407 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.433149 12650 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.435391  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2891) took 1064 ms. Trace:
I20260709 12:27:30.437129  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.513803  7812 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.444134  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:30.445300  7810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2881) took 2980 ms (client timeout 1883 ms). Trace:
W20260709 12:27:30.517750  7810 rpcz_store.cc:269] 0709 12:27:27.464492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.540111 (+ 75619us) service_pool.cc:224] Handling call
0709 12:27:30.445057 (+2904946us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54003f4b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:30.445212 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4844158848}
I20260709 12:27:30.446206  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:30.445978 12954 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.456310 12929 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.460865  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.479444 12898 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:30.407721 12792 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.413544 12779 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.430040 12863 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.497429 12822 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.512576  7808 rpcz_store.cc:276] 0709 12:27:29.371021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.450122 (+ 79101us) service_pool.cc:224] Handling call
0709 12:27:29.496532 (+ 46410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.497570 (+  1038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.435079 (+937509us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54003728d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:30.435313 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54560512}
I20260709 12:27:30.526341  7734 raft_consensus.cc:2368] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
W20260709 12:27:30.551990  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144901us
W20260709 12:27:30.527742  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 339807us
I20260709 12:27:30.554309  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.527331  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:30.529076  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:30.527933  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.557014  7811 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:30.501194 12925 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:30.558220  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.514353 12771 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:30.554556  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:30.563298  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.578768  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:30.473784 12927 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:27:30.572738  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:30.564162 12777 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:27:30.512140 12796 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:27:30.618598  7809 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:30.534087 12898 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:30.603873  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.609s (SENT)
I20260709 12:27:30.583003  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:30.629525  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:30.616602  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62155.5us
I20260709 12:27:30.550181 12904 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:30.551384 12944 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.636324  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.645401  7661 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:30.575469  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:30.637481  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54578.2us
I20260709 12:27:30.585059  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:30.654296  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.655553  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.656680  7657 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:30.660565  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.661790  7657 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:30.599695 12897 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:30.628077 12932 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:27:30.664417  7810 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:30.664697  7653 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:30.632556 12771 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:30.664446  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60656us
W20260709 12:27:30.668592  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 93262.2us
W20260709 12:27:30.669001  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.700s (SENT)
I20260709 12:27:30.669289  7811 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:30.669591  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.571590 12759 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:30.672482  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.677239  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.678160  7658 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 9: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:30.584100 12894 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:30.673108  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.583833 12779 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:30.676627  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.683151  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.551939 12680 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:27:30.583364 12928 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:30.673907  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.602778 12792 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:30.687783  7656 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:30.687539  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.682049  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:30.694139  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.644589 12749 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:30.695376  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:30.695160  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 460036us
I20260709 12:27:30.676501 12935 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:30.700103  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.708529 12863 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.682926 12759 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.711619 12759 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.713589  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:30.717010  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.717962  7655 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:30.583971 12943 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 12:27:30.722640  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.738485  7656 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:30.688701 12739 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:30.679510 12811 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.586722  7555 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 12, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 12 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:30.695106  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.874s (SENT) [suppressed 51 similar messages]
I20260709 12:27:30.686004 12822 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:30.739538  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.773398  7656 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:30.740247  7657 pending_rounds.cc:85] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 6
I20260709 12:27:30.767861  7653 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
I20260709 12:27:30.713860  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.691008 12530 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:30.774688  7658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.774750  7657 pending_rounds.cc:107] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:30.772492  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.743443  7660 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:27:30.772935  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50307.2us
I20260709 12:27:30.777801  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:30.787271  7807 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:30.722882 12956 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:30.771188  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76066.4us
I20260709 12:27:30.772186  7812 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:30.775483  7657 raft_consensus.cc:2889] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:30.742309  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.676913 12771 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.817819  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:30.824054  7653 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:30.832012 12963 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:30.832710 12963 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.833191 12963 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:30.780579 12811 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:30.840881 12929 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.841815 12929 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:30.841928  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 141927us
I20260709 12:27:30.842356 12929 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:30.842898 12929 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.852219 12811 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:30.864177 12822 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:30.855993 12954 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:30.859902  7659 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:30.872222  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 54495.4us
W20260709 12:27:30.883790  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.909466  7624 leader_election.cc:400] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:27:30.909920  7624 leader_election.cc:403] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:30.801453 12857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.015s	sys 0.017s
W20260709 12:27:30.844700  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55900.8us
I20260709 12:27:30.848353  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.917450  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.919055  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.920275  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:30.921607  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.922650  7812 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.848011 12923 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:30.854108  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.781545 12739 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.874850  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:30.889828  8413 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:30.772755 12898 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:30.896064  7655 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:30.782567 12749 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:30.911402  8407 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:30.933813  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:30.950325  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:30.955092  7660 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:30.955334  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:30.958374  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:30.960261  7810 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:30.924662 12857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.016s	sys 0.018s
I20260709 12:27:30.987545  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:30.926460 12604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.015s	sys 0.013s
I20260709 12:27:30.923707 12822 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:31.000274  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 406930us
I20260709 12:27:30.938220 12739 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:30.997684  7809 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:31.004330  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:31.008687  7812 raft_consensus.cc:2410] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:30.873338 12851 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:31.015969  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 161938us
I20260709 12:27:31.017561  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:31.001884 12941 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:27:31.019634  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:31.015980  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 106709us
W20260709 12:27:31.013386  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.026192  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:31.028002  7812 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:30.990854 12863 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:30.998692 12604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.015s	sys 0.013s
I20260709 12:27:30.999469 12956 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:31.056841  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 211726us
I20260709 12:27:31.059674  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:31.014737 12930 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
I20260709 12:27:31.060815  7731 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:31.057140  7659 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:31.061548  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44118us
I20260709 12:27:31.018728 12904 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.003512 12822 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:30.998240 12954 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:31.054924 12943 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:31.066210  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:30.988235 12952 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:27:31.041599  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:30.997334 12967 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
W20260709 12:27:31.111985  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 70479.5us
I20260709 12:27:31.070817 12863 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.113760  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.069214 12965 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:31.080353 12956 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.012198 12966 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:27:31.060228 12657 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:27:31.093889 12530 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.159204 12981 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:31.159936 12981 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.160331 12981 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:31.159261 12530 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.172271 12936 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:31.172869 12936 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:31.116942 12954 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:31.173866 12936 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.085686 12604 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:27:31.087925 12749 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.178861 12792 raft_consensus.cc:993] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:31.083952 12904 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.179597 12792 raft_consensus.cc:1081] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:31.182096 12904 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:31.187989  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2894) took 1788 ms. Trace:
W20260709 12:27:31.189199 12962 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:31.133622 12750 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.190248  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:31.190169 12750 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.191511 12927 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:31.191515 12976 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:27:31.192494  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.192821 12750 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:31.194463  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:31.195482  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:31.195775  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74670.1us
I20260709 12:27:31.196599 12811 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:31.197180 12450 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:31.197376 12927 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:31.197912 12927 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.198328 12927 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:31.198062 12450 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:31.197214 12811 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.199635 12967 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:31.200137 12967 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:31.200210 12851 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:31.203557  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89895.4us
I20260709 12:27:31.175725 12604 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:27:31.143668 12971 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:27:31.209512  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:31.188668  7807 rpcz_store.cc:276] 0709 12:27:29.399854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.460730 (+ 60876us) service_pool.cc:224] Handling call
0709 12:27:30.409291 (+948561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.787778 (+378487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.186569 (+398791us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400331b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:31.187717 (+  1148us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400331b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:31.187908 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2975301248}
I20260709 12:27:31.152858 12399 raft_consensus.cc:993] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:31.155983 12956 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.174633 12863 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.175948 12936 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:31.175897 12960 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:31.121135 12924 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:31.139971 12969 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:31.208069  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.218147  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:31.229487  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:31.260114  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55669.7us
I20260709 12:27:31.233124  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:31.223767  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:31.135571  7735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.006s	sys 0.022s
W20260709 12:27:31.120602  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54497.1us
W20260709 12:27:31.262007  7735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.007s	sys 0.022s
I20260709 12:27:31.233590  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:31.232394  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:31.264192  7653 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:31.264312  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:31.232787  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:31.187196 12611 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.661s	user 0.005s	sys 0.005s
I20260709 12:27:31.272413  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:31.275977  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:31.275454  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57367.5us
I20260709 12:27:31.278343  7660 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:31.200570 12811 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:31.231609 12925 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.651s	user 0.008s	sys 0.003s
I20260709 12:27:31.309998  8409 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:31.233340 12796 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:31.305408  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:31.314168  8415 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:31.314539  8413 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:31.306545 12985 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:31.282382 12944 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:31.237356 12739 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.331689  8406 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:31.309710 12925 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:31.270627 12968 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
I20260709 12:27:31.231544 12954 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.239513 12966 raft_consensus.cc:1081] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:27:31.262156 12960 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:31.346410  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 287021us
I20260709 12:27:31.311688 12611 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:27:31.302982  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:31.311010  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:31.389600  7660 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:27:31.303089 12943 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:31.313567 12796 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:31.331965 12779 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.402531  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:31.249943 12963 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:31.347584  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 289312us
I20260709 12:27:31.369401  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:31.369684  8405 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:27:31.408479  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 211787us
W20260709 12:27:31.321954  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9911840200002189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 12:27:31.411620  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9704840200001854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 12:27:31.339397 12944 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:31.392238  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89343.8us
I20260709 12:27:31.346175 12985 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.400556  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:31.383831  8379 leader_election.cc:336] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.787s (SENT)
I20260709 12:27:31.423005  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:31.412591  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9101840200000879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 12:27:31.390743  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2383) took 1282 ms. Trace:
W20260709 12:27:31.423786  7702 connection.cc:256] RPC call timeout handler was delayed by -1.868784020000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 12:27:31.424235  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7742840200003229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 12:27:31.424296  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:31.390202 12650 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.864s	user 0.013s	sys 0.001s
I20260709 12:27:31.419159  8406 raft_consensus.cc:2368] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14.
W20260709 12:27:31.425139  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7598840200002996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
I20260709 12:27:31.415170  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:31.428006  7811 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:31.428439  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6509840200001236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
I20260709 12:27:31.333792 12927 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:31.428848  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6104840200000581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
W20260709 12:27:31.429365  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6059840200000508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260709 12:27:31.429858  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5456840200004081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 12:27:31.430364  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5042840200003411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 12:27:31.430745  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4547840200002611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 12:27:31.431118  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4295840200002203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 12:27:31.431500  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4205840200002058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 12:27:31.431862  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3512840200000937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22660000000000002s for a response
W20260709 12:27:31.426730  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 128956us
I20260709 12:27:31.437451  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:31.438663  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:31.437040  7777 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:31.443002  8412 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:31.445848  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.346585 12954 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.424000  7654 rpcz_store.cc:276] 0709 12:27:30.108729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.110418 (+  1689us) service_pool.cc:224] Handling call
0709 12:27:30.154777 (+ 44359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.202377 (+ 47600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.202426 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.239518 (+ 37092us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.713096 (+473578us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:30.713146 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.721185 (+  8039us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.390522 (+669337us) raft_consensus.cc:1685] finished
0709 12:27:31.390545 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.390682 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73273,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":12578,"thread_start_us":414452,"threads_started":2,"wal-append.queue_time_us":741427}
I20260709 12:27:31.457626  7660 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:27:31.460098  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:31.462237  7778 leader_election.cc:400] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13.
I20260709 12:27:31.471799  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:31.444768 12986 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 12:27:31.444274 12894 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.003s
I20260709 12:27:31.418146 12611 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:27:31.475701 12944 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.477037 12739 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:31.477726 12980 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:27:31.478654 12994 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:31.479477 12796 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.481652 12938 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.484426 12759 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.772s	user 0.009s	sys 0.005s
I20260709 12:27:31.486541 12927 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:31.487977 12978 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:27:31.462515 12857 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:31.491819  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46074.7us
I20260709 12:27:31.579735  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:31.492699 12993 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:31.456012 12957 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:31.419579 12955 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:31.491915 12985 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.495811 12694 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.496296  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 112539us
I20260709 12:27:31.415469 12943 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:31.313315 12967 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.626488 12967 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:31.508020  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46005.7us
W20260709 12:27:31.509538  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.502779 12929 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.001s
I20260709 12:27:31.632452 12929 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.511291  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2387) took 1388 ms. Trace:
W20260709 12:27:31.511271  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 106555us
I20260709 12:27:31.422278 12779 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.497917 12923 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:31.516687  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:31.497768 12530 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:31.501449 12814 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:31.532431  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:31.510869 12982 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:31.639461  8409 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:31.638535  7810 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:31.508977 12604 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:31.582898 12796 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:31.571105 12938 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.521176 12894 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:31.612877  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:31.624524 12963 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:31.639668 12929 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:31.638188  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:31.641614  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.588554 12954 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:31.641461  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:31.640123  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:31.639750  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:31.651094  7736 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:31.594710  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:31.656311  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 203960us
I20260709 12:27:31.636588  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1060) took 1095 ms. Trace:
W20260709 12:27:31.658623  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.032s (SENT)
W20260709 12:27:31.641626  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.638002  7654 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:31.657320  7735 rpcz_store.cc:276] 0709 12:27:30.541040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.541235 (+   195us) service_pool.cc:224] Handling call
0709 12:27:30.541593 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.544431 (+  2838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.544487 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.545278 (+   791us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.262521 (+717243us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:31.262563 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.262633 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.636244 (+373611us) raft_consensus.cc:1685] finished
0709 12:27:31.636276 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.636482 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38385,"raft.queue_time_us":3544,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":12707,"thread_start_us":664951,"threads_started":1,"wal-append.queue_time_us":611734}
I20260709 12:27:31.669871  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.670423  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.671520  7729 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:31.579082 12857 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:31.590442 12739 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:31.635142  7662 rpcz_store.cc:276] 0709 12:27:30.123059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.123194 (+   135us) service_pool.cc:224] Handling call
0709 12:27:30.164143 (+ 40949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.232076 (+ 67933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.232110 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.232648 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.490967 (+258319us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:30.491036 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.491165 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.246249 (+755084us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:31.506669 (+260420us) raft_consensus.cc:1685] finished
0709 12:27:31.506707 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.511197 (+  4490us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82852,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":3753216,"thread_start_us":265495,"threads_started":2,"wal-append.queue_time_us":634818}
I20260709 12:27:31.686237  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.674696  7728 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 9.
W20260709 12:27:31.697410  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84878.2us
I20260709 12:27:31.699992  8409 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:31.697520  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102926us
I20260709 12:27:31.701184  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:31.709971  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51419.2us
I20260709 12:27:31.711956  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.712983  7732 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:31.655894 12680 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.578574 12771 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 12:27:31.575560 12759 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.733379  7808 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:31.717967  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.734371  7735 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:27:31.641741 13001 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.649818 12530 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.739717 13001 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.740684 12530 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.729243  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.745473 12759 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:31.746208  7728 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:27:31.742915 13001 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:31.649720 12923 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.747289 13001 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:31.764230  7735 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:31.765901  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.769418  7735 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:31.772871  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:31.776119  7735 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:31.781858  7735 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:27:31.785458  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.792505  7735 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:31.777086 12923 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:31.811347  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.697885 12857 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:31.697106 12993 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:31.837050  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136586us
W20260709 12:27:31.822256  7700 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.691s (SENT)
I20260709 12:27:31.823344  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:31.708395 12776 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:31.690326 12957 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.826371  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:31.828950  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 127877us
I20260709 12:27:31.833123 12898 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:31.654978 12694 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.819352 12851 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:31.702363 12739 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:31.626700 12943 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:31.835918  7811 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:31.896502  7736 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:31.697651 12925 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:31.854374  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.858692  7728 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:31.819598 12611 consensus_queue.cc:1243] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:31.903309  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.912724  7728 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:31.912895  7627 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.646s (SENT)
I20260709 12:27:31.865995  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:31.819779 12771 consensus_queue.cc:1243] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:31.873908  7729 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:31.876317  7732 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:31.897888  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:31.866180 12999 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
W20260709 12:27:31.913866  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.715s (SENT)
I20260709 12:27:31.923393  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:31.915191  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:31.926527  7729 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:31.915465  7736 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:31.915719  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:31.928220  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:31.917040  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51177.4us
I20260709 12:27:31.937542  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:31.940491  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:31.985355  7662 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:31.818984 12680 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:31.874096 12955 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:31.965687 12979 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:27:31.899103 12739 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:31.966427  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52565.4us
W20260709 12:27:32.008147  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70702.4us
W20260709 12:27:32.008414  7627 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.829s (SENT)
I20260709 12:27:32.009302  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:31.974962  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 120729us
I20260709 12:27:32.010890  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:31.899295 13023 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.921451 12814 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:31.942790 12986 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:32.001515 12680 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:32.012732  7728 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:32.013517  7729 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:31.968267 12993 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:32.036726  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2390) took 1365 ms. Trace:
I20260709 12:27:32.021967  7662 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:32.035207  7736 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:32.014730  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:32.054184  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:32.054916  7662 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.037339  7656 rpcz_store.cc:276] 0709 12:27:30.671643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.787236 (+115593us) service_pool.cc:224] Handling call
0709 12:27:30.787641 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:30.865336 (+ 77695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.865372 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:31.213117 (+347745us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:31.251924 (+ 38807us) log.cc:844] Serialized 52 byte log entry
0709 12:27:31.440189 (+188265us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:31.440235 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.440322 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.004737 (+564415us) raft_consensus.cc:1685] finished
0709 12:27:32.004762 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.036626 (+ 31864us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7247,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":150,"spinlock_wait_cycles":29313920,"thread_start_us":187304,"threads_started":1,"wal-append.queue_time_us":300905}
W20260709 12:27:32.064128 13004 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:27:32.065155  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.053650 12955 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:27:32.069655 12779 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:32.080858  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70113.8us
I20260709 12:27:32.081190 12779 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:32.081956 12779 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.082320  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.087576  7728 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:27:32.087963  7654 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:32.091753  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83408.5us
I20260709 12:27:31.980969 12923 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:27:32.103276  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.036798 12851 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:32.038515  7736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "629e345c827545598250d02a9f39e2fb"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
I20260709 12:27:32.111939  7736 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:32.112485  7736 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.080976 12955 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:32.123308  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 437508us
I20260709 12:27:32.083458 13012 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:32.128155  7736 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.136572  7728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
I20260709 12:27:32.108253 12993 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:32.083031 12779 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.159119  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:32.170468  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77678.4us
I20260709 12:27:32.161386  7662 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:32.120577 13008 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:32.122015 12955 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.177107  7736 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:32.164863 12993 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:32.171540  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.862s (SENT)
I20260709 12:27:32.021840 12957 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.159644 12952 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:32.183988  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.202224  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:32.208226  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.134s (SENT)
I20260709 12:27:32.183141 12955 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:32.214479 13024 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:32.215991 12952 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:32.230602 13015 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:27:32.210700 13012 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:32.165989 12954 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:27:32.214648  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:32.158963 12771 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 12:27:32.257994  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.287s (SENT)
W20260709 12:27:32.226539 12996 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
W20260709 12:27:32.227749 13009 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:27:32.156348 12814 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.192337 13017 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:32.208882 12694 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:32.226380 13020 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:27:32.295712 12530 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:27:32.268817  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58367us
W20260709 12:27:32.285908 12822 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.180s	user 0.021s	sys 0.001s
W20260709 12:27:32.298225 13014 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.057s	user 0.000s	sys 0.009s
W20260709 12:27:32.292438 13019 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.168s	user 0.000s	sys 0.004s
W20260709 12:27:32.293179 13011 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.132s	user 0.002s	sys 0.002s
I20260709 12:27:32.199149 12611 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 12:27:32.295792  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.290452 12796 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.649s	user 0.014s	sys 0.000s
I20260709 12:27:32.312367 12739 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.303510  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2388) took 1633 ms. Trace:
I20260709 12:27:32.366353  7657 rpcz_store.cc:276] 0709 12:27:30.669932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.739364 (+ 69432us) service_pool.cc:224] Handling call
0709 12:27:30.739881 (+   517us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:30.855708 (+115827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.855742 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:31.149222 (+293480us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:31.150299 (+  1077us) log.cc:844] Serialized 78 byte log entry
0709 12:27:31.366434 (+216135us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:31.366510 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.366631 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.303086 (+936455us) raft_consensus.cc:1685] finished
0709 12:27:32.303109 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.303434 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36908,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":36194176,"thread_start_us":215462,"threads_started":1,"wal-append.queue_time_us":419868}
W20260709 12:27:32.312314 12981 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.151s	user 0.010s	sys 0.006s
I20260709 12:27:32.268661 13024 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.321875 12822 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:32.318413  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 215264us
I20260709 12:27:32.333940  8405 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:32.348259  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2914) took 1610 ms. Trace:
I20260709 12:27:32.372117  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.372020  7810 rpcz_store.cc:276] 0709 12:27:30.737972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.988629 (+250657us) service_pool.cc:224] Handling call
0709 12:27:30.989081 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.640616 (+651535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.347906 (+707290us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400134cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:32.348157 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492953344}
W20260709 12:27:32.318874  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61009.2us
I20260709 12:27:32.318552  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:32.360359  7728 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:32.300396 12993 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:32.341602 12936 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.068s	user 0.008s	sys 0.008s
I20260709 12:27:32.375535  7728 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:32.318143 13003 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.374s	user 0.001s	sys 0.001s
W20260709 12:27:32.377395  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58948.5us
W20260709 12:27:32.332432 12967 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.006s
I20260709 12:27:32.030373 13023 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.369450 12981 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.312297 12530 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.387902  7735 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:32.402613  7729 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:32.405261  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.391644  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:27:32.389402 12981 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:32.390503 13000 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:27:32.396260 12530 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.413317  7658 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:27:32.413725 12822 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.386952 13023 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.379890 13029 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:32.387665 12739 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.393752 12952 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.398038 12993 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.415683  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2402) took 1198 ms. Trace:
W20260709 12:27:32.421736  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.894s (SENT)
I20260709 12:27:32.419582  7655 rpcz_store.cc:276] 0709 12:27:31.217053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.217328 (+   275us) service_pool.cc:224] Handling call
0709 12:27:31.217941 (+   613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.258751 (+ 40810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.258799 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.265971 (+  7172us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.535174 (+269203us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:31.535287 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.535404 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.415422 (+880018us) raft_consensus.cc:1685] finished
0709 12:27:32.415447 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.415610 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40114,"raft.queue_time_us":40880,"spinlock_wait_cycles":20571520,"thread_start_us":215979,"threads_started":1,"wal-append.queue_time_us":523702}
W20260709 12:27:32.423302  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.080s (SENT)
I20260709 12:27:32.425247  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.432677  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.435623  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.437085  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 938) took 1053 ms. Trace:
I20260709 12:27:32.437649  7661 rpcz_store.cc:276] 0709 12:27:31.383981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.082402 (+698421us) service_pool.cc:224] Handling call
0709 12:27:32.082792 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.083037 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:32.288383 (+205346us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.436698 (+148315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:32.436755 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:32.436798 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.436887 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.437019 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2653,"prepare.queue_time_us":87760,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":7332,"thread_start_us":349751,"threads_started":2}
I20260709 12:27:32.445987  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.383634 13034 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:32.446250  7728 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.458352  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.432878 12985 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:32.440265 12739 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:32.445457 12857 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.000s
I20260709 12:27:32.400204 12690 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:27:32.475570 13034 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.476756 13034 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.479969  7655 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:32.480427  7655 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:32.467052 12985 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.481462  7655 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.370217 13024 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.483789  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.484941  7657 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:32.486572  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.456750 13029 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:32.488204 13029 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:32.488590  7659 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:32.479631 12739 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:32.489565 12960 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.443343 12796 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:32.492650 12955 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.424201 12993 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.442646 12851 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:32.498459 12985 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:32.470896 12857 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.490227  7662 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:32.487407  7660 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:32.506340  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.483058  7655 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.530324 12960 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:32.530015  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71792.1us
I20260709 12:27:32.554963  7655 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:32.561066  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:32.563922 12739 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:32.565366 12739 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:32.565887 12739 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:32.567538 12982 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:32.568202 12982 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:32.538928 12993 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:32.569617 12959 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:27:32.569191 12982 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.569806 12993 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.570600 12982 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:32.572661 12993 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.536999 12796 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:27:32.574180 12796 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:32.574622 12796 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:32.575722 12796 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.576529 13035 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:32.577088 12796 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.579166  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.579619 12851 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:32.580016  7736 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:32.580276  7810 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:32.580396 12851 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:32.581262  7736 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:32.581900  7810 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:32.583449  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:32.584257  7805 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:32.586501  7736 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:32.587069  7736 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:32.567528 12739 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.592005  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:32.595469  7736 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.597144  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:32.598896  7736 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.601059  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.603883  7804 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:32.604851 12957 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:32.605561 12957 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.609663  7809 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:32.595356 12959 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:27:32.630954 12957 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.577109 13035 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:32.594907 12993 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:32.647403  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.725s (SENT)
I20260709 12:27:32.648165  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:32.676856  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.615741 12851 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:32.688849  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.690613  7732 raft_consensus.cc:2376] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13. Re-sending same reply.
I20260709 12:27:32.606765 12965 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 12:27:32.691938  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83815.6us
I20260709 12:27:32.692034 12739 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:32.693400  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.699071  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.610169 12935 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:27:32.651710 12967 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:32.703730  7732 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:32.708052  7736 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:32.709076  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.709725  7736 raft_consensus.cc:2376] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9. Re-sending same reply.
I20260709 12:27:32.710860  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.708992  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.711660  7732 raft_consensus.cc:2376] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10. Re-sending same reply.
I20260709 12:27:32.712647  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.713676  7736 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:32.689882 12851 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:32.716071  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.718114  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.719615  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.720659  7732 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:32.721756  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.707676 12935 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:32.719136  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:32.725289  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.736879  7732 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:27:32.726207  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:32.727068 12984 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.382s	user 0.002s	sys 0.002s
W20260709 12:27:32.716066 13028 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.485s	user 0.004s	sys 0.003s
I20260709 12:27:32.696476 12959 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:27:32.717761 12851 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:32.747687  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.742956 12857 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:32.722688 12792 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:27:32.752019 12959 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:32.752138  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:32.752660 13053 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:27:32.752063 13050 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:27:32.753010  7732 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:32.753901 13054 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:27:32.754419  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.755815  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:32.774696  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.107s (SENT)
I20260709 12:27:32.781857  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.745841 12530 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:32.798640  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.799651  7736 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:32.802920  7732 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:32.803936  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.804577  7732 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:32.752704 12959 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:32.750344 13035 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.800570 12965 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:32.723551 12814 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:27:32.824445  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.825433  7736 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:32.830783  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.838593 12811 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.839284 12811 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.846239  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.847558  7737 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:27:32.849784  7737 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:32.851326  7737 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:32.851755 13035 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:32.852802  7737 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:32.853196 12959 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.854830 12959 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:32.798974 12530 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.857147 12530 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:32.844583  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.795887 12857 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:27:32.865753 12935 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.870383  7803 pending_rounds.cc:85] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 5
W20260709 12:27:32.874737  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.03673e+06us
W20260709 12:27:32.873684  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.889s (SENT)
I20260709 12:27:32.877300  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.867264 12967 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.874192  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.884119  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:32.885360  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.886149  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.888110  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:32.878831  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.868448 12450 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.869078 12611 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.891651  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 834) took 1509 ms. Trace:
I20260709 12:27:32.874794  7803 pending_rounds.cc:107] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:32.892969  7803 raft_consensus.cc:2889] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:32.895305  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 967) took 1609 ms. Trace:
I20260709 12:27:32.895706  8407 rpcz_store.cc:276] 0709 12:27:31.281858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.283131 (+  1273us) service_pool.cc:224] Handling call
0709 12:27:31.283578 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.283873 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:31.673559 (+389686us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.890304 (+1216745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:32.890365 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:32.890415 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.890485 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.890625 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943776,"prepare.queue_time_us":281282,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":76,"thread_start_us":490523,"threads_started":2,"wal-append.queue_time_us":404443}
W20260709 12:27:32.899019  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 161369us
I20260709 12:27:32.899587  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:32.905838  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:32.920212  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.921329  7733 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:32.894466 12822 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:32.922730  7736 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:32.906327 12650 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:32.932934  7736 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:27:32.941622  7733 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:27:32.889474 12935 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.894667 12967 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.943424  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:32.950080  7733 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:32.950728  7733 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:32.906399 12611 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.926251 12811 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:32.957540  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.960894  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:32.965055  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.965238  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:32.966279  7660 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:32.966356  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.973724  7661 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:32.979616  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:32.980852  7661 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:27:32.982149  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.314s (SENT)
I20260709 12:27:32.982965  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:32.986057  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:32.989128  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:32.990144  7805 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:32.990537  7805 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:32.945888  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:32.994912  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:32.901319 12450 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:32.982141  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:33.005264  7660 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:33.005777  7655 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:33.007268  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:33.010370  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.011767  7812 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:33.015406  7807 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:33.026201  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:32.945506 12928 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.040269  7662 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:33.049155  7655 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:33.012866  7733 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.053810  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.055505  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.056573  7812 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:32.953050 12898 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.967823 12932 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.076610  7733 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:32.937119 12650 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:33.080600  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 98508.1us
I20260709 12:27:33.077960  7660 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:33.074545  7805 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.016057 12925 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:33.083262  7805 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:33.086802  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:33.088303  8409 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:33.074669 12796 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.083365 13056 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
W20260709 12:27:33.082844 13066 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:27:33.075454 12657 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.079840 12982 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.102090  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:33.086501 12938 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.087180 12925 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:33.115837  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 8.
I20260709 12:27:33.088413  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:33.126411  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2440) took 1903 ms. Trace:
I20260709 12:27:33.127094  7808 rpcz_store.cc:276] 0709 12:27:31.223269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.223416 (+   147us) service_pool.cc:224] Handling call
0709 12:27:33.126100 (+1902684us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54003c50d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:33.126315 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083582848}
I20260709 12:27:33.123736  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.075052 12928 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.140273 12928 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.133867 12814 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.142652 12814 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.098799 12796 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.144987  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2404) took 1705 ms. Trace:
I20260709 12:27:33.145265 12796 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.108263 12657 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.145491  7656 rpcz_store.cc:276] 0709 12:27:31.439765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.101733 (+661968us) service_pool.cc:224] Handling call
0709 12:27:32.102190 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.143888 (+1041698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.143921 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.143962 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:33.143996 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.144715 (+   719us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.144763 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.144892 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395120512}
I20260709 12:27:33.146612  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.147112  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.148036 12657 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.148401  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2680) took 1444 ms. Trace:
I20260709 12:27:33.148494  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1134) took 4505 ms. Trace:
I20260709 12:27:33.083674 12898 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.101544 12992 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:33.083827 12932 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.110886 12822 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:33.134374  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:33.148939  7730 rpcz_store.cc:276] 0709 12:27:31.703679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.107617 (+403938us) service_pool.cc:224] Handling call
0709 12:27:32.127260 (+ 19643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.127504 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:32.323000 (+195496us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.533726 (+210726us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b18002bcf00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:27:32.677821 (+144095us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.730630 (+ 52809us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18002bcf00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:33.147856 (+417226us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b18002bcf00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:33.148027 (+   171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.148082 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:33.148159 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.148227 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.148342 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":448016,"prepare.queue_time_us":235413,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":86,"raft.queue_time_us":483865,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":13954,"spinlock_wait_cycles":126678272,"thread_start_us":275673,"threads_started":2,"wal-append.queue_time_us":395820}
I20260709 12:27:33.134445  7733 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
W20260709 12:27:33.155656  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 267619us
I20260709 12:27:33.175148  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.145179 12814 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:33.133646 12690 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.181023  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.110483 13079 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:33.175187  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44708.7us
I20260709 12:27:33.118846  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:33.148418  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.177546  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.177603  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.149060  8412 rpcz_store.cc:276] 0709 12:27:28.630783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.631016 (+   233us) service_pool.cc:224] Handling call
0709 12:27:28.632451 (+  1435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.443696 (+2811245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.129054 (+1685358us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b54006310d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.135940 (+  6886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6445073280}
I20260709 12:27:33.152439 12611 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.174649  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56027.1us
I20260709 12:27:33.173573  7733 pending_rounds.cc:85] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:27:33.189378  7733 pending_rounds.cc:107] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:27:33.189714  7733 raft_consensus.cc:2889] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:33.190155  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.182828 13051 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:33.183373 12650 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.151983 12761 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.186571  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:33.151453 13067 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 12:27:33.184589  8408 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:33.179656 12993 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.117540 12982 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.192836  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:33.193555  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74808.1us
W20260709 12:27:33.195622  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109523us
I20260709 12:27:33.204052  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.199378  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.174430 13029 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:33.184933 13008 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:33.215379  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.205246 12851 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.217203 13080 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:33.216953  7555 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.200219 12611 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:33.205171 12761 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.180143 12822 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:32.892079  8414 rpcz_store.cc:276] 0709 12:27:31.370405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.371735 (+  1330us) service_pool.cc:224] Handling call
0709 12:27:31.372480 (+   745us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:31.396347 (+ 23867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.396412 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:31.417908 (+ 21496us) log.cc:844] Serialized 52 byte log entry
0709 12:27:31.676329 (+258421us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:31.729861 (+ 53532us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.870895 (+1141034us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18003aaaa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:32.872533 (+  1638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.872636 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.872696 (+    60us) raft_consensus.cc:1685] finished
0709 12:27:32.879818 (+  7122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.880006 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165738,"prepare.queue_time_us":32037,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":8233,"raft.queue_time_us":1157234,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":20863,"spinlock_wait_cycles":28321024,"thread_start_us":176180,"threads_started":1,"wal-append.queue_time_us":454400}
I20260709 12:27:33.205909  8413 raft_consensus.cc:2368] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14.
I20260709 12:27:33.189623 13041 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.204022  8415 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:33.186792 13073 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:33.221763  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2641) took 1999 ms. Trace:
I20260709 12:27:33.229947 12898 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.216977  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:33.236147  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.229457  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1068) took 1587 ms. Trace:
I20260709 12:27:33.234200  7810 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:33.199514 13051 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.239305 13073 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.186376 12690 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.202704 12925 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.177502 12992 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.232298 12822 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:33.271417  7556 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 268) took 36220 ms (client timeout 15000 ms). Trace:
I20260709 12:27:33.208653 12993 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.209580 13049 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:33.284137 13049 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:33.285010 13049 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:33.235153  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.285890 12993 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.287025  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 835) took 1913 ms (client timeout 1691 ms). Trace:
W20260709 12:27:33.287199  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.287602  8405 rpcz_store.cc:269] 0709 12:27:31.373471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.424022 (+ 50551us) service_pool.cc:224] Handling call
0709 12:27:33.286806 (+1862784us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400587854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:33.286947 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086777728}
I20260709 12:27:33.288954 13073 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.287889  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.278530 12935 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.278098 12982 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:33.274391 12938 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.288971  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.293037 12925 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:33.235635  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45564.7us
W20260709 12:27:33.294082  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.335835  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 99808.5us
I20260709 12:27:33.318073  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.357635  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:33.325820  7728 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.358165  7732 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:33.326238  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.327039  7734 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:33.300729  8409 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 12:27:33.313887  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:33.361924  7733 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:33.304098  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.315894  7729 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:33.364289  7804 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
W20260709 12:27:33.365211  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52528.6us
I20260709 12:27:33.325819  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.366490  7736 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:33.327314  7730 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:33.421674  7732 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:33.422746  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.303445  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.305778  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.354584  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.428404  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.304929  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:33.429900  7729 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:33.429981  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46548 (request call id 2681) took 1654 ms. Trace:
I20260709 12:27:33.430781  7728 rpcz_store.cc:276] 0709 12:27:31.704089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.118876 (+414787us) service_pool.cc:224] Handling call
0709 12:27:33.311145 (+1192269us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400348854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f19a3d 00007fc006ee9cd0 00007fc00046e254 00007fc00046e1e2 00007fc00046e111 00007fc00046e08c 00007fc00046d342 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:33.358639 (+ 47494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512800128}
I20260709 12:27:33.307580  7735 rpcz_store.cc:276] 0709 12:27:31.642543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.799628 (+157085us) service_pool.cc:224] Handling call
0709 12:27:31.802153 (+  2525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.406238 (+604085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.229097 (+822859us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.229365 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346018560}
W20260709 12:27:33.390969  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 191173us
I20260709 12:27:33.434028  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:33.435823  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.336299  7737 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:33.298360  7805 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:33.219084  7555 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.460419  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.207306 13029 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.200529 13077 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:33.239319  7658 rpcz_store.cc:276] 0709 12:27:31.222637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.312868 (+ 90231us) service_pool.cc:224] Handling call
0709 12:27:31.331598 (+ 18730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.413816 (+1082218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.221429 (+807613us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.221673 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487092608}
W20260709 12:27:33.278276 12739 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 12:27:33.439756  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:33.476004  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 259081us
I20260709 12:27:33.237571 13041 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:33.461954  7810 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:33.228284 12761 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.384124  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.333796 12990 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:33.520246  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136259us
I20260709 12:27:33.489904 13035 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:33.425480  7806 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:33.491483 13029 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:33.495710  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:33.534628  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:33.528395 12761 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:33.535481  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.500152  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.291157 13051 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.538470  7736 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:33.265491 13008 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.497865 12936 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:27:33.545025  7700 leader_election.cc:400] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14.
I20260709 12:27:33.547111  7700 leader_election.cc:403] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:33.546121  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.549705  7653 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:33.559818  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.302189 13073 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.240952 12680 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:33.311148 12986 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:33.293424 12690 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.484128  7555 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.297906 12992 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.482391 13075 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.225876 12932 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.474356 12657 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.274149 12530 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.548342 13035 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:33.578629 13075 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.575481 12986 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.578907 12657 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.583186 13083 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:33.583727 12998 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.584302 12998 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.584980 12657 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.584878 12986 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.587421 12991 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:33.588173 12991 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.587843 13035 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.588621 12991 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:33.590680 13035 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.591970 12680 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:33.307309 12993 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:33.526705  7733 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:33.490367 12822 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:33.529699  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.599399  7656 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:33.475502  7735 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:33.523756  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:33.507848 12927 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:27:33.531948  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:33.509208 13041 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.293815 12938 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.536134  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.537545  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.538290  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.616874  7659 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:33.220357 12851 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.263914 12967 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.560925  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:33.555961  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:33.560873  7653 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:33.564672  7660 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:33.568414  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.632269  8409 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:33.283741  7556 rpcz_store.cc:269] 0709 12:26:57.051050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.051511 (+   461us) service_pool.cc:224] Handling call
0709 12:27:33.271042 (+36219531us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:18.858400 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:19.068581 (+210181us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:19.068644 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:27:19.068678 (+    34us) tablet.cc:662] Decoding operations
0709 12:27:19.070648 (+  1970us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:19.070768 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:19.070800 (+    32us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:27:19.197443 (+126643us) tablet.cc:701] Row locks acquired
0709 12:27:19.197584 (+   141us) write_op.cc:260] PREPARE: finished
0709 12:27:19.197970 (+   386us) write_op.cc:270] Start()
0709 12:27:19.198133 (+   163us) write_op.cc:276] Timestamp: P: 1783600039197880 usec, L: 0
0709 12:27:19.198173 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:27:19.199050 (+   877us) log.cc:844] Serialized 8955 byte log entry
0709 12:27:21.088185 (+1889135us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b18001f9a20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:21.701580 (+613395us) op_driver.cc:464] REPLICATION: finished
0709 12:27:21.848820 (+147240us) write_op.cc:301] APPLY: starting
0709 12:27:21.848946 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:22.154212 (+305266us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:22.154260 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:23.321082 (+1166822us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b18001f9a20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001c610ec 00007fc001c63af7 00007fc001c64fb0 00007fc0059791bb 00007fc0059c05d4 00007fc00586c8da 00007fc00586f3cb 00007fc00586efd3 00007fc00596de34
0709 12:27:25.299353 (+1978271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:25.645387 (+346034us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:27:25.670798 (+ 25411us) write_op.cc:312] APPLY: finished
0709 12:27:25.877300 (+206502us) log.cc:844] Serialized 204 byte log entry
0709 12:27:26.483564 (+606264us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:26.493779 (+ 10215us) write_op.cc:454] Released schema lock
0709 12:27:26.505267 (+ 11488us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:26.506194 (+   927us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:26.506234 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":129685,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":143182,"apply.run_cpu_time_us":81240,"apply.run_wall_time_us":4657325,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100267,"num_ops":23,"prepare.queue_time_us":449035,"prepare.run_cpu_time_us":18104,"prepare.run_wall_time_us":749934,"raft.queue_time_us":274272,"raft.run_cpu_time_us":36680,"raft.run_wall_time_us":206412,"replication_time_us":2443210,"spinlock_wait_cycles":39608448,"thread_start_us":1204544,"threads_started":5,"wal-append.queue_time_us":1010355,"wal-append.run_cpu_time_us":28269,"wal-append.run_wall_time_us":2507388}]]}
I20260709 12:27:33.569489  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.633970  8415 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:33.582032 13084 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:33.635253  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:33.635710  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.548765 13051 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:33.548977 13077 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:33.493392  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.576345  7555 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.555986 12450 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.586822 12998 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.601338  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:33.608495 12530 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.601670 12739 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:33.597823 13035 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:33.659627 13035 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.660363 13008 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.613242  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89619us
I20260709 12:27:33.665846  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:33.629539  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93529us
I20260709 12:27:33.445076 12960 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.642863  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:27:33.646641  8408 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:33.569407 13073 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.648743  7730 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:33.593235 12680 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:33.657936  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56713.1us
I20260709 12:27:33.679877  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.633450 13082 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:33.689122  7555 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.634837 13083 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.686558  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:33.693917  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149112us
I20260709 12:27:33.698555  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.701090  8415 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:27:33.704142  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 268373us
I20260709 12:27:33.617125 12822 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.695878  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.717406  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.720955  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.649791 12851 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14.
I20260709 12:27:33.723742  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.690366 12530 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:33.697990 12936 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.670372 13035 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.715785 12897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.025s	sys 0.019s
I20260709 12:27:33.759531  7658 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:27:33.700032 13083 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.689684 13082 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:33.758297  7555 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 12, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 12 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.704950 13008 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:33.743011 12450 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.689854 13076 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:27:33.776291  7555 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 5 to 6, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 6 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.690191 12960 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.704293 13034 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:27:33.705504 12998 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:27:33.760866 12936 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.781036  8408 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:33.788074  7555 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.790278  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.791157  8408 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:33.766757 12897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.025s	sys 0.022s
W20260709 12:27:33.781706  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76993.7us
I20260709 12:27:33.782934  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2917) took 2346 ms. Trace:
I20260709 12:27:33.794324  7811 rpcz_store.cc:276] 0709 12:27:31.436830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.437009 (+   179us) service_pool.cc:224] Handling call
0709 12:27:31.475577 (+ 38568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.896442 (+420865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.782653 (+1886211us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54003ea8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.782865 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828027776}
I20260709 12:27:33.771524 13082 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:33.794529  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:33.803239  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79592us
I20260709 12:27:33.784857  8409 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:33.793057  7555 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 9, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 9 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.805181  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.767707 13083 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.774418 12851 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.799858  7655 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:33.809907  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:33.778429 12530 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:33.793212 12936 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.779846 13076 consensus_queue.cc:1243] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:33.787277 12993 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:33.786201 13091 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:33.814373 13086 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 12:27:33.782137 12450 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.793280 12998 consensus_queue.cc:1243] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:33.824175  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.839807  7555 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.823580 13098 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:33.842725 13091 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:33.909662  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.845582 12993 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:33.811908 13092 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:27:33.855928 13073 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.856449 13075 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:33.812433 12851 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:33.861680 13088 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:27:33.861878 12822 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:33.829778 12925 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.001s
I20260709 12:27:33.924290 13099 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:33.925190 13099 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.858562 12991 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:33.875911 12450 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:33.883823 12530 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.866560 12960 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:33.935518  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.936327  8409 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:33.888792  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1152) took 2521 ms. Trace:
I20260709 12:27:33.893124  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:33.897850  7555 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 9, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:33.883011 13049 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.002s
I20260709 12:27:33.930748  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.942694  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.931834  8415 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:27:33.933696  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 139262us
W20260709 12:27:33.930874  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.945681  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:33.946784  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.939355  8405 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:27:33.937674  8411 rpcz_store.cc:276] 0709 12:27:31.367933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.368194 (+   261us) service_pool.cc:224] Handling call
0709 12:27:31.368732 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.411029 (+1042297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.888357 (+1477328us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:33.888691 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2351581568}
I20260709 12:27:33.910823 13098 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.921630 12851 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.910122 12965 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:27:33.937222 12822 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.961318  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:33.968272  7731 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:33.964280  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:33.969871  7555 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 9, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 9 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.954509 12938 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:33.975389  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.961465 13074 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:33.961308 12925 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.955894 13098 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:33.978855  7555 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 11, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 11 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:33.996681  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2928) took 1140 ms. Trace:
I20260709 12:27:33.996814  7807 raft_consensus.cc:1217] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[8.6-12.8]   Dedup: 10.7->[12.8-12.8]
I20260709 12:27:33.960932 13073 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:27:33.998720  7807 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: [8.6-12.8]
I20260709 12:27:33.998365  7803 rpcz_store.cc:276] 0709 12:27:32.856420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.863937 (+  7517us) service_pool.cc:224] Handling call
0709 12:27:32.870167 (+  6230us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:33.206720 (+336553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.206752 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:33.235872 (+ 29120us) log.cc:844] Serialized 78 byte log entry
0709 12:27:33.557154 (+321282us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:33.557194 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.557274 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.996342 (+439068us) raft_consensus.cc:1685] finished
0709 12:27:33.996374 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.996597 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5936,"raft.queue_time_us":49026,"spinlock_wait_cycles":26140416,"thread_start_us":561107,"threads_started":2,"wal-append.queue_time_us":364524}
I20260709 12:27:33.980058 12450 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.980849 12960 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.976954 13101 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.954927 13049 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:33.988420 13074 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.957880 12927 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:27:34.031566 13074 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:33.993183 12925 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.032944 12925 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.979820 12822 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.037384 12822 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:34.049589  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.050227  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.050307  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:34.051427 13077 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.052098 13077 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.052920 12959 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.053313 12694 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:34.053828 12694 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.053573 12959 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.054311 12694 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:34.054653 13077 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.055912  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:34.056649  7661 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:34.058121 13100 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:27:34.058983 13100 consensus_queue.cc:1243] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:27:34.061275  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:34.062647 13048 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.399s	user 0.006s	sys 0.001s
I20260709 12:27:34.062799  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:34.037714 12998 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.063613  7661 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:34.064105 12998 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.064654 12925 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.065793  8409 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:34.065929 12761 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:34.066494 12761 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.066968 12761 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.066763 12936 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.067996  8408 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:34.008013 12960 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.071029 12960 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:34.073060  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:34.077589 13026 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.878s	user 0.012s	sys 0.000s
I20260709 12:27:34.081069  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.081969  7661 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:33.988526 12776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.016s	sys 0.022s
W20260709 12:27:34.017079  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58409.3us
W20260709 12:27:34.083490 12776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.016s	sys 0.022s
I20260709 12:27:34.083844 12776 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:34.085144 13024 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:34.084816  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:34.009318 12450 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.088385 13022 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:34.088996 13022 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:34.090293  7731 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:27:34.090140 13022 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.091877  7735 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:27:34.092880 13029 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.551s	user 0.015s	sys 0.000s
I20260709 12:27:34.093561 13008 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:34.094010 13008 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:34.094046  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.094344 12851 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.096057 13099 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.095883 12694 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.096746  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:34.029018 12991 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:34.006178  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196409us
I20260709 12:27:34.084842 12776 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.085341 12399 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:34.093333 13029 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.098305  7728 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:27:34.017917 13093 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:27:34.082348 12955 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:34.086735 12739 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.016950 13101 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.101357  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2450) took 1041 ms. Trace:
I20260709 12:27:34.116871  7807 rpcz_store.cc:276] 0709 12:27:33.060417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.080600 (+ 20183us) service_pool.cc:224] Handling call
0709 12:27:33.996570 (+915970us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:34.005716 (+  9146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.097496 (+ 91780us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400372dd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:34.101279 (+  3783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072421760}
I20260709 12:27:34.102442  7555 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:27:34.113473  7730 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:34.101964  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:34.119407  7555 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:34.120038  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:34.066807 12998 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.126562  7728 pending_rounds.cc:85] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:34.112672 12450 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.109956 12650 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.109267 12896 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:34.107622 12399 consensus_queue.cc:1243] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:34.131978 12959 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.132354 13029 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.136205  7737 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:34.137326  7731 pending_rounds.cc:85] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 4
I20260709 12:27:34.086790 13082 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.138779 12955 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:27:34.142344  7728 pending_rounds.cc:107] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:27:34.144706  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:34.190618  7728 raft_consensus.cc:2889] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.144546  7555 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:34.145195  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:34.099486 13008 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.154623  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:34.191735  8415 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:34.192049  8407 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:34.159356  7661 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:34.167879  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.169601  7731 pending_rounds.cc:107] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:27:34.193944  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:34.194370  7731 raft_consensus.cc:2889] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.125263 13101 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:34.139663 13024 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:27:34.174695 12450 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.175468 12936 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.177390 13049 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.168918 12857 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:27:34.180791 12925 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.182475 12694 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.183878 12966 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:34.184808  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.140857 13091 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:34.128695 13099 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:34.191121  7728 pending_rounds.cc:107] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:34.215044  7728 raft_consensus.cc:2889] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.148919 12896 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.146311 12851 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:34.196130  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51531.8us
I20260709 12:27:34.204066  7653 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:34.128018 12739 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.191269 12955 consensus_queue.cc:1243] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:27:34.232452  8415 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:34.167057 12650 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.227727  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:34.200675 13102 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:34.234432  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:34.234122  7555 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 8, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 8 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.237037  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:34.238548  8407 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:34.150856 12690 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:27:34.201084 12450 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.250296 12690 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:34.250777 12690 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:34.251387  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:34.237712  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:34.246199  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.252161  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 8.
I20260709 12:27:34.247076  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.253664  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:34.236724 12399 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.254680 12694 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:34.254550 12399 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.205560 12981 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:34.249022 12650 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:34.239578 13023 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:34.210305 12959 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.267124  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:34.214725 13099 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:34.231397 12896 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:34.240789 13041 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.002s
I20260709 12:27:34.274427  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:34.241163 12925 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.192556 13098 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.264635 13091 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:34.240756 13102 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.315166  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:34.255959 13082 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:34.312824  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:34.243530  7555 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index 4 to 8, term changed from 8 to 10, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to <none>, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194), VOTER d72176ccaed049958ac68f706d4f052d (127.6.76.195), VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 10 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:34.316298  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:34.328964  7660 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:34.255213 12694 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.273886 13029 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:34.281389  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:34.295502  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:34.271478 12998 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.316815 12956 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:34.306905 13099 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:34.308739 12851 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:34.301852 12981 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:34.302139 12965 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.342883  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.383791  7662 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:34.344110  7555 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 12, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:27:34.270920 12927 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:34.335151  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:34.251750 12690 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.277820 12959 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.419672  7804 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:34.330833  7653 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:34.273697 13023 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.339871 13077 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:34.332934 12694 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:34.332288 13092 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.317803 13073 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:27:34.301592 12985 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:27:34.332064 13102 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:34.273658 12966 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:27:34.412456  7555 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 12, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 12 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:34.285557  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.278182 12935 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:34.416328  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.417536  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82487.9us
I20260709 12:27:34.343493 12956 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:34.275259 12993 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:27:34.319342 12761 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.315126 13041 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.310889 13103 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 12:27:34.317189 12925 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.352123 12851 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:34.457952  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.332646 13069 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
W20260709 12:27:34.462620  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.499154  7803 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:34.448421 13077 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:34.492120  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.419598 13034 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:34.492950  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.462052 12450 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.496356  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.495889  7555 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:34.471760 13105 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:27:34.492594  7662 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:34.490350 12739 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.343287 12998 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.501597  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.501667  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.502987  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.465998 12530 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.495566  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.469655 12956 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:34.508003  7653 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:27:34.414215 12965 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.472467  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 975) took 1264 ms. Trace:
I20260709 12:27:34.537992  8414 rpcz_store.cc:276] 0709 12:27:33.208265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.233904 (+ 25639us) service_pool.cc:224] Handling call
0709 12:27:34.472245 (+1238341us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54004691d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:34.472392 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491728768}
I20260709 12:27:34.520579  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:34.497946 12851 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:34.527508  7805 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:34.509435 13077 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.530417  7555 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 11, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 11 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.487329 12935 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.470854 13092 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:34.532356 13074 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.000s
I20260709 12:27:34.526363 12993 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:34.521906 12981 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.539033 12960 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.546828  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.566385  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.570732  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.587473  7555 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 11, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 11 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:34.587568  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:34.611971  7812 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:34.596786  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 945) took 1070 ms. Trace:
I20260709 12:27:34.613518  7655 rpcz_store.cc:276] 0709 12:27:33.526436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.527397 (+   961us) service_pool.cc:224] Handling call
0709 12:27:33.527889 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.813434 (+285545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.596424 (+782990us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400435dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:34.596691 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624466304}
I20260709 12:27:34.608752  7809 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:34.536782 12530 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.543766  7733 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:34.619812  7555 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 11, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 11 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:34.624145  8413 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:34.549798 13083 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 12:27:34.632485  7555 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:34.572453 12739 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.650651  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 369338us
I20260709 12:27:34.642724  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.608090 13098 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.653755  7733 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:34.651412  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 317287us
I20260709 12:27:34.581620 13023 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.641423  7555 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 12, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 12 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.674194  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:34.676916  7733 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:27:34.677475  7733 pending_rounds.cc:85] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:27:34.677783  7733 pending_rounds.cc:107] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:27:34.678109  7733 raft_consensus.cc:2889] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.633610 12796 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:34.671046 12952 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.015s	sys 0.026s
I20260709 12:27:34.584431 13074 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:34.448894 13076 consensus_queue.cc:1175] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "992d6df95ef74f7f8b12534534a45501" responder_term: 12 status { last_received { term: 10 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:34.702910  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:34.586196 12993 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.697459  7555 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 8, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:27:34.640316 13029 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:34.676062  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.700418  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.736456  7733 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:34.642921 13083 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.643605 12959 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:34.711874  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:34.737972  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.783684  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.785543  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.740496  8415 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:27:34.742173  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:34.786007  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98956.6us
I20260709 12:27:34.786664  7736 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:34.786938  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.744115 12998 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.675529 12739 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.724076 13041 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.750784  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.751945  7732 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:34.790350  7728 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:34.791132  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.791427  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.791692  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:34.791911  7732 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:34.792924  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.793568  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:34.794610  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.734608  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:34.795912  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61450.9us
I20260709 12:27:34.753803 12851 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:34.756537 12965 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:34.757370 13111 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:34.757930  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.774822  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.775566 12796 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.817597  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.818660  7736 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:34.783413 13122 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.787779  7809 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:34.799948  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.800825  8411 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:34.798835  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:34.830396  7733 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:34.811064  7810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2448) took 1816 ms (client timeout 1741 ms). Trace:
W20260709 12:27:34.831295  7810 rpcz_store.cc:269] 0709 12:27:32.994347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.994527 (+   180us) service_pool.cc:224] Handling call
0709 12:27:34.810586 (+1816059us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:34.810935 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548288256}
I20260709 12:27:34.804116  7732 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:34.812305  7807 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:34.814806  7729 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:34.787524  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:34.756604 13092 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.790503 12761 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:34.828296  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 144052us
W20260709 12:27:34.822158 13108 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 12:27:34.846902  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.838331  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:34.841765  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 184968us
I20260709 12:27:34.844101  7733 pending_rounds.cc:85] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 5
I20260709 12:27:34.849319  7733 pending_rounds.cc:107] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:27:34.849354  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.849828  7733 raft_consensus.cc:2889] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.852093  7736 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:34.845132  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57687.9us
W20260709 12:27:34.834388 12896 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.007s
I20260709 12:27:34.864355  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.865051  7658 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:27:34.873402  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.874679  7809 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:34.828361 12998 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.888909  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.879886  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.867890  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:34.886188  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.825086 13033 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:34.899783  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:34.801910 12959 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:27:34.887702  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.902472  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.893770  7657 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:34.897066  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.905880  8415 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:34.907553  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:34.907981  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:34.908259  7803 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:34.913654  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.844949 13041 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:34.827723 13023 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:27:34.827277 12796 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:34.840865 13122 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:35.010381  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100684us
I20260709 12:27:34.956888  7657 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:34.935496  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.137866  7653 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:34.952280  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:34.954321  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:35.010175  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 118344us
I20260709 12:27:35.149293  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:34.964846  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:34.987200  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 976) took 1662 ms (client timeout 1523 ms). Trace:
I20260709 12:27:34.938912  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:35.174441 13101 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.965s	user 0.014s	sys 0.002s
I20260709 12:27:35.141089 12993 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:35.031454 13033 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:34.877785 12935 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:35.065230 13131 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:27:35.185791  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 250377us
I20260709 12:27:34.906356 13123 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:35.189304  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2934) took 1882 ms. Trace:
W20260709 12:27:35.143128 12694 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.005s
I20260709 12:27:34.905493 12998 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:35.045713 13102 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 12:27:35.191782  7657 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:27:35.141517 13127 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:27:34.917037 13092 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.034560 13041 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:34.914039 13088 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:27:35.064404 12530 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:34.969607 12960 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:34.917002 13128 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:35.170063  8406 rpcz_store.cc:269] 0709 12:27:33.324965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.325922 (+   957us) service_pool.cc:224] Handling call
0709 12:27:34.986904 (+1660982us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:34.987088 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443596160}
I20260709 12:27:34.889588 12896 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.193248 12990 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:35.215812 13101 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:35.081055 13116 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.357s	user 0.000s	sys 0.002s
I20260709 12:27:35.035955 13034 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.178s
I20260709 12:27:35.036195 12987 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.178s
W20260709 12:27:35.216234 13117 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.480s	user 0.000s	sys 0.002s
I20260709 12:27:35.205065 12739 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.239725  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:34.889456 13029 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.243256  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:35.295918  7806 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:35.296185  7653 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:35.296667  7806 raft_consensus.cc:614] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:35.246719  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:35.222647 13083 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.225332 13123 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:35.235558 13119 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.414s	user 0.000s	sys 0.002s
I20260709 12:27:35.261592  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:35.272598 12993 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:35.225418  7804 rpcz_store.cc:276] 0709 12:27:33.306396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.477701 (+171305us) service_pool.cc:224] Handling call
0709 12:27:34.031085 (+553384us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:27:34.031371 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.448006 (+416635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.188705 (+740699us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54003f4654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:35.189204 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894984704}
I20260709 12:27:35.276711  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:35.238997 12694 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.243950 13102 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:35.294152 13149 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:27:35.294559 13145 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:27:35.274816 12990 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:35.290424 13130 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.319s	user 0.002s	sys 0.000s
I20260709 12:27:35.280326 13136 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:35.318372  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.330224  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:35.331312  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 983) took 1259 ms. Trace:
I20260709 12:27:35.375016  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.322219 12993 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.375334  8409 rpcz_store.cc:276] 0709 12:27:34.071670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.098051 (+ 26381us) service_pool.cc:224] Handling call
0709 12:27:34.110631 (+ 12580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.110996 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.111027 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.126090 (+ 15063us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.522951 (+396861us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:34.800535 (+277584us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.827168 (+ 26633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.827322 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.330952 (+503630us) raft_consensus.cc:1685] finished
0709 12:27:35.330977 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.331185 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":381980,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":27977,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":19084,"thread_start_us":628075,"threads_started":3,"wal-append.queue_time_us":490680}
I20260709 12:27:35.348498 13148 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:35.352689  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91182.4us
I20260709 12:27:35.381130  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.354452 12990 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.355132  7653 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:35.310122 13123 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.357313  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2723) took 1223 ms. Trace:
I20260709 12:27:35.395013  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:35.395292  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:35.357880 13156 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:35.395458  7730 rpcz_store.cc:276] 0709 12:27:34.133725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.827806 (+694081us) service_pool.cc:224] Handling call
0709 12:27:34.845588 (+ 17782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.845994 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.846028 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.857322 (+ 11294us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.993101 (+135779us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:34.993166 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.993278 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.357005 (+363727us) raft_consensus.cc:1685] finished
0709 12:27:35.357030 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.357216 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8905,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":16347,"thread_start_us":135333,"threads_started":1,"wal-append.queue_time_us":306382}
I20260709 12:27:35.360608  7654 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:27:35.360915 12998 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.362408  7732 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:35.363649 13102 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.363986  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2721) took 1230 ms. Trace:
I20260709 12:27:35.288364 13137 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:35.368399 13143 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:27:35.318207 12530 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:35.375829  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:35.379361  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:35.383747  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.429076  7810 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:35.429433  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45788.1us
I20260709 12:27:35.325369 12960 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:35.412114 12990 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:35.421754  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:35.403277  7737 rpcz_store.cc:276] 0709 12:27:34.133135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.179894 (+ 46759us) service_pool.cc:224] Handling call
0709 12:27:34.180343 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.180799 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.180832 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.181423 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.530626 (+349203us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:34.530664 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.530730 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.363690 (+832960us) raft_consensus.cc:1685] finished
0709 12:27:35.363717 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.363910 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2651,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":142,"thread_start_us":348711,"threads_started":1,"wal-append.queue_time_us":479334}
I20260709 12:27:35.395812  7806 raft_consensus.cc:606] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:35.429807  7810 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:35.430164  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:35.354542 13139 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:27:35.430588  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.441033  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.442049  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:35.391651 12959 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:35.439409  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2720) took 1306 ms. Trace:
I20260709 12:27:35.444132  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:35.441044  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.381423 13034 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:35.440513 13122 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.288610 13033 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:35.449952  7810 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.383848 13041 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:35.457238  7810 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:35.458185 13083 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:35.413471 12796 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:35.414021 13137 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.458796 13134 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:35.461333  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.464869 13101 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:35.434583 12998 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:35.471958  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2729) took 1146 ms. Trace:
I20260709 12:27:35.452629 12959 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.475785  7662 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:27:35.476445  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.482712  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2730) took 1072 ms. Trace:
W20260709 12:27:35.459791 12927 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.048s	user 0.012s	sys 0.007s
I20260709 12:27:35.434455 12530 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:35.465691 13156 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:35.490051 12998 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.500345 12998 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.496038 13041 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:35.508745  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:35.509418 12993 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:35.510051  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:35.498129 13157 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:35.475255 13137 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:35.488757  7736 rpcz_store.cc:276] 0709 12:27:34.325629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.909077 (+583448us) service_pool.cc:224] Handling call
0709 12:27:34.968773 (+ 59696us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:35.062131 (+ 93358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.062188 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:35.075697 (+ 13509us) log.cc:844] Serialized 52 byte log entry
0709 12:27:35.196597 (+120900us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:35.471083 (+274486us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.471474 (+   391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.471584 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.471647 (+    63us) raft_consensus.cc:1685] finished
0709 12:27:35.471667 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.471866 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277479,"prepare.queue_time_us":268656,"raft.queue_time_us":277175,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":6880,"thread_start_us":53414,"threads_started":1,"wal-append.queue_time_us":119789}
I20260709 12:27:35.381486 13136 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:35.495077  7730 rpcz_store.cc:276] 0709 12:27:34.410728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.456378 (+1045650us) service_pool.cc:224] Handling call
0709 12:27:35.481854 (+ 25476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.482158 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.482199 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:35.482271 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:35.482319 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.482401 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.482608 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.503726  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.506429 12530 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.442787 12960 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.479739 13134 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.507622 12927 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:35.548022  7736 raft_consensus.cc:1217] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-11.6]   Dedup: 7.4->[9.5-11.6]
I20260709 12:27:35.548707  7736 pending_rounds.cc:85] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 4
I20260709 12:27:35.548950  7736 pending_rounds.cc:107] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:27:35.549176  7736 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:35.488945 13083 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:35.480880 12986 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:35.526477  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:35.560117  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.511267 13159 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:35.518162 13122 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:35.539975 12530 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:35.540985  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.552261  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:35.505435 12959 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.554793 12690 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.554922 12991 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:27:35.527846  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.561549  7730 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:35.550522 13134 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:35.466904 13034 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:35.548022 13099 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.158s	user 0.009s	sys 0.008s
I20260709 12:27:35.516803 13157 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:35.563194 13159 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.517834 13156 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:35.781679 13156 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.782238 13156 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:35.564145 13077 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.988s	user 0.010s	sys 0.007s
I20260709 12:27:35.580741  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:35.784080  7730 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:27:35.785627  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.785362 13077 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.786492  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:35.786736  7655 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:35.787346 13159 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:35.787997 13077 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:35.787830 13157 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:35.789018  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 964) took 1147 ms (client timeout 1520 ms). Trace:
I20260709 12:27:35.591202 13136 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:35.595106  7624 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.065s	sys 0.119s
W20260709 12:27:35.790102  7624 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.065s	sys 0.119s
W20260709 12:27:35.790517  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 888519us
I20260709 12:27:35.595518 13160 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:35.791891 13160 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.706434 13164 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:35.708235 13161 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.708451  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.567242 13122 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.812960 13122 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:35.556025 13083 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:35.723649 12694 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:35.557606 12896 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:35.747088  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:35.789592  7655 rpcz_store.cc:269] 0709 12:27:34.642101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.642311 (+   210us) service_pool.cc:224] Handling call
0709 12:27:35.788782 (+1146471us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:35.788925 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603369856}
I20260709 12:27:35.453425  7731 rpcz_store.cc:276] 0709 12:27:34.132744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.136627 (+  3883us) service_pool.cc:224] Handling call
0709 12:27:34.137070 (+   443us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:34.204116 (+ 67046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.204168 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:34.205198 (+  1030us) log.cc:844] Serialized 78 byte log entry
0709 12:27:34.619940 (+414742us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:34.675777 (+ 55837us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.406011 (+730234us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b180028d3e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:27:35.427915 (+ 21904us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.428257 (+   342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.428380 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.438931 (+ 10551us) raft_consensus.cc:1685] finished
0709 12:27:35.438956 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.439227 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":747756,"prepare.queue_time_us":821827,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":62378,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":24233,"spinlock_wait_cycles":18943616,"thread_start_us":345657,"threads_started":1,"wal-append.queue_time_us":473980}
I20260709 12:27:35.565877 13102 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.893080  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.564561 13148 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:35.798144  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:35.950486  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:35.704485 12993 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.758800 13099 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:35.794265 13164 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:35.809235 12897 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:35.885622  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:35.968541  7655 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:35.811618  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:35.865020 13083 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:35.962049  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:35.978022  8407 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:35.985780  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:35.986966  7731 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:35.988691  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:35.992882  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:36.002233  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.011181  7728 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:36.013552  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 227038us
I20260709 12:27:36.014854  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:36.002236  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:36.018723  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.023962  7655 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:36.025811  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:36.026852  7655 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:36.028288  7731 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:27:36.029090  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2826) took 1346 ms. Trace:
I20260709 12:27:36.029417  7731 rpcz_store.cc:276] 0709 12:27:34.682877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.027494 (+1344617us) service_pool.cc:224] Handling call
0709 12:27:36.027996 (+   502us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:36.028765 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.029030 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.029884  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.031471  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:36.032684  7655 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:36.062553  7731 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:35.566583 12986 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:35.958330 13102 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.027403 12450 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:35.845149 13161 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.082538  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:36.082538  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 390041us
I20260709 12:27:36.062427 13024 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:27:36.007946  7810 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.004s
I20260709 12:27:35.965008 12993 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:36.086009 12935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.005s	sys 0.008s
W20260709 12:27:36.066176 12759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.011s	sys 0.032s
I20260709 12:27:36.053552 12694 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.049686 12897 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:36.098534 13041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.008s	sys 0.009s
I20260709 12:27:36.105875  7810 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:36.101802 12960 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:35.967672 13148 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.103933 13024 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:36.102958  7555 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.010s	sys 0.006s
W20260709 12:27:36.139410  7555 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.011s	sys 0.006s
W20260709 12:27:36.122327 12956 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.558s	user 0.029s	sys 0.065s
W20260709 12:27:36.112210 12935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.006s	sys 0.009s
W20260709 12:27:36.120188  7736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.009s
W20260709 12:27:36.152663  7736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.009s
I20260709 12:27:36.096419  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:36.105324 12991 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:36.106383 13029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.003s	sys 0.011s
I20260709 12:27:36.128702  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:36.118667 12759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.012s	sys 0.033s
W20260709 12:27:36.119832  7656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.000s	sys 0.017s
W20260709 12:27:36.123432 13041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.008s	sys 0.009s
W20260709 12:27:36.153581  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.000s	sys 0.017s
I20260709 12:27:36.154729  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2422) took 1289 ms. Trace:
W20260709 12:27:36.219830  7656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.000s	sys 0.020s
W20260709 12:27:36.152107 13091 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.739s	user 0.022s	sys 0.053s
W20260709 12:27:36.223203  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.004s	sys 0.021s
I20260709 12:27:36.154713  7730 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:36.221231  7661 rpcz_store.cc:276] 0709 12:27:34.865336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.865513 (+   177us) service_pool.cc:224] Handling call
0709 12:27:34.915696 (+ 50183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.916015 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:35.144184 (+228169us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.154208 (+1010024us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18001b91c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:27:36.154327 (+   119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.154384 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:36.154435 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.154512 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.154665 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36285,"prepare.queue_time_us":340111,"prepare.run_wall_time_us":68,"raft.queue_time_us":107754,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"spinlock_wait_cycles":13344256,"thread_start_us":298249,"threads_started":3,"wal-append.queue_time_us":139814}
W20260709 12:27:36.156337 13069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.038s	sys 0.089s
I20260709 12:27:36.142174 13034 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:36.204573 13029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.003s	sys 0.012s
W20260709 12:27:36.242748  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138742us
I20260709 12:27:36.168038 12991 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.248215  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:36.240315  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.020s	sys 0.013s
W20260709 12:27:36.244756  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71491.5us
W20260709 12:27:36.227406 13091 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.818s	user 0.023s	sys 0.054s
I20260709 12:27:36.260316  7661 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:36.171135 13160 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:36.235838 13069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.038s	sys 0.101s
W20260709 12:27:36.160804  7611 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.003s	sys 0.011s
I20260709 12:27:36.287617  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:36.258780 12960 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.300537  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1162) took 2519 ms. Trace:
I20260709 12:27:36.265482 12991 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:36.318516 12450 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:36.319278 12450 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.319787 12450 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:36.321125  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 974) took 3238 ms. Trace:
I20260709 12:27:36.321599  8413 rpcz_store.cc:276] 0709 12:27:33.082428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.242942 (+160514us) service_pool.cc:224] Handling call
0709 12:27:33.243305 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.626198 (+1382893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.320831 (+1694633us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:36.321049 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3174912256}
I20260709 12:27:36.323998 13049 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.324749 13049 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.325795 12399 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.236406 13041 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:36.339931 13103 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:27:36.341434  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2726) took 2085 ms. Trace:
I20260709 12:27:36.341981  7732 rpcz_store.cc:276] 0709 12:27:34.256147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.843171 (+587024us) service_pool.cc:224] Handling call
0709 12:27:34.920578 (+ 77407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.402753 (+482175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.341068 (+938315us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b5400144bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:36.341356 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1015315200}
I20260709 12:27:36.156378 12956 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.290162 13099 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:36.264485  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.021s	sys 0.017s
I20260709 12:27:36.305687 12986 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:36.307222 13084 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.357017 13084 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.357909 12925 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:36.294837  7611 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.007s	sys 0.026s
I20260709 12:27:36.264019 13091 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:36.286614 13184 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.294389 13183 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 12:27:36.315454 13137 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.779s	user 0.007s	sys 0.007s
W20260709 12:27:36.265694  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 214094us
W20260709 12:27:36.292114 13092 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.036s	user 0.015s	sys 0.007s
W20260709 12:27:36.314990  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.332608  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.339592 13041 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.299306  7626 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.342690 13161 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.345459  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.301103  8408 rpcz_store.cc:276] 0709 12:27:33.781659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.792412 (+ 10753us) service_pool.cc:224] Handling call
0709 12:27:33.792753 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.068424 (+275671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.300214 (+2231790us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:36.300458 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632551808}
I20260709 12:27:36.386531  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.351162  7811 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:36.353463 13178 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:36.341303 13174 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:36.239174 13024 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.361343 13184 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.401528  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.327189 13049 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.381659 13102 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.382874 13092 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:36.380983  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.4421e+06us
I20260709 12:27:36.388677 13084 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.381214 12966 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.392277 13126 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:36.462110 13183 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:36.385304  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:36.476696  7804 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:36.385699  7732 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:36.402885  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.394884 12990 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.943s	user 0.013s	sys 0.007s
I20260709 12:27:36.343814 12960 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.486217 12690 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.007s
I20260709 12:27:36.486979 12960 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.405787  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:36.487849 13103 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.406715  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 107540us
I20260709 12:27:36.408032 12986 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:27:36.489697  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:36.413281  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9.
I20260709 12:27:36.347733 13164 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.361840 13091 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.466966  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65529.8us
I20260709 12:27:36.464354 12966 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.465065 12450 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.472107 12959 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.000s
I20260709 12:27:36.475422 13069 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:36.512729  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:36.404812 12993 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.480712  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:36.488961 12986 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:36.488488 13103 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.500367  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:36.527709  7661 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:36.523506 13069 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.504272  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.637s (SENT) [suppressed 9 similar messages]
I20260709 12:27:36.506457  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:36.508706  8405 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 12. Ops: []
I20260709 12:27:36.468222 13160 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:36.510084  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.507122  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.466153 13077 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.518169  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.542135  7804 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:27:36.518743 12956 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.519038  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.519798  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.520401 13132 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:36.462035 12965 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.526449 12986 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:36.531291  7806 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:36.532842  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.501719  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:36.534829  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.559190  7732 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:36.560411  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.561290  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:36.562505  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.504930 12960 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:36.552027  7661 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
W20260709 12:27:36.564046 13175 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:27:36.565310  7661 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:27:36.566653  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65004us
I20260709 12:27:36.566123  7732 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:36.567548  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:36.567880  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2441) took 1119 ms. Trace:
I20260709 12:27:36.568405  7661 rpcz_store.cc:276] 0709 12:27:35.448861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.564262 (+1115401us) service_pool.cc:224] Handling call
0709 12:27:36.564910 (+   648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.566895 (+  1985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.567826 (+   931us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:36.569708  7777 connection.cc:256] RPC call timeout handler was delayed by -1.706201133000377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 12:27:36.575567  7777 connection.cc:256] RPC call timeout handler was delayed by -1.6639011330003086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 12:27:36.576666  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.571642 12822 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:36.571731  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.575769  7808 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:27:36.531921 12998 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.031s	user 0.012s	sys 0.012s
I20260709 12:27:36.569730  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:36.577497  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.501266 13024 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:36.381789 13137 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.495579 13159 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.707s	user 0.021s	sys 0.017s
I20260709 12:27:36.593134  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.524295 12959 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.604228 12450 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.604931 12986 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.606695 13074 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:36.606838 13041 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.557078 13132 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:36.575057 12965 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.609493 12530 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.034s	user 0.011s	sys 0.011s
I20260709 12:27:36.610251 12967 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:36.612387 12694 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:36.612396 12868 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:36.585175 12956 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:36.615206 13157 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:36.616135  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.618813  7662 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:36.629117 13008 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:36.629298 12399 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:36.727483  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:36.621622  7732 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:36.729689  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.730626  7732 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:36.731949  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:36.733021  7732 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:36.734161  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.583256 12960 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:36.735893 12960 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:36.736654 12960 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.633194 12822 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:36.557972 13077 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.739953 12822 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.740736 12822 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.742231 13077 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.643985  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:36.743835  7806 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:36.654060  7653 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:36.655639  7736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.017s	sys 0.031s
I20260709 12:27:36.682353 12986 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:36.659209  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:36.684127  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.690616  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.752400  7809 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:36.698267  8413 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:36.672632  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:36.754624  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 82110.2us
I20260709 12:27:36.660255 13132 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:36.668205 12990 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.659894 13136 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.659926  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2424) took 1767 ms. Trace:
I20260709 12:27:36.650144 12690 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:27:36.752310  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.735150  7732 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:36.594043 12998 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.755198  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.758426  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.759428  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:36.770587  8407 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:36.688259 13074 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.765350  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:36.724963 13162 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:27:36.780529 13126 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:36.725487 13069 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.782507  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:36.687664 12923 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:27:36.766855  7659 rpcz_store.cc:276] 0709 12:27:34.892376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.892690 (+   314us) service_pool.cc:224] Handling call
0709 12:27:34.952699 (+ 60009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.952984 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:35.177474 (+224490us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.649952 (+1472478us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.651034 (+  1082us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.659446 (+  8412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.659518 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.659566 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.659643 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.659857 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43861,"prepare.queue_time_us":449455,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":13355,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":17152,"spinlock_wait_cycles":46157056,"thread_start_us":1601182,"threads_started":3,"wal-append.queue_time_us":1198342}
I20260709 12:27:36.711688 12956 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.790104  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:36.798133  7732 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:36.784507  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:36.801240  7809 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:36.640923 13159 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.777364 12650 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:36.771616 12694 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.821270  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:36.800395 13083 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.769s	user 0.014s	sys 0.000s
I20260709 12:27:36.826848  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:36.800607 13156 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.018s	user 0.024s	sys 0.009s
W20260709 12:27:36.814428 13134 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.043s	user 0.012s	sys 0.009s
I20260709 12:27:36.842286  7655 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:36.795971 13071 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:27:36.779148 12990 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.824841 12987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.032s	sys 0.050s
W20260709 12:27:36.858526  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.021s	sys 0.008s
I20260709 12:27:36.778017 13136 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.787139 13074 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:36.844738 13166 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:27:36.864681  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:36.866742  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.021s	sys 0.009s
W20260709 12:27:36.840559 13122 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.027s	user 0.013s	sys 0.011s
I20260709 12:27:36.845798 12650 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:36.874089  7733 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.013s	sys 0.014s
I20260709 12:27:36.832715 12822 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.847138 12960 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:36.871361 12987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.033s	sys 0.050s
I20260709 12:27:36.849730 13134 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.891322 13153 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:36.892225 13182 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:27:36.892118 12965 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.831408 12694 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.892994  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:36.883085  7733 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.013s	sys 0.014s
I20260709 12:27:36.858199 13189 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:36.901274  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:36.841097 13083 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:36.908195  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 215218us
I20260709 12:27:36.920258  7659 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:36.925362  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:36.907773 13123 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.472s	user 0.022s	sys 0.006s
I20260709 12:27:36.905201 13184 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.844564 13156 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.931026  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2725) took 2754 ms. Trace:
I20260709 12:27:36.885280 13122 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.939352  7812 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:36.913518 12959 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.942873 12650 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.950430  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:36.927559 13189 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:36.952044 12924 raft_consensus.cc:993] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:27:36.975838 13137 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:36.976661 13083 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.940078 12923 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:36.947125 13122 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:36.950696 12987 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:27:36.924506 13153 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:36.937294 13123 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:36.903692  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 399418us
I20260709 12:27:36.938324  7733 rpcz_store.cc:276] 0709 12:27:34.177069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.843104 (+666035us) service_pool.cc:224] Handling call
0709 12:27:34.843621 (+   517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:35.988298 (+1144677us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b1800383140. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f1abfb 00007fc001f2564b
0709 12:27:35.988474 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.988500 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:35.989225 (+   725us) log.cc:844] Serialized 52 byte log entry
0709 12:27:36.261996 (+272771us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:36.916687 (+654691us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.916985 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.917081 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.917149 (+    68us) raft_consensus.cc:1685] finished
0709 12:27:36.930792 (+ 13643us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.930964 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":261480,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":19684,"spinlock_wait_cycles":5857664,"thread_start_us":1175928,"threads_started":3,"wal-append.queue_time_us":381091}
W20260709 12:27:37.184836  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 426629us
W20260709 12:27:37.116607  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 166168us
I20260709 12:27:36.951869 12530 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.240s
I20260709 12:27:37.187007  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:37.184051  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.193105  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.192931  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:36.985077 12399 raft_consensus.cc:1081] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:27:37.110038  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2731) took 2698 ms. Trace:
I20260709 12:27:37.192200 12694 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.196339 12956 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:37.200820  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.194967  7736 rpcz_store.cc:276] 0709 12:27:34.411514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.519956 (+1108442us) service_pool.cc:224] Handling call
0709 12:27:35.547771 (+ 27815us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:35.558144 (+ 10373us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:27:35.558559 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.744999 (+1186440us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:36.745728 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:27:36.745969 (+   241us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:37.108418 (+362449us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.109053 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.109255 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.109355 (+   100us) raft_consensus.cc:1685] finished
0709 12:27:37.109370 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.109696 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":207800,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":3292,"raft.queue_time_us":163970,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":8941,"thread_start_us":1547466,"threads_started":3,"wal-append.queue_time_us":665528}
W20260709 12:27:37.204864  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.204749  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:37.187561  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.205891  7736 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:37.112360  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 399996us
I20260709 12:27:37.187542  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.210837  8415 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:37.096632  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.115948  7732 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:37.215067  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.232929  7654 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:36.993822 12998 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.217338  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:37.218114 12993 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:37.219137 12990 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.222043 12923 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:27:37.238391 12993 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.241011 13041 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.242420 12993 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.242134 13041 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.243803 12998 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:37.244858 12998 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.965979 13033 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:37.123200  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:37.212545  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:37.245388  7734 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:27:37.266175  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.096601  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2442) took 1647 ms. Trace:
W20260709 12:27:37.268441 13104 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 2.855s	user 0.007s	sys 0.000s
I20260709 12:27:37.269733  7654 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:27:37.266346  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:36.977057 13125 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:37.267617  7661 rpcz_store.cc:276] 0709 12:27:35.449168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.569352 (+1120184us) service_pool.cc:224] Handling call
0709 12:27:36.572836 (+  3484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.573245 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.573275 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.573924 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.920861 (+346937us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.920933 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.921050 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.095553 (+174503us) raft_consensus.cc:1685] finished
0709 12:27:37.095656 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.096178 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11302,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":31218,"thread_start_us":346453,"threads_started":1,"wal-append.queue_time_us":425567}
I20260709 12:27:37.273985  7733 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:27:37.277436  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.279449  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:37.287357  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.096632  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.280265 13153 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.033392 13189 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.016443 13159 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.247982 12998 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:37.283890  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.274993 13125 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:37.301748 12991 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.986s	user 0.012s	sys 0.016s
I20260709 12:27:37.309949  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.279044  7661 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:37.325768  7733 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:37.243988 13074 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.328599 13125 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:37.324353  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 59923.9us
I20260709 12:27:37.332204  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:37.012272 13134 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.348686  7659 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:37.336445  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.028s	sys 0.057s
I20260709 12:27:37.338501 13123 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.340659  7657 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.340416  7731 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14. Re-sending same reply.
I20260709 12:27:37.350721  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2744) took 1002 ms. Trace:
W20260709 12:27:37.353739  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.029s	sys 0.057s
I20260709 12:27:37.354144  7733 rpcz_store.cc:276] 0709 12:27:36.341370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.265770 (+924400us) service_pool.cc:224] Handling call
0709 12:27:37.344015 (+ 78245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.364214 13156 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.365513  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:37.344317 13041 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.325625 12991 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:37.388751  7736 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:37.390839 13077 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.392052 13077 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:37.394208  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 865) took 2848 ms (client timeout 1565 ms). Trace:
I20260709 12:27:37.397428  7658 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:37.398931 12796 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.399443  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.399652 12796 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.397538 13033 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:37.401628 12986 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.015s
I20260709 12:27:37.401896 13033 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.406656 13033 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:37.420073  8405 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:37.420334  7803 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:27:37.394841  7811 rpcz_store.cc:269] 0709 12:27:34.545834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.546443 (+   609us) service_pool.cc:224] Handling call
0709 12:27:37.393551 (+2847108us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003f3754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.393894 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4004262784}
I20260709 12:27:37.395155 13077 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.433959  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:37.435199  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.436682  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:37.436810  7736 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:37.439903  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2734) took 1110 ms. Trace:
W20260709 12:27:37.446789 12690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.019s	sys 0.054s
I20260709 12:27:37.448282  7736 rpcz_store.cc:276] 0709 12:27:36.329318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.407471 (+1078153us) service_pool.cc:224] Handling call
0709 12:27:37.439477 (+ 32006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.450992  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:37.460115 12690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.020s	sys 0.054s
I20260709 12:27:37.465971  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2445) took 2012 ms. Trace:
I20260709 12:27:37.469419  7661 rpcz_store.cc:276] 0709 12:27:35.450152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.325461 (+1875309us) service_pool.cc:224] Handling call
0709 12:27:37.423360 (+ 97899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.453657 (+ 30297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.453682 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.461355 (+  7673us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:37.461431 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.461672 (+   241us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.462276 (+   604us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.471609  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46884.5us
I20260709 12:27:37.471810  7732 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:37.470736  7806 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:37.477208  8408 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:37.473675  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:37.482043 13024 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.846s	user 0.016s	sys 0.014s
I20260709 12:27:37.485745  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:37.487699  7731 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:27:37.488498  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2844) took 1020 ms. Trace:
I20260709 12:27:37.495782  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.496785  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:37.488930  7731 rpcz_store.cc:276] 0709 12:27:36.467733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.471242 (+1003509us) service_pool.cc:224] Handling call
0709 12:27:37.488373 (+ 17131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.505676 13148 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.507331 12943 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.508692 12943 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.499150  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:37.512877  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2739) took 1173 ms. Trace:
I20260709 12:27:37.513911  7733 rpcz_store.cc:276] 0709 12:27:36.339570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.425704 (+1086134us) service_pool.cc:224] Handling call
0709 12:27:37.512787 (+ 87083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.515344  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2842) took 1048 ms. Trace:
I20260709 12:27:37.521355  7736 rpcz_store.cc:276] 0709 12:27:36.467071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.459767 (+992696us) service_pool.cc:224] Handling call
0709 12:27:37.515007 (+ 55240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.529560  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.530443  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:37.530824  7806 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:37.532457  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.533288 12943 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.541527  7661 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
W20260709 12:27:37.542996 13140 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.428s	user 0.006s	sys 0.000s
I20260709 12:27:37.543525  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.545290  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2716) took 3433 ms. Trace:
I20260709 12:27:37.545653  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.545971  7735 rpcz_store.cc:276] 0709 12:27:34.111585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.111882 (+   297us) service_pool.cc:224] Handling call
0709 12:27:34.113191 (+  1309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.113537 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.113566 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.124028 (+ 10462us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.784833 (+660805us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b18003e53c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:34.830562 (+ 45729us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:34.830609 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.830695 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.521213 (+2690518us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:37.544698 (+ 23485us) raft_consensus.cc:1685] finished
0709 12:27:37.544734 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.545051 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2845,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":42592,"spinlock_wait_cycles":33016576,"thread_start_us":45512,"threads_started":1,"wal-append.queue_time_us":897139}
I20260709 12:27:37.548861 13148 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:37.562608  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 103421us
I20260709 12:27:37.548101 12991 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.566720 13029 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:37.573333  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.525982 13077 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:37.578204 13077 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:37.583987  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.587180  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.587824  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.570889 13029 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:37.588328  7809 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.016s
I20260709 12:27:37.553923 13148 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.599920 13029 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.615954 13022 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.616884 13022 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:37.612654  7809 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:37.623174  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.628335 13103 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.628701  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.629627  7660 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:37.630380  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2957) took 2098 ms. Trace:
I20260709 12:27:37.632143  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2440) took 2183 ms. Trace:
I20260709 12:27:37.633484 12987 raft_consensus.cc:2764] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:27:37.634842  7728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2836) took 2948 ms (client timeout 3075 ms). Trace:
W20260709 12:27:37.635613  7728 rpcz_store.cc:269] 0709 12:27:34.685870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.799631 (+113761us) service_pool.cc:224] Handling call
0709 12:27:37.634333 (+2834702us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400282b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.634609 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2716732800}
I20260709 12:27:37.637120  7731 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:37.639799  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2732) took 1311 ms. Trace:
I20260709 12:27:37.640338  7730 rpcz_store.cc:276] 0709 12:27:36.328302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.332299 (+  3997us) service_pool.cc:224] Handling call
0709 12:27:37.639336 (+1307037us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b540037e954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.639581 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230106496}
I20260709 12:27:37.631037  7810 rpcz_store.cc:276] 0709 12:27:35.531340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.124940 (+593600us) service_pool.cc:224] Handling call
0709 12:27:36.125461 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.125821 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.125865 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.382951 (+1257086us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:37.416364 (+ 33413us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.448474 (+ 32110us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:37.448531 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.453359 (+  4828us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.629661 (+176302us) raft_consensus.cc:1685] finished
0709 12:27:37.629691 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.630082 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12040448,"thread_start_us":30941,"threads_started":1,"wal-append.queue_time_us":95271}
I20260709 12:27:37.642549  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.643067  7729 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:37.643546 13077 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.644407  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.645180  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2727) took 3388 ms. Trace:
I20260709 12:27:37.630823  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.650847  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.651044  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2735) took 1309 ms. Trace:
I20260709 12:27:37.645831  7729 rpcz_store.cc:276] 0709 12:27:34.256509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.843078 (+586569us) service_pool.cc:224] Handling call
0709 12:27:37.642605 (+2799527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.643599 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.644677 (+  1078us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b5400282b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.645057 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6335115520}
I20260709 12:27:37.652809  7731 rpcz_store.cc:276] 0709 12:27:36.338279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.524461 (+1186182us) service_pool.cc:224] Handling call
0709 12:27:37.596534 (+ 72073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.637709 (+ 41175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.647117 (+  9408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.648541  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.632628  7655 rpcz_store.cc:276] 0709 12:27:35.448536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.035260 (+586724us) service_pool.cc:224] Handling call
0709 12:27:36.035690 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.842913 (+807223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.631480 (+788567us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540033e5d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:37.631849 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854468992}
I20260709 12:27:37.645970  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:37.650964  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2962) took 1010 ms. Trace:
I20260709 12:27:37.666376  7809 rpcz_store.cc:276] 0709 12:27:36.637823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.753679 (+115856us) service_pool.cc:224] Handling call
0709 12:27:37.624961 (+871282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.668862  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.669368  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.666278  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2740) took 1326 ms. Trace:
I20260709 12:27:37.671444  7735 rpcz_store.cc:276] 0709 12:27:36.339860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.548555 (+1208695us) service_pool.cc:224] Handling call
0709 12:27:37.553242 (+  4687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.560563 (+  7321us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:37.563965 (+  3402us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.648951 (+ 84986us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.664584 (+ 15633us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.665688 (+  1104us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.665774 (+    86us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:37.665852 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.665966 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.666181 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":705,"prepare.queue_time_us":3224,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":6609,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":3141,"thread_start_us":39714,"threads_started":1,"wal-append.queue_time_us":46354}
I20260709 12:27:37.673028  7660 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.674492  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.675778  7660 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.671052  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.677407  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.675729  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:37.679740  7661 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.680863  7660 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:37.681420  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.683980  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.685133  7660 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:37.686688  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.688119  7660 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.684149  7661 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:37.657639  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:37.690099  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.692512  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.693866  7660 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:37.695067  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.696301  7660 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:37.690227  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.698220  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.697842  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:37.703042  7661 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:37.709452 13141 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:27:37.712939  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.718338 13189 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:37.719513 13022 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.720659 13189 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:37.697688 13132 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.912s	user 0.023s	sys 0.003s
I20260709 12:27:37.726540  7655 raft_consensus.cc:2376] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10. Re-sending same reply.
W20260709 12:27:37.729120  7701 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 2.221s	user 0.010s	sys 0.013s
I20260709 12:27:37.729939  7655 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:37.729808  7701 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 2.222s	user 0.010s	sys 0.013s
W20260709 12:27:37.729691  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2433) took 2521 ms (client timeout 2477 ms). Trace:
I20260709 12:27:37.733754 13148 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:37.731911  7657 rpcz_store.cc:269] 0709 12:27:35.208323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.246338 (+ 38015us) service_pool.cc:224] Handling call
0709 12:27:37.729290 (+2482952us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54003994d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.729577 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4694996352}
I20260709 12:27:37.736774  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2745) took 1395 ms. Trace:
I20260709 12:27:37.737407  7731 rpcz_store.cc:276] 0709 12:27:36.341755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.666921 (+1325166us) service_pool.cc:224] Handling call
0709 12:27:37.731753 (+ 64832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.736282 (+  4529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.736312 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.736390 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:37.736421 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.736498 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.736681 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.738539  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2423) took 2862 ms (client timeout 1632 ms). Trace:
W20260709 12:27:37.738996  7658 rpcz_store.cc:269] 0709 12:27:34.876589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.879510 (+  2921us) service_pool.cc:224] Handling call
0709 12:27:37.738262 (+2858752us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b5400219dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.738465 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5363238656}
I20260709 12:27:37.740696  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2446) took 2280 ms. Trace:
I20260709 12:27:37.741158  7657 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:37.741151  7655 rpcz_store.cc:276] 0709 12:27:35.450456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.729243 (+2278787us) service_pool.cc:224] Handling call
0709 12:27:37.729730 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.730395 (+   665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.730609 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.744241  7658 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:37.745150  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2450) took 2226 ms. Trace:
I20260709 12:27:37.745569  7658 rpcz_store.cc:276] 0709 12:27:35.518680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.740043 (+2221363us) service_pool.cc:224] Handling call
0709 12:27:37.740551 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.744805 (+  4254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.745056 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.746577  7729 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:37.747725  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2743) took 1407 ms. Trace:
I20260709 12:27:37.748111  7729 rpcz_store.cc:276] 0709 12:27:36.340758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.690532 (+1349774us) service_pool.cc:224] Handling call
0709 12:27:37.745988 (+ 55456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.747388 (+  1400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.747661 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.666036 13033 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.750615  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2447) took 2292 ms. Trace:
I20260709 12:27:37.746683 12851 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.757025  7657 rpcz_store.cc:276] 0709 12:27:35.450724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.736853 (+2286129us) service_pool.cc:224] Handling call
0709 12:27:37.737437 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.741848 (+  4411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.742720 (+   872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.757419  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:37.760677  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 967) took 1739 ms. Trace:
I20260709 12:27:37.761417  7658 rpcz_store.cc:276] 0709 12:27:36.017901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.746497 (+1728596us) service_pool.cc:224] Handling call
0709 12:27:37.746987 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.756348 (+  9361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.756397 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.756475 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:37.756528 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.756649 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.756872 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.740865 13135 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.497s	user 0.026s	sys 0.025s
I20260709 12:27:37.755405  7660 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:37.764632  7661 raft_consensus.cc:2376] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 9. Re-sending same reply.
W20260709 12:27:37.784272  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138496us
W20260709 12:27:37.772186 13135 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.657s	user 0.026s	sys 0.025s
I20260709 12:27:37.782768 13041 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.742751 13132 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.788508  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2741) took 1445 ms. Trace:
W20260709 12:27:37.791359  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.792650  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65834us
I20260709 12:27:37.790282  7728 rpcz_store.cc:276] 0709 12:27:36.340139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.636614 (+1296475us) service_pool.cc:224] Handling call
0709 12:27:37.637116 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.637572 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:37.638038 (+   466us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.763730 (+125692us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.784823 (+ 21093us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.785072 (+   249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.785139 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:37.785207 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.785296 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.785458 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24243,"prepare.queue_time_us":29403,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":94,"raft.queue_time_us":4682,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":9934,"thread_start_us":120746,"threads_started":2,"wal-append.queue_time_us":104398}
I20260709 12:27:37.796597  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1176) took 1043 ms. Trace:
I20260709 12:27:37.797148  8405 rpcz_store.cc:276] 0709 12:27:36.753508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.763839 (+ 10331us) service_pool.cc:224] Handling call
0709 12:27:37.419434 (+655595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.430582 (+ 11148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.796247 (+365665us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400183b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.796483 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505447168}
I20260709 12:27:37.803926  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 970) took 1735 ms. Trace:
I20260709 12:27:37.804563  7657 rpcz_store.cc:276] 0709 12:27:36.068862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.768334 (+1699472us) service_pool.cc:224] Handling call
0709 12:27:37.786278 (+ 17944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.803345 (+ 17067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.803403 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.803503 (+   100us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:37.803548 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.803630 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.803820 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.781083 13189 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:37.784621 12923 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:37.814248  7805 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:27:37.817819  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1097) took 1029 ms. Trace:
I20260709 12:27:37.813555 13041 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.818529  7732 rpcz_store.cc:276] 0709 12:27:36.782511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.799103 (+ 16592us) service_pool.cc:224] Handling call
0709 12:27:37.806960 (+1007857us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540037e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:37.812202 (+  5242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661094144}
I20260709 12:27:37.822237  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2746) took 1480 ms. Trace:
I20260709 12:27:37.822713  7735 rpcz_store.cc:276] 0709 12:27:36.342031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.673842 (+1331811us) service_pool.cc:224] Handling call
0709 12:27:37.674965 (+  1123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.675708 (+   743us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:37.705579 (+ 29871us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.821921 (+116342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.821984 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:37.822019 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.822068 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.822181 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2769,"prepare.queue_time_us":21523,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"thread_start_us":85500,"threads_started":2}
W20260709 12:27:37.824234  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.826277  7811 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:37.827781  7661 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:37.839952 13189 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:37.812331 12923 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:37.832360  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2451) took 1181 ms. Trace:
I20260709 12:27:37.843969  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 971) took 1249 ms. Trace:
I20260709 12:27:37.843420  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 966) took 1968 ms. Trace:
I20260709 12:27:37.847558  7660 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:37.845139  7658 rpcz_store.cc:276] 0709 12:27:36.593029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.773459 (+1180430us) service_pool.cc:224] Handling call
0709 12:27:37.808783 (+ 35324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.841821 (+ 33038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.841862 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.841965 (+   103us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:37.842018 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.842117 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.842286 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.846448  7655 rpcz_store.cc:276] 0709 12:27:36.003977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.759293 (+1755316us) service_pool.cc:224] Handling call
0709 12:27:37.789248 (+ 29955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.834009 (+ 44761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.834058 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.834118 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:37.834155 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.834236 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.834450 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.845494  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:37.817282 13148 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:37.859751  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 972) took 1217 ms. Trace:
I20260709 12:27:37.844044  7661 rpcz_store.cc:276] 0709 12:27:36.651368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.782403 (+1131035us) service_pool.cc:224] Handling call
0709 12:27:37.815723 (+ 33320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.828638 (+ 12915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.832261 (+  3623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.856981  7657 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:27:37.857649 13185 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:37.860514  7660 rpcz_store.cc:276] 0709 12:27:36.642267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.775613 (+1133346us) service_pool.cc:224] Handling call
0709 12:27:37.820552 (+ 44939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.859281 (+ 38729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.859661 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.861347  7657 pending_rounds.cc:85] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 7
W20260709 12:27:37.851593 12998 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.013s
I20260709 12:27:37.846725 12923 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.859846  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2748) took 1517 ms. Trace:
W20260709 12:27:37.869212  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58251.7us
W20260709 12:27:37.870618 13215 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:27:37.871475 12927 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:37.872609 13041 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.871470 12943 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:27:37.882265  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:37.883920 13103 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.873960  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:37.885802 13219 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:27:37.887485 13185 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.841041 13189 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.889288 13041 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:37.890671 13185 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.891578  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:37.892510 12943 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:37.893193 12943 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:37.894376 13148 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.894559 12943 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:37.896795 13033 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:37.897123 12943 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:37.898018 13148 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.898856 13148 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:37.863138  7657 pending_rounds.cc:107] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:27:37.888685 13132 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:37.881809 12927 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:37.841068 13212 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:27:37.900612  7661 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:27:37.877961  7729 rpcz_store.cc:276] 0709 12:27:36.342572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.749471 (+1406899us) service_pool.cc:224] Handling call
0709 12:27:37.749939 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.759597 (+  9658us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:37.784595 (+ 24998us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.859403 (+ 74808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.859467 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:37.859526 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.859622 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.859777 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32377,"prepare.queue_time_us":26531,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":8096,"raft.queue_time_us":136,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":5087,"thread_start_us":29543,"threads_started":1}
I20260709 12:27:37.883836  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:37.894869  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2456) took 1222 ms. Trace:
I20260709 12:27:37.911782  7655 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:27:37.912138  7658 rpcz_store.cc:276] 0709 12:27:36.672062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.864784 (+1192722us) service_pool.cc:224] Handling call
0709 12:27:37.893633 (+ 28849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.894349 (+   716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.894444 (+    95us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.894498 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:37.894537 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.894626 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.894789 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":900}
I20260709 12:27:37.904654  7657 raft_consensus.cc:2889] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:37.908934  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2742) took 1563 ms. Trace:
I20260709 12:27:37.890972 13189 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:37.911298 12927 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.920586  7658 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:27:37.921399  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:37.879105 12998 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:37.928624  7661 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 12. Ops: [10.11-10.11]
I20260709 12:27:37.929538  7729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:37.914270  7730 rpcz_store.cc:276] 0709 12:27:36.340422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.641433 (+1301011us) service_pool.cc:224] Handling call
0709 12:27:37.642041 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.652459 (+ 10418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.652497 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.726775 (+ 74278us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.774758 (+ 47983us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:37.774800 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.774876 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.903325 (+128449us) raft_consensus.cc:1685] finished
0709 12:27:37.903352 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.903599 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1130,"raft.queue_time_us":5913,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":200,"thread_start_us":29158,"threads_started":1,"wal-append.queue_time_us":71207}
I20260709 12:27:37.928668  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2454) took 1267 ms. Trace:
I20260709 12:27:37.937192  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:37.940330  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:37.917968 13123 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:37.947360  7655 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:37.947440  7657 rpcz_store.cc:276] 0709 12:27:36.661802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.805772 (+1143970us) service_pool.cc:224] Handling call
0709 12:27:37.807437 (+  1665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.928305 (+120868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.928592 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.951568  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2738) took 1607 ms. Trace:
I20260709 12:27:37.952575  7736 rpcz_store.cc:276] 0709 12:27:36.339281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.933553 (+1594272us) service_pool.cc:224] Handling call
0709 12:27:37.934098 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.934690 (+   592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.934745 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.934853 (+   108us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:37.934895 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.934992 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.946803 (+ 11811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.955968  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2754) took 1556 ms. Trace:
I20260709 12:27:37.956709  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2752) took 1592 ms. Trace:
I20260709 12:27:37.956629  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2426) took 3062 ms. Trace:
I20260709 12:27:37.957324  7735 rpcz_store.cc:276] 0709 12:27:36.364683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.834886 (+1470203us) service_pool.cc:224] Handling call
0709 12:27:37.902346 (+ 67460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.956227 (+ 53881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.956263 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.956334 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:37.956396 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.956472 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.956638 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1437,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156}
I20260709 12:27:37.950081 13123 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:37.958170  7656 rpcz_store.cc:276] 0709 12:27:34.893458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.893782 (+   324us) service_pool.cc:224] Handling call
0709 12:27:34.894462 (+   680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.257080 (+362618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.257131 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.257805 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.220959 (+963154us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:36.221010 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.955827 (+1734817us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540002ec54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:37.955920 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.955999 (+    79us) raft_consensus.cc:1685] finished
0709 12:27:37.956013 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.956310 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3210,"raft.queue_time_us":99789,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8744,"spinlock_wait_cycles":825129728,"thread_start_us":944303,"threads_started":1,"wal-append.queue_time_us":986730,"wal-append.run_cpu_time_us":2226,"wal-append.run_wall_time_us":1028736}
I20260709 12:27:37.960361 13123 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:37.961728  7737 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:37.956691  7730 rpcz_store.cc:276] 0709 12:27:36.399248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.954395 (+1555147us) service_pool.cc:224] Handling call
0709 12:27:37.955005 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.955445 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.955476 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.955572 (+    96us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:37.955618 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.955687 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.955866 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.963696  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.962641  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1092) took 1195 ms. Trace:
I20260709 12:27:37.965031  7737 rpcz_store.cc:276] 0709 12:27:36.767645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.960754 (+1193109us) service_pool.cc:224] Handling call
0709 12:27:37.961262 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.962308 (+  1046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.962572 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:37.975118  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103870us
I20260709 12:27:37.978435  7654 raft_consensus.cc:2368] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12.
I20260709 12:27:37.942924 13029 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:37.984840  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2751) took 1621 ms. Trace:
I20260709 12:27:37.987432  7737 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:37.987780  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1096) took 1211 ms. Trace:
I20260709 12:27:37.989774  7736 rpcz_store.cc:276] 0709 12:27:36.776172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.972246 (+1196074us) service_pool.cc:224] Handling call
0709 12:27:37.986959 (+ 14713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.987343 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.987369 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.987440 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:37.987472 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.987541 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.987713 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.007617  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1094) took 1228 ms. Trace:
W20260709 12:27:38.006974  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.008251  7737 rpcz_store.cc:276] 0709 12:27:36.768373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.966466 (+1198093us) service_pool.cc:224] Handling call
0709 12:27:37.982242 (+ 15776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.996151 (+ 13909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.997007 (+   856us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:38.010979  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70870.2us
W20260709 12:27:38.011155  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 169773us
I20260709 12:27:38.021222  7729 raft_consensus.cc:606] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:38.020690  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2747) took 1678 ms. Trace:
I20260709 12:27:38.007575 13185 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:38.021840  7729 raft_consensus.cc:614] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:38.032995  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:38.008152 12987 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:38.038527  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1093) took 1260 ms. Trace:
W20260709 12:27:38.039330  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102207us
I20260709 12:27:38.043612  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.042804  7730 rpcz_store.cc:276] 0709 12:27:36.768045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.963485 (+1195440us) service_pool.cc:224] Handling call
0709 12:27:37.974354 (+ 10869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.008372 (+ 34018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.008422 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.019405 (+ 10983us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:38.026965 (+  7560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.027084 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.027918 (+   834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.019313 13148 raft_consensus.cc:2764] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:27:38.024658  7731 rpcz_store.cc:276] 0709 12:27:36.342293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.738570 (+1396277us) service_pool.cc:224] Handling call
0709 12:27:37.739064 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.775716 (+ 36652us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:37.833876 (+ 58160us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.020013 (+186137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.020090 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:38.020143 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.020420 (+   277us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.020597 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22394,"prepare.queue_time_us":53547,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":98,"raft.queue_time_us":8121,"thread_start_us":102534,"threads_started":1}
I20260709 12:27:38.038173 13185 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:38.068220 13221 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.069305 13221 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.071419  7736 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.073015  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1098) took 1285 ms. Trace:
I20260709 12:27:38.074806  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:38.073676  7737 rpcz_store.cc:276] 0709 12:27:36.787627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.009427 (+1221800us) service_pool.cc:224] Handling call
0709 12:27:38.010251 (+   824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.064998 (+ 54747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.065060 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.072587 (+  7527us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:38.072658 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.072758 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.072926 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.062868 13185 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:38.078878  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:38.060175 12987 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.080147  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:38.081315 13177 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:38.088549  8413 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:27:38.090775 13103 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:38.093452 12897 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:38.095316  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2750) took 1752 ms. Trace:
I20260709 12:27:38.095391  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:38.103319 13189 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:38.103739 12987 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.106444  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2438) took 2708 ms. Trace:
W20260709 12:27:38.109706  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2449) took 2622 ms (client timeout 2041 ms). Trace:
W20260709 12:27:38.110507  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 862) took 3609 ms (client timeout 1579 ms). Trace:
I20260709 12:27:38.111334 13029 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.113291  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2960) took 1478 ms. Trace:
I20260709 12:27:38.081358 13135 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:38.111393 13228 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:38.086603 13177 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.124544  7804 rpcz_store.cc:276] 0709 12:27:36.634898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.637993 (+  3095us) service_pool.cc:224] Handling call
0709 12:27:36.638361 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.828359 (+189998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.828412 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.828481 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:36.828577 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.112810 (+1284233us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:38.112875 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.113108 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436215168}
I20260709 12:27:38.118398  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2939) took 3267 ms. Trace:
W20260709 12:27:38.120925  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53690.1us
I20260709 12:27:38.116770  7728 rpcz_store.cc:276] 0709 12:27:36.343159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.819492 (+1476333us) service_pool.cc:224] Handling call
0709 12:27:37.859151 (+ 39659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.891627 (+ 32476us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:37.892103 (+   476us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.911389 (+ 19286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.911458 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:37.915538 (+  4080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.094890 (+179352us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b54002477d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:38.094963 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.095203 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1282,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":3471,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3480,"spinlock_wait_cycles":73568000,"thread_start_us":18287,"threads_started":1,"wal-append.queue_time_us":169759}
W20260709 12:27:38.122759  7808 rpcz_store.cc:269] 0709 12:27:34.501032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.501220 (+   188us) service_pool.cc:224] Handling call
0709 12:27:38.109968 (+3608748us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54003f3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:38.110300 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4689292928}
W20260709 12:27:38.125420  7662 rpcz_store.cc:269] 0709 12:27:35.487402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.568293 (+ 80891us) service_pool.cc:224] Handling call
0709 12:27:38.109208 (+2540915us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54000f4e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:38.109483 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710879104}
I20260709 12:27:38.131688 12998 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.132498 12759 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.132938 12759 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.134169  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:38.134080 12759 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.134889  8409 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.130056  7812 rpcz_store.cc:276] 0709 12:27:34.848418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.848538 (+   120us) service_pool.cc:224] Handling call
0709 12:27:34.852559 (+  4021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.956638 (+2104079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.115396 (+1158758us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:38.115721 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4798626048}
I20260709 12:27:38.123790 13041 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:27:38.141029 13135 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.122483  7653 rpcz_store.cc:276] 0709 12:27:35.398239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.400235 (+  1996us) service_pool.cc:224] Handling call
0709 12:27:35.446718 (+ 46483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.744436 (+1297718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.105696 (+1361260us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540021bbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:38.106182 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2775988608}
I20260709 12:27:38.146236  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:38.146966  8405 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.119112 13189 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:38.151265  7736 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:38.153359  7737 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:27:38.155160  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:38.159479 12897 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.160131 13221 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.162165 12897 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.153496  7731 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:38.162859  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2843) took 1695 ms. Trace:
I20260709 12:27:38.163424  7735 rpcz_store.cc:276] 0709 12:27:36.467419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.958973 (+1491554us) service_pool.cc:224] Handling call
0709 12:27:37.968285 (+  9312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.995843 (+ 27558us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:27:37.995878 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.998468 (+  2590us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.058525 (+ 60057us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:27:38.058583 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.058667 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.162601 (+103934us) raft_consensus.cc:1685] finished
0709 12:27:38.162626 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.162778 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4804,"raft.queue_time_us":12382,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":22401,"thread_start_us":59504,"threads_started":1,"wal-append.queue_time_us":123756}
I20260709 12:27:38.163924  8409 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:38.167656  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:38.168778 12998 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:38.169386 12998 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:38.169895 12927 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.171378 13033 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:38.171365  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:38.170496 12998 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.173978 13231 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:38.174783 13231 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.175704  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:38.175995 13231 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.176987 13234 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:38.143396 13177 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.156051  7662 raft_consensus.cc:2376] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12. Re-sending same reply.
I20260709 12:27:38.179709 13234 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.183430 12966 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:38.183806  8405 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:38.183980 13234 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.184697 12925 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.178066 13189 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:38.185293 12925 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.186558 12925 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.187949 12925 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:38.189710  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 138070us
I20260709 12:27:38.190680  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:38.183455 13227 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:27:38.191819 12811 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:38.192297 12811 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.148046 13228 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.192719 12811 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:38.199648 13033 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:38.202742 12966 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:27:38.203614  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51747.9us
I20260709 12:27:38.204723 12966 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.205271 12966 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.217159  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:38.225641  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:38.206527 12966 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:38.229218  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.247347  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.247865  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.248265  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.249666  7653 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:38.252907 13071 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:37.989169  7732 rpcz_store.cc:276] 0709 12:27:36.363028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.821661 (+1458633us) service_pool.cc:224] Handling call
0709 12:27:37.895384 (+ 73723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.895960 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:37.899970 (+  4010us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.984288 (+ 84318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.984367 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:37.984436 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.984562 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.984756 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7906,"prepare.queue_time_us":3439,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":16016,"thread_start_us":72052,"threads_started":1}
I20260709 12:27:38.255000 12779 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:38.255638 12779 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.257064 12779 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.257720  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.258443  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:38.258662  7655 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:38.259815  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.261533  7653 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:38.273666  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.233309 12759 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.277019 13228 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.277061 13033 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.277441  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:38.258845 12779 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:38.285802  8405 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:38.285923  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:38.290496  7662 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:38.286564  7654 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:38.219249 13123 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.306278  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.307375  7803 raft_consensus.cc:2376] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15. Re-sending same reply.
I20260709 12:27:38.308425 12927 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.302804  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.314261 13123 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.319622  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.323355  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:38.325311  7662 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:38.327769 13033 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.273559 13135 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.341199  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:38.341954  7656 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 9.
I20260709 12:27:38.347940  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.352645  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:38.353982  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:38.362236  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.368631  7803 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:27:38.369406  7803 pending_rounds.cc:85] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 8
I20260709 12:27:38.369690  7803 pending_rounds.cc:107] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:27:38.369935  7803 raft_consensus.cc:2889] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.371407 13136 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.371764  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.372499  7807 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:38.375336 13229 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.153s	user 0.001s	sys 0.001s
I20260709 12:27:38.375108 13103 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:38.381491 13177 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:38.387311  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 43782.1us
W20260709 12:27:38.387727  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 193681us
I20260709 12:27:38.388947  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:38.389168 13177 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.389509  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:38.390985  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.404147  7807 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:38.309677 12811 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.410538 13103 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:38.461548 13103 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:38.461982 13103 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.462596 12811 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:38.411510 13037 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:38.412257 12927 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:38.463043 13103 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.466825 13103 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:38.414358  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:38.414489 12952 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:38.415477  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.416664  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.299750 13228 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:27:38.417563 13068 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:38.418558 12749 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:38.420008 12868 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:38.421522 13071 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:38.427510 12898 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:38.431561 13204 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:38.432092  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:38.495148 12811 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:38.495747  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:38.496356 12749 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:38.496855 12749 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.497473 12896 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:38.497655  7737 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:38.498027 12749 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.349555 12796 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.433472 13106 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:38.499573 12814 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.500025 12814 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.501510 12814 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:38.434904  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.506166 12927 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.506856 12927 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:38.508286  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.511420  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:38.435532 12923 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.013s
I20260709 12:27:38.513170 12923 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.515698  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.517068  7807 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:38.518205  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.519408  7803 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:38.520710  7808 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:38.293340 13185 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:38.436650  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47267.4us
I20260709 12:27:38.436767 13075 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:38.410382 13126 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:38.459893 12966 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.446473  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:38.475250  7653 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:38.476307  7731 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:38.393312 13135 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:38.489845  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.482021 12868 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.481496 13037 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.432766 13033 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.544620  7662 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:38.563829 13126 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.566756 13136 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:38.567385 12868 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.567698 13136 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.567888 13126 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.569236 12868 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.570349 13136 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.557650 13037 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.571784 12749 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.572402 13037 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.574050 13037 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.503121 12814 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.578398 12923 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:38.556843 13106 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.580606 13106 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.530646 12927 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.585316 13024 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:38.586562 13024 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:38.586946 13024 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.586989 12897 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:27:38.497926 12896 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:38.495776 12811 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:38.558012 13135 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:38.597205 13135 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.597106 13228 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:38.592351 13185 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:38.590762 12694 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:38.588183 13024 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.632144 13136 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.632683 13024 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:38.639829  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.627732 12811 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:38.641863  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 300289us
I20260709 12:27:38.643680  7810 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:27:38.621114 12694 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:38.658004  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.658612  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.661324  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.608436 13189 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:38.665207  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 276379us
I20260709 12:27:38.671464  7810 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.676220 13185 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.676901 13185 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.679328  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:38.680702  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84625.1us
I20260709 12:27:38.682230 12998 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:38.688218  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:38.691534  8379 leader_election.cc:400] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12.
I20260709 12:27:38.692106  8379 leader_election.cc:403] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:38.694916 13124 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.695699 13124 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:38.698263 13124 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.699126 13075 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12.
I20260709 12:27:38.701181  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:38.705214  7804 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:38.705370  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.708213 12759 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:38.711760 12896 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.713544 12896 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:38.719751 13124 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:38.721238  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.722474  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.726724  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.726859  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.632952 13136 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.682396  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:38.730548  7804 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:38.733826  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.740052  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.728525 12811 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:38.751830  7810 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:38.753178  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:38.754313  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.755446  7810 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:27:38.732602 13124 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:38.742858 12796 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:38.736251 13185 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:38.752211 12694 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.807803  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.809129  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:38.809916  7810 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:38.825774  7806 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:38.833480  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2941) took 3984 ms (client timeout 1758 ms). Trace:
I20260709 12:27:38.839836  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:38.840240  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.836824  7805 rpcz_store.cc:269] 0709 12:27:34.863460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.863678 (+   218us) service_pool.cc:224] Handling call
0709 12:27:38.812467 (+3948789us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b5400132ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:38.833362 (+ 20895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6765237632}
I20260709 12:27:38.825973 12796 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:38.859833  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.565s (SENT)
I20260709 12:27:38.875093  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:38.875598  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:38.859249  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:38.878638  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 196449us
I20260709 12:27:38.883750  7730 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:38.886447  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:38.898229  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3061883320001471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 12:27:38.898849  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2656883320000816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 12:27:38.899976  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:38.906558 13185 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:38.908953  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:38.912243  7733 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:38.920089  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.926512  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2737) took 2577 ms. Trace:
I20260709 12:27:38.927214  7733 rpcz_store.cc:276] 0709 12:27:36.338979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.910950 (+2571971us) service_pool.cc:224] Handling call
0709 12:27:38.911684 (+   734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.915725 (+  4041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.916039 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.890762 13204 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:38.932807  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.890967 12694 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:38.930619  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.928102  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.937899  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:38.938887 12796 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.929541 13124 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:38.943416  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48303.8us
W20260709 12:27:38.944393 13205 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.492s	user 0.008s	sys 0.000s
I20260709 12:27:38.948298  7810 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:38.950042  7736 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:38.936333  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1177) took 2176 ms (client timeout 2041 ms). Trace:
W20260709 12:27:38.953498  8408 rpcz_store.cc:269] 0709 12:27:36.753881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.754768 (+   887us) service_pool.cc:224] Handling call
0709 12:27:38.915426 (+2160658us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:38.930206 (+ 14780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506459392}
I20260709 12:27:38.955654 12868 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.956367 12868 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:38.958207  7730 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:38.959267  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:38.959992  7728 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:38.960136  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:38.960969  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52183.8us
I20260709 12:27:38.967661  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:38.976579  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53573.7us
I20260709 12:27:38.952178 13189 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:38.991032  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139608us
I20260709 12:27:38.993889  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:38.994645  7661 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:38.994827  7658 raft_consensus.cc:2376] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14. Re-sending same reply.
I20260709 12:27:38.998535  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.999626  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:38.967278 13259 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:39.002866  7806 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:27:39.004106  8379 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.579s (SENT)
I20260709 12:27:38.978276 13221 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:38.978530  7659 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:27:39.014681  7659 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:39.015461  7659 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:39.036180  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.042776  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:39.048128  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80559.3us
I20260709 12:27:38.994906 13136 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:39.056767  7807 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:27:39.056996 12868 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:39.057294  7807 pending_rounds.cc:85] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 9
I20260709 12:27:39.057598  7807 pending_rounds.cc:107] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:27:39.057838  7807 raft_consensus.cc:2889] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:39.058137  7807 pending_rounds.cc:107] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:27:39.058374  7807 raft_consensus.cc:2889] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:39.059855 13256 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:27:39.059579 13135 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:39.061268 13250 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:27:39.052928  7658 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:39.064065 12796 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.066664 12796 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.067358  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.068316  7803 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:39.069499  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.070255 13126 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:39.072270 13126 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:39.072744 13126 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:39.077262  7654 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:39.078408  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:39.079078  7661 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.080677 12796 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:39.082314 12680 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.083039 12680 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.083917  7658 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:39.085638 12680 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:39.070277  7803 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:39.093664  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.073967 13126 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.117630 13126 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:39.118126  7661 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:27:39.100771  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:39.120582 13189 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.121143  7662 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:39.121356 13189 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.018443 13148 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:27:39.123323 13254 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:39.124338 13189 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:39.120545  7661 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:39.126118  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.127467 13148 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:39.128158 13148 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:39.130796  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 132565us
I20260709 12:27:39.132540  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.011262 13221 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.007750 13259 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:39.089236  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:39.112820 13258 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:27:39.113719  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:39.116312 13135 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:39.142349  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.145114  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:39.190325  7806 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.147178  7657 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:39.158628  7730 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:39.191504  7730 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:39.161325  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79849a645ae64c3787630b03e7da6307"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:39.192476  7807 raft_consensus.cc:606] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:39.193022  7807 raft_consensus.cc:614] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:39.192924  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60528.7us
I20260709 12:27:39.173272 13125 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:39.176941  7736 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:27:39.177276  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.184526 12868 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.204741  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2466) took 1614 ms (client timeout 2063 ms). Trace:
I20260709 12:27:39.143896  7659 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:39.190996  7803 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:39.204720  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:39.203617  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2712) took 5102 ms. Trace:
W20260709 12:27:39.222998  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.205379  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.204721  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:39.231243  7810 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:39.231551  7807 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.205617  7808 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:27:39.231312  7811 rpcz_store.cc:269] 0709 12:27:37.581722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.587408 (+  5686us) service_pool.cc:224] Handling call
0709 12:27:39.190609 (+1603201us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:39.190787 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519692032}
I20260709 12:27:39.185480  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.235651  7812 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:39.251868  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:39.252765  7736 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:39.230361  7734 rpcz_store.cc:276] 0709 12:27:34.089452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.098584 (+  9132us) service_pool.cc:224] Handling call
0709 12:27:34.729434 (+630850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.247494 (+2518060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.528917 (+1281423us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b5400248bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:39.191915 (+662998us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400248bd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:39.192091 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7208819328}
I20260709 12:27:39.145835 13221 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:39.267050  7777 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.632s (SENT)
I20260709 12:27:39.270613  7659 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:39.274282  7734 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:39.144515 13259 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:39.206406 13228 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:39.279357  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.271296  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 76808us
I20260709 12:27:39.239476 13185 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:39.286367  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.239596 12998 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:39.290038  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:39.289714  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.293766  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:39.285305  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 235854us
I20260709 12:27:39.289201  7730 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.298872  7812 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:39.306334  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:39.271164 13141 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.309947  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.311034  7737 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:39.311157  7730 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:39.299448  7657 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.316290  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:39.317631  7662 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:39.296320  7808 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:39.251935 12868 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:39.286104 13135 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:39.324252  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:39.328857  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:39.334354  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 129767us
I20260709 12:27:39.334756  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:39.342682 13252 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:27:39.343484 13189 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:39.344635  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77641.2us
W20260709 12:27:39.350574 13024 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.002s
I20260709 12:27:39.352495  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.338217  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.284734 13181 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:27:39.339133 13185 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:39.361953  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.365417  7662 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:39.367009  7653 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:39.368880 12898 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 12:27:39.373636 12966 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.816s	user 0.008s	sys 0.006s
I20260709 12:27:39.368554  7809 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:39.334048 13106 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:39.350626 12896 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.007s
I20260709 12:27:39.384272  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.362107 13124 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:39.387054  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:39.388019  8405 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.396382  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:39.358830 13103 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.891s	user 0.016s	sys 0.000s
I20260709 12:27:39.376431 13185 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:39.401750  7807 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:27:39.410096 13245 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:27:39.411566  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.327541 13141 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.314890 12998 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:39.369822 12990 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:39.423563  8405 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:39.425989  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:39.432638  7728 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:39.428534  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:39.435978  7662 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:39.440243  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.442966  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.414319 13185 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.448669  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2762) took 1044 ms. Trace:
I20260709 12:27:39.413932 12796 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:39.449980  8408 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:39.453279  7735 rpcz_store.cc:276] 0709 12:27:38.404299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.469109 (+ 64810us) service_pool.cc:224] Handling call
0709 12:27:38.484944 (+ 15835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.564270 (+ 79326us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:38.832443 (+268173us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.448116 (+615673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:39.448177 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:39.448348 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.448429 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.448585 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54318,"prepare.queue_time_us":349237,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4883,"raft.queue_time_us":84081,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":796095,"threads_started":3,"wal-append.queue_time_us":230140}
I20260709 12:27:39.463147  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.468436  8405 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:39.480940  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:39.481978  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:39.444901 13077 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:39.447003 12990 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:39.504565  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 127453us
W20260709 12:27:39.498587 12694 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.001s
I20260709 12:27:39.545042  8405 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:39.543258  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.611510  7658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.004s	sys 0.008s
W20260709 12:27:39.612154  7658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.004s	sys 0.008s
I20260709 12:27:39.599357  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:39.615885 12990 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.506476 13268 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:39.646199  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.615971  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:39.676055 13272 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:27:39.679387 12868 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:39.680685 13189 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:39.681527 13141 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:39.683676  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 442415us
W20260709 12:27:39.549151 12811 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.004s
I20260709 12:27:39.521994 13077 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:39.707696  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.711935  7626 connection.cc:256] RPC call timeout handler was delayed by -1.2044412930004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 12:27:39.720147  7626 connection.cc:256] RPC call timeout handler was delayed by -1.1353085000000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 12:27:39.720579  7626 connection.cc:256] RPC call timeout handler was delayed by -1.0894084999999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
I20260709 12:27:39.718423  7730 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:27:39.721158  7730 pending_rounds.cc:85] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:27:39.721594  7730 pending_rounds.cc:107] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:27:39.721989  7730 raft_consensus.cc:2889] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:39.725715  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.726234 13077 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.726879  8411 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:39.728070  7777 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.041s (SENT)
W20260709 12:27:39.728684  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.730669  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:39.744516  7805 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:27:39.745167  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.730743 13125 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 12:27:39.761793  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.798986  8409 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:27:39.780401  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:39.814851  7735 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:39.770675 13189 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:39.771360  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.744071 12868 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.838333  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:39.839448  8405 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:39.790478  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2968) took 1064 ms. Trace:
W20260709 12:27:39.840073  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 109496us
I20260709 12:27:39.840205  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:39.821924  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.822549  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:39.826629 13136 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.827468 13264 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:39.831622 12898 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:39.836320  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.839027  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:39.818581  7804 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:39.813632  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:39.840113  7809 rpcz_store.cc:276] 0709 12:27:38.726360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.726546 (+   186us) service_pool.cc:224] Handling call
0709 12:27:39.790143 (+1063597us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400134cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:39.790379 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472906624}
W20260709 12:27:39.850668  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85788us
W20260709 12:27:39.859267  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 447747us
I20260709 12:27:39.862732  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:39.854554  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:39.759212 12811 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:39.857803  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 313103us
I20260709 12:27:39.770614 13271 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:39.887609  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:39.891206  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77674.7us
W20260709 12:27:39.891448  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52546.5us
I20260709 12:27:39.842535 12897 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.888099  8411 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:39.897253  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:39.900663  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:39.902900  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:39.842298 13231 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.909973  7806 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.001s
W20260709 12:27:39.910605  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.837908 13189 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:39.914914  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55256us
I20260709 12:27:39.783614 13267 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 12:27:39.926851  7660 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:39.926760  7731 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:39.928468  7806 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:39.857461 12898 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:39.933837  7735 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:39.796746 13024 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:27:39.924576 12897 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:39.943159 13277 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:27:39.943869  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48235.7us
W20260709 12:27:39.944540  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48755us
I20260709 12:27:39.894783 13271 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:39.890208 12998 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:39.933534 12811 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:39.929901 13270 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:39.947118  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49972.7us
I20260709 12:27:39.914179 13228 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:39.925129 13106 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:39.948251 13269 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:27:39.951596  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:39.961858  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2970) took 1230 ms (client timeout 1563 ms). Trace:
W20260709 12:27:40.014102  7804 rpcz_store.cc:269] 0709 12:27:38.731104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.731868 (+   764us) service_pool.cc:224] Handling call
0709 12:27:39.961529 (+1229661us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54003c55d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:39.961738 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566515584}
I20260709 12:27:39.962946  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2975) took 1084 ms. Trace:
I20260709 12:27:40.015758  7805 rpcz_store.cc:276] 0709 12:27:38.878266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.878477 (+   211us) service_pool.cc:224] Handling call
0709 12:27:39.962594 (+1084117us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400133dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:39.962845 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938474368}
W20260709 12:27:39.972326  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.886158 13135 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:39.894313 13041 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.944425 12898 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.988588  7660 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:27:39.852823 12868 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:39.926043 13189 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:39.983477 13126 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.865s	user 0.015s	sys 0.001s
I20260709 12:27:39.992082 13204 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:40.005688 12998 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:40.006577  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:39.874146 13264 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.007493 13106 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.010437 13228 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:40.010497  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:40.005746 13075 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:27:40.011714 13270 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:39.999698 12811 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.977540 13103 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:40.022241  7657 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 12:27:40.004145 12966 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:27:39.993772 13271 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:40.028611  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 2979) took 1081 ms. Trace:
I20260709 12:27:39.874042 13136 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:40.031420  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79942.6us
I20260709 12:27:39.985178 13231 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.036724  7657 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
W20260709 12:27:40.029879 13274 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 12:27:40.046826  7660 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:40.109462  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:40.057147  7803 rpcz_store.cc:276] 0709 12:27:38.943754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.088791 (+145037us) service_pool.cc:224] Handling call
0709 12:27:39.190764 (+101973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.230211 (+ 39447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.025093 (+794882us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400372dd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:40.025366 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233531648}
I20260709 12:27:40.059760  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:40.035285 12898 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:40.059093 13282 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.112288  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52628.5us
I20260709 12:27:40.112744 13282 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.033463 12998 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.035463 13106 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:40.048128 12811 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.050519 13103 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.027684 13041 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.047881 13270 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:40.130728  8409 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
W20260709 12:27:40.136152  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129694us
I20260709 12:27:40.118795  7657 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:27:40.156500  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:40.115571 12998 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.165287  7660 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:40.165179  8409 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:40.153894 13270 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:40.168581  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:40.170405 13279 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:27:40.173584 13123 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:40.174808 13123 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.178020 13123 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.178530 13106 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.177490 13284 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:27:40.181470 12897 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.185740 13231 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:40.191512  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1110) took 1035 ms. Trace:
W20260709 12:27:40.159112 13235 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.266s	user 0.000s	sys 0.002s
I20260709 12:27:40.218993  7729 rpcz_store.cc:276] 0709 12:27:39.156292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.183443 (+ 27151us) service_pool.cc:224] Handling call
0709 12:27:39.183934 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.205450 (+ 21516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.205482 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.206308 (+   826us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.747878 (+541570us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:39.758170 (+ 10292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.758268 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.191146 (+432878us) raft_consensus.cc:1685] finished
0709 12:27:40.191177 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.191428 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14443,"raft.queue_time_us":22372,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":32989,"thread_start_us":460565,"threads_started":1,"wal-append.queue_time_us":585005}
I20260709 12:27:40.176921 13041 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.150213 13103 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:40.146507 13282 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:40.167866 13277 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:27:40.224558  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:40.247958  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:40.229642  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 300826us
I20260709 12:27:40.248968  7660 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
W20260709 12:27:40.234783  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 328716us
W20260709 12:27:40.237694  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 474313us
I20260709 12:27:40.250737  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:40.211014 13123 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.253518  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:40.251963  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 364449us
I20260709 12:27:40.264361  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 989) took 1198 ms. Trace:
W20260709 12:27:40.254026 13148 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.125s	user 0.016s	sys 0.003s
I20260709 12:27:40.255347 13282 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.264817  7654 rpcz_store.cc:276] 0709 12:27:39.065448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.079013 (+ 13565us) service_pool.cc:224] Handling call
0709 12:27:39.079439 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.079890 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.079936 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.080554 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.400017 (+319463us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:39.400077 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.400160 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.264099 (+863939us) raft_consensus.cc:1685] finished
0709 12:27:40.264120 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.264290 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":159,"thread_start_us":318965,"threads_started":1,"wal-append.queue_time_us":499347}
I20260709 12:27:40.267182  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:40.254864 13273 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.289889  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.637s (SENT)
I20260709 12:27:40.292824  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:40.233655 13231 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:27:40.275619  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 375846us
I20260709 12:27:40.244226 12897 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 12:27:40.314000  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.295861 13185 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.005s
I20260709 12:27:40.291090 13282 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:40.310918 13294 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.260353 12680 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:27:40.326282  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.303874 13204 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:40.256600 13228 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.288383 13148 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.316237 13041 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:40.360658  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:40.361585  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:40.362833  7731 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
W20260709 12:27:40.367369  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93713.3us
I20260709 12:27:40.334756 13282 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.367769 13106 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.369577 13295 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:40.370160 13295 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:40.352550 13294 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.371371 13295 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.372736 13295 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:40.374846 13264 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:40.335372 13185 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.376551  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:40.377652 13185 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.378428 13185 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:40.381556 13148 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:40.383456  7733 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:40.383661 12966 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:40.384339 12966 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.385119 12966 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:40.389055 13221 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.083s	user 0.014s	sys 0.000s
I20260709 12:27:40.389595 13221 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.355275 12868 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:40.298887 13273 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.383496  7735 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:27:40.397944  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 144614us
I20260709 12:27:40.405390  8412 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:40.374336 13294 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:40.405668  7737 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:27:40.407593  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 156944us
W20260709 12:27:40.445425  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126219us
I20260709 12:27:40.355873 13204 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.444916  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:40.458411 13283 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:27:40.460425  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2482) took 1052 ms. Trace:
I20260709 12:27:40.450438 13204 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:40.460871  7659 rpcz_store.cc:276] 0709 12:27:39.407835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.408170 (+   335us) service_pool.cc:224] Handling call
0709 12:27:39.440312 (+ 32142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.440652 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.440694 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.450695 (+ 10001us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.891146 (+440451us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:39.891197 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.891318 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.460151 (+568833us) raft_consensus.cc:1685] finished
0709 12:27:40.460183 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.460352 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":381363,"threads_started":1,"wal-append.queue_time_us":673479}
I20260709 12:27:40.447052 13291 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:40.464174 13291 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:40.465345 13291 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.466856 13135 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:40.447484 13290 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.447115 13285 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:27:40.467403 13301 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:40.376582 13024 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.490864  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.404729 13267 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.456364  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:40.406229 12680 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.498343  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:40.498818  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:40.446888 13286 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:27:40.504870  7804 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:40.449682 13268 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.702s	user 0.014s	sys 0.001s
W20260709 12:27:40.447202 13287 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 12:27:40.416400 13264 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:40.360291 12694 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 12:27:40.494931  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.498255  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:40.500392  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 237187us
W20260709 12:27:40.546630  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 394640us
W20260709 12:27:40.547531  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 140118us
W20260709 12:27:40.503597  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.507905  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:40.503841  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.647s (SENT)
I20260709 12:27:40.549243  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:40.504968  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:40.467412 13135 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:40.541574  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:40.458436 13124 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.068s	user 0.013s	sys 0.002s
I20260709 12:27:40.541994  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.542725  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.548969  7737 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.548848  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:40.494441 13298 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.550287  7660 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:40.563930  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.555287  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:40.565085  7808 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:40.555856  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:40.551090  7775 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:40.565493  7812 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:40.499565  8413 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:40.547910  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:40.493026 12868 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:40.492797 13136 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:40.562271  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:40.469026 12897 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:27:40.572561  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2468) took 1626 ms. Trace:
I20260709 12:27:40.565514  7808 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:40.452270 13294 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:27:40.579113  7735 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:40.563030  7809 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:27:40.578926  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 243924us
I20260709 12:27:40.549134 13268 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:40.604606  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.605631  7804 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:27:40.607369  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58893.2us
I20260709 12:27:40.542309 12680 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:40.588361 13280 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.514s	user 0.000s	sys 0.002s
W20260709 12:27:40.602881  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55027us
I20260709 12:27:40.617686  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:40.583370 13277 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:40.561460 13124 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:40.573271 12868 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.584923  7655 rpcz_store.cc:276] 0709 12:27:38.946421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.946570 (+   149us) service_pool.cc:224] Handling call
0709 12:27:38.947032 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.954775 (+  7743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.954828 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.990583 (+ 35755us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.443864 (+453281us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:39.701376 (+257512us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.701638 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.701721 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.392984 (+691263us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b18004a6140. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:40.543005 (+150021us) raft_consensus.cc:1685] finished
0709 12:27:40.572264 (+ 29259us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.572464 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1370,"prepare.queue_time_us":245498,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":4804,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":38401,"spinlock_wait_cycles":5308160,"thread_start_us":391063,"threads_started":1,"wal-append.queue_time_us":746988}
I20260709 12:27:40.681492  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:40.616284  7808 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.678997  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.674535  7660 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:40.589304 13185 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:40.701050  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:40.489018 13290 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.704291  7804 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:40.505342 13267 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.564265 13024 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.620843 13264 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:40.763626  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82259.2us
I20260709 12:27:40.637903 13221 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.715842 13126 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:27:40.742233  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 193429us
I20260709 12:27:40.740708  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1114) took 1583 ms. Trace:
I20260709 12:27:40.778718  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:40.781185  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2485) took 1333 ms. Trace:
I20260709 12:27:40.772360  7733 rpcz_store.cc:276] 0709 12:27:39.157544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.158429 (+   885us) service_pool.cc:224] Handling call
0709 12:27:39.158706 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.383965 (+1225259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.740279 (+356314us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002d7b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:40.740597 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816240384}
W20260709 12:27:40.704231 12811 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.000s
I20260709 12:27:40.808861  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:40.699900 12868 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.787618 13303 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 4, Time since last communication: 0.205s
W20260709 12:27:40.787992 13306 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:27:40.788461 13313 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.826068 13313 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.826560 13313 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.790146 13300 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.831584 13300 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.834028 13300 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:40.791872  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 990) took 1726 ms. Trace:
I20260709 12:27:40.834582  7658 rpcz_store.cc:276] 0709 12:27:39.065725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.085075 (+ 19350us) service_pool.cc:224] Handling call
0709 12:27:39.085436 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.085906 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:39.085939 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.086466 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.619992 (+533526us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:39.620045 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.620136 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.458514 (+838378us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:40.791458 (+332944us) raft_consensus.cc:1685] finished
0709 12:27:40.791492 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.791795 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6284,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":10506,"spinlock_wait_cycles":69051264,"thread_start_us":525734,"threads_started":1,"wal-append.queue_time_us":771649}
I20260709 12:27:40.792667 12680 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.795827 13309 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.837020 13309 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.771513 13126 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:40.702824 13185 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:40.718216 13268 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:40.840261 13268 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:40.840808 13268 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:40.840312 13126 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.842167 13126 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:40.781741  7662 rpcz_store.cc:276] 0709 12:27:39.451625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.451808 (+   183us) service_pool.cc:224] Handling call
0709 12:27:39.452284 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.510057 (+ 57773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.510119 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.527423 (+ 17304us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.055211 (+527788us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:40.055287 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.055366 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.780900 (+725534us) raft_consensus.cc:1685] finished
0709 12:27:40.780935 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.781112 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42942,"raft.queue_time_us":57397,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":72483,"thread_start_us":438702,"threads_started":1,"wal-append.queue_time_us":656379}
W20260709 12:27:40.803366  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 252351us
I20260709 12:27:40.578233 13298 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:40.765331 12998 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.004s
W20260709 12:27:40.802623 13270 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.005s
I20260709 12:27:40.786823 13293 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:40.820680  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:40.853303  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2487) took 1366 ms. Trace:
W20260709 12:27:40.845575  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.853797  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 173470us
I20260709 12:27:40.839689 13185 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:40.774391 12694 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:40.872262  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:40.873672  8405 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
W20260709 12:27:40.874388  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53874.2us
I20260709 12:27:40.883260  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:40.871037  7661 rpcz_store.cc:276] 0709 12:27:39.487354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.497378 (+ 10024us) service_pool.cc:224] Handling call
0709 12:27:39.547525 (+ 50147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.548574 (+  1049us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:40.062526 (+513952us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.852873 (+790347us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:40.852958 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:40.853010 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.853082 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.853239 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":280736,"prepare.queue_time_us":219768,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":113746,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":42114,"thread_start_us":959838,"threads_started":3,"wal-append.queue_time_us":666473}
I20260709 12:27:40.884327  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:40.843222 13124 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.817396 13305 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:40.885831 13277 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:27:40.892989 13123 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260709 12:27:40.883726 13148 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:27:40.912660  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 110445us
I20260709 12:27:40.845527 12811 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:40.899542 13103 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.003s
I20260709 12:27:40.915512 13273 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:40.893880 13124 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:40.847064 13264 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.906199 13126 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:40.873607 12998 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.894539 13300 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:27:40.945585  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2466) took 2000 ms. Trace:
W20260709 12:27:40.939172  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56038us
I20260709 12:27:40.887799 12694 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.941494  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:40.844309 13135 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.946178  7656 rpcz_store.cc:276] 0709 12:27:38.945636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.947585 (+  1949us) service_pool.cc:224] Handling call
0709 12:27:38.968042 (+ 20457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.984952 (+ 16910us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:39.054286 (+ 69334us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.055616 (+1001330us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.417700 (+362084us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.457325 (+ 39625us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.457502 (+   177us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:40.457801 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.457979 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.458083 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.945220 (+487137us) raft_consensus.cc:1685] finished
0709 12:27:40.945256 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.945483 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203251,"prepare.queue_time_us":346362,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":162,"raft.queue_time_us":183072,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":85192,"spinlock_wait_cycles":86141824,"thread_start_us":1002794,"threads_started":3,"wal-append.queue_time_us":646355}
I20260709 12:27:40.912937 13148 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:40.881273 13270 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:40.954576  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:40.933209 13259 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.614s	user 0.009s	sys 0.009s
I20260709 12:27:40.934676 13124 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.967442  7733 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:40.967558  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:40.973556  7728 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:40.920809 13228 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:27:40.974416  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:40.968081  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50367.5us
I20260709 12:27:40.943819 13305 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:40.941298 13273 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:40.918428 13123 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:40.978368  7731 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:40.964232 13148 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:40.931835 13103 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:41.023845  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43709.6us
I20260709 12:27:41.009413  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:40.998646  7729 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:41.010176  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.036043  7733 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:40.931365 13328 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:27:41.057519  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:40.975039 13294 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:41.082077  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 211097us
W20260709 12:27:41.100175  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 191298us
I20260709 12:27:41.114358  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.114820  7731 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:27:41.100594 13204 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.008s
I20260709 12:27:41.123586  7812 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.008867 13307 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.124954  7732 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.030164 13309 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.127467  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.137300  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.137863  7733 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:41.138513  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.124747  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:41.139786  7731 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:41.131978  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.009577 13310 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:41.010584 13319 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:41.009692 13314 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:41.143445  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.006s
W20260709 12:27:41.144956  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.007s	sys 0.006s
I20260709 12:27:40.976282 13259 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:41.145709  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 226179us
W20260709 12:27:41.141368  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.059607  7737 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.002s
I20260709 12:27:41.150408  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:41.120015 13024 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.014220 12868 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.163754  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.165135  7732 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:41.008690 13318 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.222s
I20260709 12:27:41.008209 13135 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:41.130024 13290 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.171828  7734 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:41.009799 13296 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:41.168645  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.183634  7737 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:41.144012 13328 consensus_queue.cc:1243] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:27:41.181770  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 98573.7us
I20260709 12:27:40.993767 13273 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.128458 13294 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.187880  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:41.189222  7806 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:41.179011  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 675149us
I20260709 12:27:41.157698 13310 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:41.148138 13267 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:41.194738  7627 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.701s (SENT)
I20260709 12:27:41.200378  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.196229  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.201417  7737 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:41.198976  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85754.2us
I20260709 12:27:41.134408 13204 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:41.240506  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45803.3us
W20260709 12:27:41.242127  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.834s (SENT)
I20260709 12:27:41.242729  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.191645 13290 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.245939  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.247140  7737 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:41.156881 13307 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:41.253537 13330 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:41.254755 13307 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:41.255978 13327 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:27:41.257858  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1184) took 1812 ms (client timeout 1928 ms). Trace:
I20260709 12:27:41.258455 13293 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:41.258347  8412 rpcz_store.cc:269] 0709 12:27:39.445560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.455708 (+ 10148us) service_pool.cc:224] Handling call
0709 12:27:41.257591 (+1801883us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:41.257758 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603161472}
I20260709 12:27:41.192855 13332 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
W20260709 12:27:41.263821 13316 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
W20260709 12:27:41.266976  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 79179.5us
W20260709 12:27:41.267694  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.269332  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:41.269704 13294 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.203985 13296 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.270296 13294 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:41.272061 13296 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.202109 13310 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:41.274344 13024 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:41.274679 13270 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.274971 13024 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:41.275488 13264 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.275959 13024 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.278496 13273 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.276439 12694 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.257210  7559 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:41.281121  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:41.294405  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.328570 12966 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.896s	user 0.009s	sys 0.005s
I20260709 12:27:41.333576 12966 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.336649 12966 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.220868 12811 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.338506  8412 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:41.336560 13268 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.340337  7809 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.193614 13298 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.342104  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.343645  7737 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.344786 13259 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:41.345106  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73856us
I20260709 12:27:41.278820 13313 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.346391 12868 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.347074 12868 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:41.347532 12868 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.347936 12868 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:41.348052 13325 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:41.348671  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:41.350167 12898 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.213s	user 0.014s	sys 0.000s
W20260709 12:27:41.350267  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 108187us
I20260709 12:27:41.350919 13313 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:41.351498  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.989s (SENT)
I20260709 12:27:41.246366 12998 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.352720 13338 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:41.353719  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.358006  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:41.377329  7735 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.004s
I20260709 12:27:41.377573  7737 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:41.377733  7735 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:41.363173  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.367393  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.370433  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.371024  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2484) took 1929 ms. Trace:
W20260709 12:27:41.342530  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217854us
I20260709 12:27:41.271159 13294 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:41.355943  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:41.386763  7662 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:41.379503  7653 rpcz_store.cc:276] 0709 12:27:39.441485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.441684 (+   199us) service_pool.cc:224] Handling call
0709 12:27:39.462477 (+ 20793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.495318 (+ 32841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.495371 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.508742 (+ 13371us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.853268 (+344526us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:39.853326 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.853435 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.350786 (+1497351us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b2400055928. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee73fb0 00007fc001aea488 00007fc001aeabe8 00007fc001ac2398 00007fc001abd76a 00007fc001abcbb6
0709 12:27:41.361418 (+ 10632us) raft_consensus.cc:1685] finished
0709 12:27:41.361442 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.370954 (+  9512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23432,"raft.queue_time_us":47112,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":31605,"spinlock_wait_cycles":9375104,"thread_start_us":233499,"threads_started":1,"wal-append.queue_time_us":591034}
I20260709 12:27:41.380741  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.388942  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.389698  8407 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.381433  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.389950  7737 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.391173  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.341662 13332 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.385097  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.385399  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.381559  7732 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:41.397912  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:41.397251  8414 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:41.403728 13294 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.409096 12966 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:41.409969 12966 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.411867  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:41.412050  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60607.2us
I20260709 12:27:41.419978  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.420723  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:27:41.423609  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67754.4us
I20260709 12:27:41.427752  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2782) took 2166 ms. Trace:
I20260709 12:27:41.428284  7734 rpcz_store.cc:276] 0709 12:27:39.260930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.309701 (+ 48771us) service_pool.cc:224] Handling call
0709 12:27:39.335515 (+ 25814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.178354 (+1842839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.427449 (+249095us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54002d7dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:41.427677 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4222682880}
I20260709 12:27:41.429795  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.430684  7732 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:41.433341  7733 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:41.385977 13123 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.394554 13331 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.428895  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.442672  8409 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:41.394985 13103 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.448818  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.457047  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.458003  7735 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.462311  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.460399  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.376047 13259 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:41.471957  7734 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:41.435971 13290 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:41.425158 13264 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:41.491158  7700 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:41.469054 13189 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.432s	user 0.010s	sys 0.006s
I20260709 12:27:41.441694 13305 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:41.450800 13300 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:41.488096  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.516242  8413 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.497540  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.518370  7732 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:27:41.508095 13333 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.252s	user 0.003s	sys 0.000s
I20260709 12:27:41.512850 13328 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.514159 13305 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.417883 13332 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.527911  7735 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:27:41.423954 13314 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:27:41.489203 13335 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.489493 13338 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:41.554105  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63049.8us
I20260709 12:27:41.546483  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:41.555449  7653 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.551386  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.556488  7732 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:41.497903 13307 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.515641 13300 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.573674  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.523087 13290 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:41.584709  8413 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.571780  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:41.545855 13041 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.140s	user 0.009s	sys 0.005s
I20260709 12:27:41.588949  7731 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
W20260709 12:27:41.579172  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.558s (SENT)
I20260709 12:27:41.590101  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.585948  7735 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.481776 13331 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.593183  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.596386  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.598726  7732 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:41.600236  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.599273  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.522775 13189 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.567062 12966 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.566404 13268 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.567304 13204 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.588303 13300 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:41.563238 13346 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:41.501142 13124 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.621014 13077 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:41.621577 13077 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:41.622406 13124 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:41.612205  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.624059  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2986) took 1128 ms. Trace:
I20260709 12:27:41.624527  7806 rpcz_store.cc:276] 0709 12:27:40.495503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.541699 (+ 46196us) service_pool.cc:224] Handling call
0709 12:27:41.623725 (+1082026us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54003737d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:41.623886 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465792768}
I20260709 12:27:41.626039  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:41.613229  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.591717 12998 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:41.597671 13041 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:41.545594 13328 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.612092  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.628938  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.646961  7735 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
W20260709 12:27:41.611078 13345 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 12:27:41.657747 13282 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.289s	user 0.010s	sys 0.004s
I20260709 12:27:41.658402 13282 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.658739 13273 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:27:41.660491  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.624557 13331 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:41.664988 13295 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.292s	user 0.015s	sys 0.000s
I20260709 12:27:41.618831  7733 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.637303  7732 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:41.638516  7736 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:41.562949 13335 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.563297 13338 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.643486  7658 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:41.644169 13346 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:41.645521 13185 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.000s
W20260709 12:27:41.685993  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 106860us
I20260709 12:27:41.619273  7653 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:41.649505 13290 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.650498 13332 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:41.653395 13271 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.546s	user 0.006s	sys 0.009s
I20260709 12:27:41.665428  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 896) took 1115 ms. Trace:
I20260709 12:27:41.665243 12868 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.582739 13307 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.632841  7806 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:41.676844  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:41.668253  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.916s (SENT) [suppressed 48 similar messages]
I20260709 12:27:41.680172  8408 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
W20260709 12:27:41.680896 13148 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.003s
I20260709 12:27:41.681195  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1003) took 1068 ms. Trace:
I20260709 12:27:41.704310  7659 rpcz_store.cc:276] 0709 12:27:40.612568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.612844 (+   276us) service_pool.cc:224] Handling call
0709 12:27:40.701108 (+ 88264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.770477 (+ 69369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.770530 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.817928 (+ 47398us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.115927 (+297999us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:41.116007 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.507194 (+391187us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:41.680837 (+173643us) spinlock_profiling.cc:243] Waited 61.5 ms on lock 0x7b540021b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:41.680904 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.680956 (+    52us) raft_consensus.cc:1685] finished
0709 12:27:41.680969 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.681131 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7250,"raft.queue_time_us":8014,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":25111,"spinlock_wait_cycles":196070400,"thread_start_us":218164,"threads_started":1,"wal-append.queue_time_us":319718}
I20260709 12:27:41.689689  7809 rpcz_store.cc:276] 0709 12:27:40.549923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.591716 (+ 41793us) service_pool.cc:224] Handling call
0709 12:27:41.665193 (+1073477us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:41.665359 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511083904}
I20260709 12:27:41.703171  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.707717  7732 raft_consensus.cc:2410] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:41.718135  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:41.719166  7806 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.622531 13077 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.683826 13204 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.730342  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:41.724994  7807 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:41.732551  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.732875  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:41.733278  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.075s (SENT)
I20260709 12:27:41.736038  7732 raft_consensus.cc:2376] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:27:41.736310  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:41.733399  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 65219.1us
I20260709 12:27:41.707352 12811 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:41.738250  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:41.740190  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.740998  7732 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:41.732228 13294 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:27:41.745715 13135 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:41.741905 13305 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.734047 13356 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:41.747767 13356 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.747473 13305 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.688573 13185 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.751185  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.752378  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:41.753019  7735 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.719211 13148 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.753391  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.706022 13271 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.738757 13295 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:41.755452  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.748485 13356 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:41.766494  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:41.769194  7735 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:41.771346 13271 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.772444  7728 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.742622 12811 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.783474  7731 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.751915 13282 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.734624 13204 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.753413 13077 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:41.748994 13355 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:41.785584  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:41.792531  7810 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:41.785979  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:41.793810  7809 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.757869 13331 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:41.756203 12694 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:41.797158  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.800734  7735 raft_consensus.cc:2410] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:41.783785 12811 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.803002  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.804894  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.806008  7734 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.809770  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:41.819579  7732 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:41.820802  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.821657  7732 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.820972  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.808841 13314 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.831048 13314 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.831596  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:41.832281  7810 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:27:41.833279  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:41.792197 13355 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.823750  7735 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:41.757548 13310 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.798913 13294 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.839169  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.798463 13331 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:41.838824 13135 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:41.851630 12898 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:41.852419 13135 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.852566 12898 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.855626  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:27:41.856349  7810 raft_consensus.cc:606] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.856717  7810 raft_consensus.cc:614] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:41.857851  7809 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:41.858564  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 897) took 1308 ms. Trace:
I20260709 12:27:41.858597  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:41.845916 13338 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.789991 13346 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.860666  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.861673 13245 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:41.863045  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.863850 13245 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:41.864280 13245 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:41.865226  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.866176 13245 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.866721 13148 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.867624 13308 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:41.861268 12898 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.848450  7805 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:41.858944  7809 rpcz_store.cc:276] 0709 12:27:40.550170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.857331 (+1307161us) service_pool.cc:224] Handling call
0709 12:27:41.857648 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.858320 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.858509 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:41.860247 12897 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:41.847545  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:41.868345 13328 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.870393  7808 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.861518  7731 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:41.895963  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 895) took 1346 ms. Trace:
I20260709 12:27:41.872218  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.834000 13189 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.873397  7806 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:41.874174 13204 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.866848  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:41.875806  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:41.877065  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 869) took 1822 ms (client timeout 1916 ms). Trace:
I20260709 12:27:41.901772  8405 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14. Re-sending same reply.
W20260709 12:27:41.901916  8408 rpcz_store.cc:269] 0709 12:27:40.054733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.054928 (+   195us) service_pool.cc:224] Handling call
0709 12:27:41.876885 (+1821957us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:41.877003 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902391040}
I20260709 12:27:41.851718 13126 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.880066 12868 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.854964 13135 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.881984 13335 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:41.882287  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.886399  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:41.883107  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.852717  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.937390  7733 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.938798  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.892270 13148 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:41.940322  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54059.8us
I20260709 12:27:41.940865  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:41.941578  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:41.834328 13106 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:41.890338  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:41.837960  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.891829  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.883625  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:41.896441  7805 rpcz_store.cc:276] 0709 12:27:40.549654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.847787 (+1298133us) service_pool.cc:224] Handling call
0709 12:27:41.848106 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.892813 (+ 44707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.895901 (+  3088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:41.898739  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2988) took 1398 ms. Trace:
I20260709 12:27:41.882958 13307 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:41.898906  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.945629  7737 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:41.945755  8406 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:41.946378  7731 raft_consensus.cc:2368] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:41.946784  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:41.947283  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.947615  7737 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:41.839891 13355 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.945827  7807 rpcz_store.cc:276] 0709 12:27:40.499914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.541232 (+ 41318us) service_pool.cc:224] Handling call
0709 12:27:41.898526 (+1357294us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400331b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:41.898672 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596915584}
I20260709 12:27:41.949721  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.950006  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.909946  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.885320 13294 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.892690 13245 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:41.911171  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:41.911489  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:41.919695  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52929.4us
I20260709 12:27:41.952422  7733 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:41.954443  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45314.2us
I20260709 12:27:41.926659  7808 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:41.951016  7656 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
W20260709 12:27:41.955332  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2989) took 1454 ms (client timeout 1800 ms). Trace:
W20260709 12:27:41.955715  7808 rpcz_store.cc:269] 0709 12:27:40.501359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.502484 (+  1125us) service_pool.cc:224] Handling call
0709 12:27:41.869879 (+1367395us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b580077aed0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e7e0aa 00007fc001f00447 00007fc001f18fe7 00007fc001f1a7de 00007fc001f2627c 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c
0709 12:27:41.955268 (+ 85389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393763584}
I20260709 12:27:41.929277  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.918159  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.929332  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.906002 12998 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:41.952838  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:41.894855 12897 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:41.975853  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.512s (SENT)
I20260709 12:27:41.975919  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:41.976902  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.948395  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.954305  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.977798  7734 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.977998  7662 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:41.979118  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.979440  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.931399  8408 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:41.911285 13185 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:41.949470 13273 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:41.955983  8405 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:41.959586  7653 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:41.961645  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.958446  7655 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:41.945062 13106 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.965628  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1122) took 1650 ms. Trace:
W20260709 12:27:41.968607 13339 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.712s	user 0.004s	sys 0.000s
I20260709 12:27:41.968777 13298 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:41.970105  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.016199  7736 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:41.943181 13024 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:41.972620 13123 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:41.983918  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.018738  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:41.953496 12898 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:41.932683 13126 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:41.943956  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:41.960026 13075 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:41.900447 13356 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:41.970952 13264 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:42.017838  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.981887  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:42.019371 13106 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:42.025501  7737 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:41.973526  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:42.019812  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:42.026119  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.563s (SENT)
W20260709 12:27:42.101471  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 128020us
W20260709 12:27:42.101929  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83310.8us
W20260709 12:27:42.102953  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.028517  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.103062  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:42.031378  7735 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.988479  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:42.105518  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 117155us
I20260709 12:27:42.015762  7729 rpcz_store.cc:276] 0709 12:27:40.314983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.315300 (+   317us) service_pool.cc:224] Handling call
0709 12:27:40.349733 (+ 34433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.072404 (+722671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.965309 (+892905us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:41.965546 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483933568}
I20260709 12:27:42.037228  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.110760  7731 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:42.031047  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.037634  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.047873  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.112432  7653 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.113251  8406 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:27:41.943301 12990 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:27:42.052275  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:42.059018  7654 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:41.981477 12897 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:42.059036  7732 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:41.953358 13148 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.037039 13341 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:27:42.068173 13331 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.073122  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2852) took 1092 ms. Trace:
I20260709 12:27:42.027661 13273 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:42.077353 13314 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:42.077899  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.078681  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.036604 13309 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:42.080515  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.081286 13126 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:42.081363 13024 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:42.082306 13305 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:42.082926 13075 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:42.125896 12990 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:42.104220  7734 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:42.026222  7655 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.114296  8407 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:42.114768  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.115926 13328 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:42.117312  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:41.981884  7656 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:42.119328 13077 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:42.123365  7810 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:42.123631  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.193156 13290 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.286258  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.277642  7735 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.287904  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.289799  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.290846  7735 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.126816  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:42.126364  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:42.135807  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.321027  7731 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:42.138240  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.149386  7654 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.149425  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.150501  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.322907  7732 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:42.159986  7728 rpcz_store.cc:276] 0709 12:27:40.980891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.981060 (+   169us) service_pool.cc:224] Handling call
0709 12:27:42.072888 (+1091828us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b5400247cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:42.073050 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700126976}
I20260709 12:27:42.286770 13310 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:42.330717  7805 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:42.336898  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 210158us
I20260709 12:27:42.337971  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.339440  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.339547  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:42.331070  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.341200  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.347911  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.349166  7735 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:42.305831 13331 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:42.169322  7702 connection.cc:256] RPC call timeout handler was delayed by -3.7299080160000813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 12:27:41.985996 12998 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:42.125574 13366 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:27:42.194371  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:42.196503  7812 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 893) took 1686 ms (client timeout 1733 ms). Trace:
I20260709 12:27:42.162055  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.356808  7808 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:42.357363  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:42.160884  7733 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:42.204453  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.334834  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:42.358806  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 156267us
I20260709 12:27:42.359923  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.214913  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:42.090476 13365 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
I20260709 12:27:42.275480  7737 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 10: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:42.110105 12898 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:42.283677  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:42.367658  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4837504980000631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 12:27:42.356606  7812 rpcz_store.cc:269] 0709 12:27:40.548925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.580742 (+ 31817us) service_pool.cc:224] Handling call
0709 12:27:42.196190 (+1615448us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54003f3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:42.196408 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083225472}
I20260709 12:27:42.124377  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:42.368234  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3802504979998957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 12:27:42.204416 13309 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.151494 13298 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:42.292164  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.285565 13335 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.375450  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.303676  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.377017  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.286971 13077 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.307337  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.312460  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1002) took 1700 ms. Trace:
W20260709 12:27:42.298707 13336 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.957s	user 0.007s	sys 0.000s
W20260709 12:27:42.314456 13369 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.115s	user 0.000s	sys 0.004s
I20260709 12:27:42.359100  7731 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:42.379527 13264 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:42.375245 13309 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:42.148947 13123 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:42.270412  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:42.368911  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2893504980002035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
I20260709 12:27:42.367726  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.368306  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:42.370452  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.370975  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.363430  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.294080 13024 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.300900  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.377103  7659 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:27:42.378034  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:42.442277  7732 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:42.302829 12811 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.301401 13290 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.462795  7729 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:27:42.456051  7812 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:42.444623  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.383047  7657 rpcz_store.cc:276] 0709 12:27:40.612188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.612339 (+   151us) service_pool.cc:224] Handling call
0709 12:27:40.679812 (+ 67473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:40.784385 (+104573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.784426 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:40.868257 (+ 83831us) log.cc:844] Serialized 52 byte log entry
0709 12:27:41.123718 (+255461us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:41.123809 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.123941 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.967936 (+843995us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:42.303840 (+335904us) raft_consensus.cc:1685] finished
0709 12:27:42.303866 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.312343 (+  8477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66585,"raft.queue_time_us":80158,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":20833,"spinlock_wait_cycles":53580544,"thread_start_us":221640,"threads_started":1,"wal-append.queue_time_us":270969}
I20260709 12:27:42.459398  7735 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:42.391835  7733 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:42.386099 13189 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:42.379180  7728 raft_consensus.cc:2376] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:27:42.443717  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:42.446274  7731 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:27:42.363306 13314 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:42.362252 13308 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:42.361425  7658 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
W20260709 12:27:42.449925  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2821504980001919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260709 12:27:42.479522  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2641504980001628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
W20260709 12:27:42.480103  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3776504980000936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2746s for a response
W20260709 12:27:42.480520  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0536504980000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3106s for a response
I20260709 12:27:42.484824  7735 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:42.488610  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:42.489682  7653 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:42.452608 12694 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.001s
W20260709 12:27:42.472024  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 179991us
I20260709 12:27:42.372393 13185 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.468798  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2794) took 1106 ms. Trace:
I20260709 12:27:42.367800 13299 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:42.472733  7731 pending_rounds.cc:85] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:42.466401  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.380434 13305 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:42.375497 13296 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:42.493893  7735 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:42.445815 13264 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:42.517561  7731 pending_rounds.cc:107] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:42.528458  7731 raft_consensus.cc:2889] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.518180  7737 rpcz_store.cc:276] 0709 12:27:41.362248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.380535 (+1018287us) service_pool.cc:224] Handling call
0709 12:27:42.468258 (+ 87723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.468452 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.468478 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:42.468524 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:42.468555 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.468608 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.468717 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:42.481334  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:42.519511  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53266us
I20260709 12:27:42.522294 12990 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:42.522799 13367 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:27:42.523111  7728 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:42.386994 13077 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:42.525099  7735 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:42.451516 12998 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:42.528992  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1126) took 2174 ms. Trace:
I20260709 12:27:42.388391 13335 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.447883 13123 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:42.450145 13294 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:42.537300  8407 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:42.541940  7730 rpcz_store.cc:276] 0709 12:27:40.354402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.354568 (+   166us) service_pool.cc:224] Handling call
0709 12:27:40.354967 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.364079 (+  9112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.364118 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.759342 (+1395224us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:41.759757 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.890985 (+131228us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:41.891023 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.891097 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.089843 (+198746us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:42.528721 (+438878us) raft_consensus.cc:1685] finished
0709 12:27:42.528746 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.528910 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7462,"raft.queue_time_us":25896,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"spinlock_wait_cycles":201107456,"thread_start_us":101049,"threads_started":1,"wal-append.queue_time_us":138320}
I20260709 12:27:42.491818 13341 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:42.539862  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:42.529896 13300 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.895s	user 0.008s	sys 0.004s
I20260709 12:27:42.386258 13298 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.532866 13264 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:42.531555 12694 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:42.474516 13189 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.541381 12990 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:42.542887 13299 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:42.578755  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:42.561290  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.545529 13346 raft_consensus.cc:2764] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
W20260709 12:27:42.586400  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 105211us
I20260709 12:27:42.587991  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:42.553388 13314 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:42.463660 13075 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:42.612437 13355 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 12:27:42.561188 13300 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.613159 13355 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.614529  7811 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:42.535713 13305 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.615825 13300 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:42.546664 13185 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.617446  7734 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:42.617993 13305 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:42.618755 13185 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:42.549167 13341 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.442701 13273 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:42.582690 13368 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.306s	user 0.000s	sys 0.002s
I20260709 12:27:42.604091 13299 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:42.545697 12898 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:42.606097 13372 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:27:42.532316 13296 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.606665 13295 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:42.621043 13335 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:42.582505  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 894) took 2033 ms. Trace:
I20260709 12:27:42.648152  7736 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
W20260709 12:27:42.621584 13370 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 12:27:42.622607 13371 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:42.623174  8413 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:42.624395 13314 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.625707 13189 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:42.627887 12990 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.554251 13123 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:42.631611  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43812.7us
I20260709 12:27:42.633111  7730 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:42.634717  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 274905us
I20260709 12:27:42.674322  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:42.674983  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:42.635046  7653 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:42.630784 13294 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:42.648562  7804 rpcz_store.cc:276] 0709 12:27:40.549320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.718588 (+169268us) service_pool.cc:224] Handling call
0709 12:27:40.774598 (+ 56010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.912312 (+137714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.913623 (+  1311us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.918020 (+  4397us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.135649 (+217629us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:41.135701 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.135788 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.200160 (+1064372us) spinlock_profiling.cc:243] Waited 932 us on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:42.564286 (+364126us) raft_consensus.cc:1685] finished
0709 12:27:42.564327 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.582391 (+ 18064us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46148,"raft.queue_time_us":141461,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":132,"spinlock_wait_cycles":2144128,"thread_start_us":313616,"threads_started":2,"wal-append.queue_time_us":399319}
I20260709 12:27:42.637184 13341 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:42.647148 13185 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:42.684739  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:42.696712  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:42.649624 13106 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.531s	user 0.000s	sys 0.012s
I20260709 12:27:42.642076 12897 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.668468 13314 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:42.674772 13338 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:42.687556 13189 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:42.877578  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 180990us
W20260709 12:27:42.881886  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 325821us
I20260709 12:27:42.697610 13300 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.147s
I20260709 12:27:42.817440 13294 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:42.887405  7627 leader_election.cc:400] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:42.889777  7627 leader_election.cc:403] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:42.892531  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:42.893338  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:42.853436 12650 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.856666 12811 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:42.844952 13338 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:42.836911 13377 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:42.909862  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 904) took 1665 ms. Trace:
I20260709 12:27:42.885650 12998 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:42.888382 13290 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:42.893311 13376 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.855688 13106 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:42.926304  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.927371  7808 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:42.920105  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.931747  7812 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:27:42.906116 13355 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:42.934729  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.935525  7808 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:42.921027  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.936483  7806 raft_consensus.cc:2376] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12. Re-sending same reply.
I20260709 12:27:42.914865  7809 rpcz_store.cc:276] 0709 12:27:41.244957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.893590 (+648633us) service_pool.cc:224] Handling call
0709 12:27:41.893957 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:41.900031 (+  6074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.900057 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:41.940952 (+ 40895us) log.cc:844] Serialized 52 byte log entry
0709 12:27:42.353428 (+412476us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:42.353488 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.353572 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.909502 (+555930us) raft_consensus.cc:1685] finished
0709 12:27:42.909534 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.909777 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17204,"raft.queue_time_us":5735,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12138,"thread_start_us":333857,"threads_started":1,"wal-append.queue_time_us":425680}
I20260709 12:27:42.938432  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.923426  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.876917 13185 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:27:42.924239  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:42.943434  7804 raft_consensus.cc:2376] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:27:42.918360 13295 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.947124  8412 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:42.948204  7812 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:42.958395  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.952016  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2796) took 1356 ms. Trace:
I20260709 12:27:42.960421  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.960132  7732 rpcz_store.cc:276] 0709 12:27:41.595922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.472966 (+877044us) service_pool.cc:224] Handling call
0709 12:27:42.473190 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.473342 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:42.637057 (+163715us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.945143 (+308086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:42.945236 (+    93us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:42.951591 (+  6355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.951712 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.951929 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343645,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":427549,"threads_started":2}
I20260709 12:27:42.933813 13377 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
W20260709 12:27:42.989324  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.935360 13379 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:42.995812  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1130) took 1325 ms. Trace:
I20260709 12:27:42.906586 12650 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.005062  7732 rpcz_store.cc:276] 0709 12:27:41.670086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.978801 (+1308715us) service_pool.cc:224] Handling call
0709 12:27:42.979516 (+   715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.995322 (+ 15806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.995368 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:42.995432 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:42.995479 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.995567 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.995731 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.002135  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2503) took 1071 ms. Trace:
I20260709 12:27:43.006594  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:43.008672  7806 raft_consensus.cc:2376] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13. Re-sending same reply.
I20260709 12:27:43.009124  7732 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:43.010090  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1131) took 1329 ms. Trace:
I20260709 12:27:43.010353  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:43.011705  7804 raft_consensus.cc:2376] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12. Re-sending same reply.
I20260709 12:27:43.010478  7732 rpcz_store.cc:276] 0709 12:27:41.681075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.008512 (+1327437us) service_pool.cc:224] Handling call
0709 12:27:43.008880 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.009747 (+   867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.010022 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:42.994550  7808 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:43.013110  7656 rpcz_store.cc:276] 0709 12:27:41.930382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.388279 (+457897us) service_pool.cc:224] Handling call
0709 12:27:42.388624 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.388867 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:42.667246 (+278379us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.000230 (+332984us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.000314 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:43.000389 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.001852 (+  1463us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.002050 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":517238,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":2859,"thread_start_us":544206,"threads_started":2}
I20260709 12:27:43.026710  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2509) took 1062 ms. Trace:
I20260709 12:27:43.029150  7656 rpcz_store.cc:276] 0709 12:27:41.963905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.025203 (+1061298us) service_pool.cc:224] Handling call
0709 12:27:43.025684 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.026214 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.026244 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.026298 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:43.026331 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.026442 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.026612 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.031834  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.929678 13376 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.043677  7626 leader_election.cc:400] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:27:43.044247  7626 leader_election.cc:403] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:43.044762  7808 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:43.036455  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:42.941699 13309 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:43.036415  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 351787us
I20260709 12:27:43.003891 13298 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.047329  7656 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:42.957638 13295 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.049537  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:43.052212  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.025470 13296 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.058947  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.065819  7804 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:43.060631  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.080080  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:43.082121  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 187748us
I20260709 12:27:43.060806  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.085393  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:43.088682  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2507) took 1136 ms. Trace:
I20260709 12:27:43.034981 12694 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:27:42.983533 13294 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.075718 13298 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:43.061885 12811 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.088594  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:43.073659 13123 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:43.119333  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 226059us
I20260709 12:27:43.108603 12868 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:43.029315 13338 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.049674 13381 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:27:43.087020 12897 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.094744 13377 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:43.123144  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48484us
I20260709 12:27:43.128592  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.089705  7657 rpcz_store.cc:276] 0709 12:27:41.955819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.483460 (+527641us) service_pool.cc:224] Handling call
0709 12:27:42.530501 (+ 47041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.530792 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:42.886621 (+355829us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.088320 (+201699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.088373 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.088420 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.088490 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.088619 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9965,"prepare.queue_time_us":403840,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"thread_start_us":521100,"threads_started":2}
I20260709 12:27:43.156369  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.124735 13388 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:43.135375 13290 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:43.192616  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.171869  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2807) took 1430 ms. Trace:
I20260709 12:27:43.158201 13377 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:43.197430 13389 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:43.197901 13377 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.178646  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.124013 13185 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:27:43.155438 13338 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:43.186599  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98063.1us
I20260709 12:27:43.174958  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2797) took 1576 ms. Trace:
I20260709 12:27:43.138463 13189 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:43.212311  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.195542  7734 rpcz_store.cc:276] 0709 12:27:41.741576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.368049 (+626473us) service_pool.cc:224] Handling call
0709 12:27:43.171576 (+803527us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400212d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.171773 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369534208}
I20260709 12:27:43.213402  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.213806  7657 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:43.214015  7734 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:43.122371 12811 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.125190 13123 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:27:43.181684  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.219731  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:43.220902  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.200588 13290 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.169235  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.207913  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:43.226121  7655 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:43.208346  7733 rpcz_store.cc:276] 0709 12:27:41.598802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.469739 (+870937us) service_pool.cc:224] Handling call
0709 12:27:42.496769 (+ 27030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.627849 (+131080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.627899 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:42.627954 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:42.627993 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.174628 (+546635us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400283054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:43.174700 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.174891 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300799744}
I20260709 12:27:43.199458  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2801) took 1592 ms. Trace:
I20260709 12:27:43.214490  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2515) took 1085 ms. Trace:
I20260709 12:27:43.158013 13106 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.179101  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.193452 13379 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:43.225224  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 139893us
I20260709 12:27:43.228060  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:43.229660  7804 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:27:43.239754  7657 rpcz_store.cc:276] 0709 12:27:42.129112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.159337 (+1030225us) service_pool.cc:224] Handling call
0709 12:27:43.213458 (+ 54121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.214218 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.214434 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:43.230742  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102428us
I20260709 12:27:43.248226  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.251865  7735 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:43.253101  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.285976  7734 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:43.227654  7731 rpcz_store.cc:276] 0709 12:27:41.607405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.471927 (+864522us) service_pool.cc:224] Handling call
0709 12:27:42.472430 (+   503us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:42.681597 (+209167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.681629 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:42.691378 (+  9749us) log.cc:844] Serialized 78 byte log entry
0709 12:27:43.029780 (+338402us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.045709 (+ 15929us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.136764 (+ 91055us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.136977 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.137067 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.199132 (+ 62065us) raft_consensus.cc:1685] finished
0709 12:27:43.199157 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.199377 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18804,"prepare.queue_time_us":36219,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":94,"raft.queue_time_us":162644,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":20233,"spinlock_wait_cycles":17891328,"thread_start_us":353435,"threads_started":3,"wal-append.queue_time_us":246829}
I20260709 12:27:43.219731 13393 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:43.256681  7733 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:43.292337  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.293406  7734 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:43.230584 13290 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:43.298321  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:43.305608  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52621.9us
I20260709 12:27:43.306768  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.310057  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:43.278249 12898 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:43.312187 12898 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.253561 13024 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:43.312659 13385 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:27:43.313138  7735 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:43.311041  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:43.300115 13077 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.750s	user 0.014s	sys 0.000s
I20260709 12:27:43.321676 13309 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:43.325145  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.246943 13392 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:27:43.325654  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1134) took 1643 ms. Trace:
I20260709 12:27:43.304128 12650 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.302073 13379 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.303442 13299 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:43.336160 13376 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.341629  7735 rpcz_store.cc:276] 0709 12:27:41.681960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.324622 (+1642662us) service_pool.cc:224] Handling call
0709 12:27:43.325022 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.325250 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.325275 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.325329 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:43.325365 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.325436 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.325584 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.428588  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1135) took 1744 ms. Trace:
I20260709 12:27:43.460096  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2514) took 1331 ms. Trace:
W20260709 12:27:43.346705 13402 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:27:43.351933 13264 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.000s
W20260709 12:27:43.380892  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.471886  7736 rpcz_store.cc:276] 0709 12:27:41.684630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.368989 (+684359us) service_pool.cc:224] Handling call
0709 12:27:43.428384 (+1059395us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.428517 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422212352}
I20260709 12:27:43.467468  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 879) took 1595 ms. Trace:
I20260709 12:27:43.317592 13393 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:43.424489 13314 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.003s
I20260709 12:27:43.342151  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1136) took 1640 ms. Trace:
I20260709 12:27:43.477558  7731 rpcz_store.cc:276] 0709 12:27:41.702188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.340898 (+1638710us) service_pool.cc:224] Handling call
0709 12:27:43.341342 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.341703 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.341741 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.341803 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:43.341847 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.341913 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.342059 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:43.429857 13405 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:27:43.429049 13331 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.064s	user 0.010s	sys 0.003s
I20260709 12:27:43.339717 13295 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:43.417340  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92298.3us
W20260709 12:27:43.390139 13328 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.090s	user 0.011s	sys 0.005s
I20260709 12:27:43.473639  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:43.482882  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:43.396591 13383 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:27:43.345664 12811 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.480865 13294 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.342226 13185 consensus_queue.cc:1243] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:43.434386 13356 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:43.400270 13123 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:43.474584  8413 rpcz_store.cc:276] 0709 12:27:41.870577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.871359 (+   782us) service_pool.cc:224] Handling call
0709 12:27:41.871721 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.669482 (+797761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.465252 (+795770us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:43.465496 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727394048}
I20260709 12:27:43.469925  7656 rpcz_store.cc:276] 0709 12:27:42.128646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.052407 (+923761us) service_pool.cc:224] Handling call
0709 12:27:43.052713 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.052958 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:43.089292 (+ 36334us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.459413 (+370121us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b1800378820. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:27:43.459553 (+   140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.459634 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.459707 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.459807 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.460009 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41282,"prepare.queue_time_us":39305,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":45205,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":21721,"spinlock_wait_cycles":6517504,"thread_start_us":125657,"threads_started":2,"wal-append.queue_time_us":121820}
I20260709 12:27:43.465180 12898 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.325057 13401 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:43.458518 13290 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.489266  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.480309 13355 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.490582 13328 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:43.359558 13077 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.576890 13077 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.487128 12966 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.577651 12966 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.565238  7655 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
W20260709 12:27:43.578941  7655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2690) took 1359 ms (client timeout 1549 ms). Trace:
W20260709 12:27:43.579402  7655 rpcz_store.cc:269] 0709 12:27:42.219257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.168788 (+949531us) service_pool.cc:224] Handling call
0709 12:27:43.578868 (+410080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.567447  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.581135 13335 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:43.581710 13335 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:43.586390  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.23362e+06us
I20260709 12:27:43.570000  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:43.588841  7735 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:43.569552  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 878) took 1699 ms. Trace:
W20260709 12:27:43.574276  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85083.4us
I20260709 12:27:43.475915 12998 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:43.475633 13264 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.580493 13376 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:43.595446 13290 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.595976 13264 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.596223  7728 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:43.596891  7654 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:43.597110  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2805) took 1856 ms. Trace:
I20260709 12:27:43.597579  7728 rpcz_store.cc:276] 0709 12:27:41.740986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.595600 (+1854614us) service_pool.cc:224] Handling call
0709 12:27:43.595992 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.596834 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.597048 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.592146  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.599591  7658 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:43.600553 13390 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.375s	user 0.005s	sys 0.000s
I20260709 12:27:43.600950  7737 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:43.601320  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2506) took 1649 ms. Trace:
I20260709 12:27:43.601761  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2810) took 1493 ms. Trace:
I20260709 12:27:43.601902 13264 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:43.602296  7737 rpcz_store.cc:276] 0709 12:27:42.108351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.600344 (+1491993us) service_pool.cc:224] Handling call
0709 12:27:43.600757 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.601477 (+   720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.601698 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:43.604701  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2493) took 1695 ms (client timeout 2136 ms). Trace:
W20260709 12:27:43.605161  7659 rpcz_store.cc:269] 0709 12:27:41.908997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.909177 (+   180us) service_pool.cc:224] Handling call
0709 12:27:43.604427 (+1695250us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540033d6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.604580 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980728192}
I20260709 12:27:43.604738  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2811) took 1496 ms. Trace:
I20260709 12:27:43.608841  7737 rpcz_store.cc:276] 0709 12:27:42.108588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.603649 (+1495061us) service_pool.cc:224] Handling call
0709 12:27:43.604071 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.604282 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.604319 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.604369 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:43.604411 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.604474 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.604659 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.568261 13377 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:43.611459 13377 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:43.612986  7660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2492) took 1706 ms (client timeout 1601 ms). Trace:
I20260709 12:27:43.571489 13413 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:43.613484  7660 rpcz_store.cc:269] 0709 12:27:41.906226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.908518 (+  2292us) service_pool.cc:224] Handling call
0709 12:27:43.612738 (+1704220us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b540033d1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.612902 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57492864}
I20260709 12:27:43.614413 13413 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:43.615762 13415 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:27:43.615965  7737 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:43.617039  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2812) took 1478 ms. Trace:
I20260709 12:27:43.617250 13413 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.617959 13377 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:43.601080  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.601804  7658 rpcz_store.cc:276] 0709 12:27:41.951897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.478799 (+526902us) service_pool.cc:224] Handling call
0709 12:27:43.599398 (+1120599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.600093 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.601098 (+  1005us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540021b6d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.601251 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576346368}
I20260709 12:27:43.480882 13314 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.581174  7736 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:43.623021  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:43.617612  7737 rpcz_store.cc:276] 0709 12:27:42.138345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.610107 (+1471762us) service_pool.cc:224] Handling call
0709 12:27:43.610514 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.616787 (+  6273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.616985 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.638636  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.629220  7803 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:27:43.616194  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2685) took 1503 ms. Trace:
I20260709 12:27:43.631879  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1020) took 1751 ms. Trace:
I20260709 12:27:43.641090  7654 rpcz_store.cc:276] 0709 12:27:42.113257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.372825 (+259568us) service_pool.cc:224] Handling call
0709 12:27:43.599095 (+1226270us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540021b6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.616129 (+ 17034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2593794816}
I20260709 12:27:43.632257  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:43.636297 12897 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:43.635046 13273 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:27:43.641149  8412 rpcz_store.cc:276] 0709 12:27:41.880040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.880523 (+   483us) service_pool.cc:224] Handling call
0709 12:27:41.880892 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.956199 (+1075307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.631608 (+675409us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:43.631813 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451452288}
I20260709 12:27:43.650116  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:43.641803  7626 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.738s (SENT)
I20260709 12:27:43.591466  8415 rpcz_store.cc:276] 0709 12:27:41.870260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.870763 (+   503us) service_pool.cc:224] Handling call
0709 12:27:41.871144 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.079830 (+208686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.079879 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.080378 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.383929 (+303551us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:42.383974 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.521994 (+1138020us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:43.522088 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.522192 (+   104us) raft_consensus.cc:1685] finished
0709 12:27:43.522212 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.569466 (+ 47254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3024,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":479031552,"thread_start_us":302827,"threads_started":1,"wal-append.queue_time_us":442206}
I20260709 12:27:43.643545  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:43.632510 13296 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:27:43.644546 13290 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:43.645596  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:43.646040 12811 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:43.647190  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:43.651695  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.587601 13335 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.569695 13294 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.640276 13401 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:43.685940  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:43.647481 12998 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:43.662569  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:43.685639  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.622125 13355 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.692051  7736 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:43.688287  8415 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:43.657094  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:43.698748  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57034.7us
I20260709 12:27:43.698978  7737 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:43.697953  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:43.700286  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2815) took 1508 ms. Trace:
W20260709 12:27:43.693506  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46425.9us
W20260709 12:27:43.646966 13404 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.245s	user 0.000s	sys 0.004s
I20260709 12:27:43.693776  8413 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:43.649024 13264 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:43.700696  7737 rpcz_store.cc:276] 0709 12:27:42.192278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.656999 (+1464721us) service_pool.cc:224] Handling call
0709 12:27:43.698660 (+ 41661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.699494 (+   834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.700233 (+   739us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:43.701129  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44120.1us
I20260709 12:27:43.656059 13106 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:43.672714 13189 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:43.685233 13308 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:43.656850 13413 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:43.715086  7736 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:43.709196  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:43.647902 13328 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:27:43.703581  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:43.709537  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.714190  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2508) took 1750 ms. Trace:
I20260709 12:27:43.714110  8406 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:27:43.712523  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:43.708760  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:43.699553 12868 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:43.716679  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.705382 13123 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:43.717571  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.711110 13189 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:43.719491 13355 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:43.701112 13356 consensus_queue.cc:1175] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
I20260709 12:27:43.724512 13294 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:43.729496 13077 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:43.731408 13401 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.735766  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2816) took 1537 ms. Trace:
I20260709 12:27:43.710129  7658 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:43.704048 13376 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:43.716702  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:43.746886  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:43.816782  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70072.2us
I20260709 12:27:43.714121  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:43.818455  7654 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 10: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:43.702032 13335 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:43.711553 13024 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:43.814890  7736 rpcz_store.cc:276] 0709 12:27:42.198272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.698516 (+1500244us) service_pool.cc:224] Handling call
0709 12:27:43.713026 (+ 14510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.715602 (+  2576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.715826 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:43.820220  7653 rpcz_store.cc:276] 0709 12:27:41.963526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.676369 (+712843us) service_pool.cc:224] Handling call
0709 12:27:42.676848 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.064201 (+387353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.064234 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.084174 (+ 19940us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.474987 (+390813us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:43.475018 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.475122 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.713911 (+238789us) raft_consensus.cc:1685] finished
0709 12:27:43.713934 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.714121 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131752,"raft.queue_time_us":159421,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":26796,"thread_start_us":608308,"threads_started":2,"wal-append.queue_time_us":536964}
W20260709 12:27:43.822613  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119171us
I20260709 12:27:43.824156  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.813550  7656 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:43.716670 13409 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:27:43.825999  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 113581us
W20260709 12:27:43.772799 13416 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:27:43.829290  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:43.826176  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.838876  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.747519 13413 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:43.749912 13308 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.714373 13300 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:27:43.853646  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.826478 12650 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:43.841506 13388 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.638s	user 0.006s	sys 0.007s
I20260709 12:27:43.842401  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.740793 13314 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:43.840438 13299 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:43.816049 12868 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.841439  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:43.860280  7660 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:43.848452 13376 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:43.862346  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 908) took 2415 ms (client timeout 2205 ms). Trace:
I20260709 12:27:43.851733 13338 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.831936 13398 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:43.851959 13413 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:43.871402  7811 rpcz_store.cc:269] 0709 12:27:41.446455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.834049 (+387594us) service_pool.cc:224] Handling call
0709 12:27:43.862143 (+2028094us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.862282 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653987712}
I20260709 12:27:43.655417 13393 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.853609 12811 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:43.826143 13401 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:43.893237  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51873.5us
W20260709 12:27:43.869451 13341 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.051s	user 0.018s	sys 0.004s
W20260709 12:27:43.869635 13298 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.749s	user 0.001s	sys 0.016s
I20260709 12:27:43.869679 13388 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:43.859577 12650 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:43.826368 13077 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:43.907572  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 193537us
I20260709 12:27:43.906015 13135 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:43.909238  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:43.914070  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2684) took 1855 ms (client timeout 1647 ms). Trace:
W20260709 12:27:43.954064  7662 rpcz_store.cc:269] 0709 12:27:42.058652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.124070 (+ 65418us) service_pool.cc:224] Handling call
0709 12:27:43.909842 (+1785772us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540002f654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:43.913980 (+  4138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948780288}
I20260709 12:27:43.911386 12796 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:43.867033 13024 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:43.876091 13376 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:43.923372 13323 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:43.997825 13323 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:43.870261 12868 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:43.856155 13308 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.001204  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.872016 13189 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.914170 13123 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:27:43.904104 13398 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:43.929899  7656 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:44.015522  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.017246  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.932734  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:44.018909  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 86293.6us
W20260709 12:27:44.020922  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.021351  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:43.952502  7660 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:43.959846  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:43.963080  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:43.963421  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.948669 13298 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:27:43.967470 13376 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:43.968935 13374 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.416s	user 0.006s	sys 0.000s
I20260709 12:27:44.041464  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:43.969657 13401 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:43.962093 12796 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.953533 13356 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.938551 12811 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:43.973047  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.057925  8408 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:43.975015  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:27:44.058547  8413 raft_consensus.cc:606] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:43.970094 13024 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.035688 13426 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.998932 13323 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.904708 13338 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:43.948814 13341 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:43.984078  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:44.075721  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91712.4us
I20260709 12:27:43.939114  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:44.024263  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.025674  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.077760  7659 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:44.034924  8411 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:44.018991 13398 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:44.014154 13189 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:43.977531 13377 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.033777 13305 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:43.952898 12650 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:44.059897  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.077983  7660 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:44.048064  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2817) took 1842 ms. Trace:
W20260709 12:27:44.078586  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 169499us
I20260709 12:27:44.077298  7658 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:44.134286  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.135644  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.136408  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.136704  7658 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:44.137399  7660 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:44.076494  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:44.085079  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.040665 13388 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:27:44.063964 13400 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:44.081682  7662 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:44.145008  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.146047  7662 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:44.133868  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:44.133314  7737 rpcz_store.cc:276] 0709 12:27:42.205841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.723711 (+1517870us) service_pool.cc:224] Handling call
0709 12:27:43.724057 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.724287 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:43.853977 (+129690us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.046150 (+192173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.046231 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:44.046296 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.047784 (+  1488us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.047979 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":203731,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":12169,"raft.run_wall_time_us":20291,"spinlock_wait_cycles":10690816,"thread_start_us":320956,"threads_started":2,"wal-append.queue_time_us":215729}
I20260709 12:27:44.142048  7810 raft_consensus.cc:2368] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:27:44.154923  7662 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:27:44.154775  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 215919us
I20260709 12:27:44.156596  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2516) took 2022 ms. Trace:
I20260709 12:27:44.157928  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:44.146101  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69698us
I20260709 12:27:44.157335  7657 rpcz_store.cc:276] 0709 12:27:42.134610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.251383 (+1116773us) service_pool.cc:224] Handling call
0709 12:27:43.251929 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.252254 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:43.289933 (+ 37679us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.062839 (+772906us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b18004ba8a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:27:44.155858 (+ 93019us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.156033 (+   175us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:44.156155 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.156289 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.156498 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10442,"prepare.queue_time_us":36926,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":9651,"spinlock_wait_cycles":6656384,"thread_start_us":48197,"threads_started":1}
I20260709 12:27:44.091948 13387 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:44.094619 13328 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:44.164034  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.167176  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.166786  7662 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:27:44.171986  7656 raft_consensus.cc:2410] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:44.060406 13356 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.095777 13305 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:44.178045  7660 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:44.186972  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:44.143846 13400 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.145044 13308 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.173635 13335 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 12:27:44.191152  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58979.8us
I20260709 12:27:44.162479 13401 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:44.168676 13264 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:44.163327 13429 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:44.196174  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1142) took 1570 ms. Trace:
I20260709 12:27:44.238210  7731 rpcz_store.cc:276] 0709 12:27:42.625767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.929842 (+1304075us) service_pool.cc:224] Handling call
0709 12:27:43.930357 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.930601 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:43.951083 (+ 20482us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.195649 (+244566us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.195722 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:44.195799 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.195892 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.196067 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4478,"prepare.queue_time_us":20710,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":5239,"thread_start_us":239547,"threads_started":1}
I20260709 12:27:44.166543 13299 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:44.170495 13328 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.092280 13426 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.188092  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:44.236258  7737 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:44.142547 13410 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:27:44.256690 13410 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:44.257126 13410 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:44.258083 13429 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.258286 13356 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.200404 13425 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:44.259815 13425 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.259255 13429 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.185583 12811 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.190021 13305 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:44.260782 13425 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:44.263729  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75707.2us
I20260709 12:27:44.267472 13437 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:44.268244 13437 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.269214  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2498) took 2340 ms. Trace:
I20260709 12:27:44.270345  7661 rpcz_store.cc:276] 0709 12:27:41.928458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.928980 (+   522us) service_pool.cc:224] Handling call
0709 12:27:41.954272 (+ 25292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.954634 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.954662 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.964425 (+  9763us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.427129 (+1462704us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b1800484f60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9
0709 12:27:43.938240 (+511111us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800484f60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:44.090296 (+152056us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:44.090341 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.090420 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.268965 (+178545us) raft_consensus.cc:1685] finished
0709 12:27:44.268988 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.269147 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6086,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":40608,"spinlock_wait_cycles":42734080,"thread_start_us":151840,"threads_started":1,"wal-append.queue_time_us":2219985}
W20260709 12:27:44.251739 13431 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:27:44.251402 13400 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.253464 13314 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:44.255828  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2824) took 1053 ms. Trace:
W20260709 12:27:44.255833 12998 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
W20260709 12:27:44.273288  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 139528us
I20260709 12:27:44.281476  7737 rpcz_store.cc:276] 0709 12:27:43.202168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.185217 (+983049us) service_pool.cc:224] Handling call
0709 12:27:44.235937 (+ 50720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.255488 (+ 19551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.255752 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:44.283627  8379 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:44.284972  8379 leader_election.cc:403] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:44.286935  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:44.288532  7661 pending_rounds.cc:85] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 11
I20260709 12:27:44.289189  7661 pending_rounds.cc:107] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:27:44.194675 13300 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:27:44.290302  7661 raft_consensus.cc:2889] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:44.292133  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 134292us
I20260709 12:27:44.272970 13328 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:44.270032 13077 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.281344 13401 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:44.284011 13299 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:44.284051 13314 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.286702 12998 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:44.314777  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.294785 13419 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:44.335064 13432 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:27:44.286126 13427 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:27:44.269202 13437 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.316051 13400 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:44.285038 13024 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:27:44.306962 13421 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:44.367542  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:44.327044 13401 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.376413 13401 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.324466 13299 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:44.379321 13314 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.380095 13299 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.381287 13314 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.382292 13428 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:44.378984 13401 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:44.383908  7737 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:44.384330  7737 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:44.386023  8411 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:27:44.368535  7655 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:44.375474 13335 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:44.371868 13421 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:44.355724 13309 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:44.385537 12796 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.367925 13341 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.423090  7737 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:44.362365 13439 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:44.419075 13401 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.414126 13338 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.416821 12998 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:44.496268 12998 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.498930 12998 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:44.499411 13435 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 12:27:44.499863 13441 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:27:44.451961 13309 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:44.502887  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.502990 13309 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.439823 13421 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:44.504415  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.505934  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.506762  7658 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:44.487162  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56219.9us
I20260709 12:27:44.512777 13433 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:44.513386  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1129) took 2843 ms. Trace:
I20260709 12:27:44.513734 13433 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.514034  7730 rpcz_store.cc:276] 0709 12:27:41.669697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.681295 (+1011598us) service_pool.cc:224] Handling call
0709 12:27:42.818030 (+136735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.840845 (+ 22815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.840901 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.854973 (+ 14072us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.184537 (+1329564us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800330f40. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:44.286123 (+101586us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:44.286168 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.286451 (+   283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.513041 (+226590us) raft_consensus.cc:1685] finished
0709 12:27:44.513070 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.513317 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83636,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":6859,"spinlock_wait_cycles":8091392,"thread_start_us":101377,"threads_started":1,"wal-append.queue_time_us":1486037}
W20260709 12:27:44.517416  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.517530  7737 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:44.518970  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.099s (SENT)
I20260709 12:27:44.519507  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:44.528654  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:44.529645  7730 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:44.531473  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60393.7us
I20260709 12:27:44.537017  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.544449  7658 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:44.451050 13296 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
W20260709 12:27:44.523455 13440 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:27:44.523705 12868 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.006s
I20260709 12:27:44.554224 12650 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:44.555437 13135 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.004s
I20260709 12:27:44.555086 13328 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:44.556921 13376 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.002s
I20260709 12:27:44.555907 13135 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:44.557402 13376 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.557741 13433 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:44.558576  7656 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:44.559242 13433 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:44.559886 13433 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.524194 13338 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:44.516533 13401 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.577363 13376 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:44.584098  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.604066 13401 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:44.608191 13398 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.002s
I20260709 12:27:44.608728 13398 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.586050 13305 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.613547 13305 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.592221  7733 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:44.614485  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 2884) took 2932 ms. Trace:
I20260709 12:27:44.614840  7733 rpcz_store.cc:276] 0709 12:27:41.681908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.591654 (+2909746us) service_pool.cc:224] Handling call
0709 12:27:44.592020 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.614200 (+ 22180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.614422 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:44.615336  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.615934  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:44.616549  7733 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:44.616645  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.591868 13426 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.563704 12868 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.595656 13421 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.596622 13308 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:44.618036 13308 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.596885  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.619757 12868 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.586917 13419 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:44.620222  7654 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:44.613410  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.593932  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2798) took 2993 ms. Trace:
I20260709 12:27:44.622416  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1133) took 2940 ms. Trace:
I20260709 12:27:44.621906  7559 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 8 to 4, term changed from 10 to 8, leader changed from <none> to d72176ccaed049958ac68f706d4f052d (127.6.76.195), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from VOTER to NON_VOTER. New cstate: current_term: 8 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:27:44.619766 13421 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.622828  7732 rpcz_store.cc:276] 0709 12:27:41.681696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.017196 (+1335500us) service_pool.cc:224] Handling call
0709 12:27:43.177247 (+160051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.190942 (+ 13695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.190996 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.191739 (+   743us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.311560 (+119821us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:43.311620 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.311697 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.927915 (+616218us) raft_consensus.cc:1685] finished
0709 12:27:43.927943 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.622124 (+694181us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400212d50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:44.622349 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13060,"raft.queue_time_us":76295,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":32756,"spinlock_wait_cycles":366788352,"thread_start_us":86910,"threads_started":1,"wal-append.queue_time_us":257631}
I20260709 12:27:44.619100 13308 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.620957  7659 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:44.617352  7653 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.630558 13379 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.630995  7654 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:44.631160 13379 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.633526 13379 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:44.636934 13228 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:44.637487 13228 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:44.638476 13228 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.628676 13308 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:44.642493  7737 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:27:44.643172  7737 pending_rounds.cc:85] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 6
I20260709 12:27:44.643558  7737 pending_rounds.cc:107] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:44.643911  7737 raft_consensus.cc:2889] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:44.646092  7810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 2997) took 1727 ms (client timeout 1738 ms). Trace:
I20260709 12:27:44.648037 13356 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:44.648816 13356 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:44.653771 13421 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.654439 13421 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.654599 13338 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:44.655471 13338 consensus_queue.cc:1243] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:44.655628  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 912) took 1595 ms (client timeout 1974 ms). Trace:
W20260709 12:27:44.656050  7806 rpcz_store.cc:269] 0709 12:27:43.060426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.060611 (+   185us) service_pool.cc:224] Handling call
0709 12:27:44.655449 (+1594838us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54003c46d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:44.655565 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385633664}
I20260709 12:27:44.656669 13421 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.656881  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.656002 13338 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:44.701040 13338 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:44.701748 13417 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.807s	user 0.004s	sys 0.000s
I20260709 12:27:44.702033 13338 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.702615 13433 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:44.650319 13356 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.703735 13356 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:44.658430  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.704233  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1139) took 2853 ms. Trace:
W20260709 12:27:44.704434  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2489) took 2565 ms (client timeout 2150 ms). Trace:
W20260709 12:27:44.661453 13413 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.003s
I20260709 12:27:44.705718 13413 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.642329  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:44.707396  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2911) took 1413 ms. Trace:
I20260709 12:27:44.707778  7734 rpcz_store.cc:276] 0709 12:27:43.294345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.641944 (+1347599us) service_pool.cc:224] Handling call
0709 12:27:44.707326 (+ 65382us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:44.708799  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2740652630000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 12:27:44.709380  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2569652629999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:27:44.646811  7810 rpcz_store.cc:269] 0709 12:27:42.918253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.923077 (+  4824us) service_pool.cc:224] Handling call
0709 12:27:44.645587 (+1722510us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000686d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:44.645943 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2764397952}
W20260709 12:27:44.709793  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2560652629999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 12:27:44.710126  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1678652629998396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 12:27:44.710430  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1417652629997974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:27:44.710745  7702 connection.cc:256] RPC call timeout handler was delayed by -1.119265262999761s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
I20260709 12:27:44.621788  7729 rpcz_store.cc:276] 0709 12:27:41.600958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.470752 (+869794us) service_pool.cc:224] Handling call
0709 12:27:42.522440 (+ 51688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.624158 (+101718us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:42.886944 (+262786us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.076295 (+189351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.076355 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.076401 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.515167 (+438766us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:43.736129 (+220962us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b1800445240. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:44.593622 (+857493us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54001c43d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:44.593694 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.593864 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243388,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":3952,"raft.queue_time_us":77154,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":38564,"spinlock_wait_cycles":284146816,"thread_start_us":446969,"threads_started":2,"wal-append.queue_time_us":307842,"wal-append.run_cpu_time_us":6441,"wal-append.run_wall_time_us":1400869}
I20260709 12:27:44.677199  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:27:44.711047  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0733652630001416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 12:27:44.711908  7812 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:27:44.712260  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0553652630001125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:27:44.578984 13394 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.277s	user 0.000s	sys 0.007s
I20260709 12:27:44.688794  7661 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:44.689863  7657 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
W20260709 12:27:44.704829  7807 rpcz_store.cc:269] 0709 12:27:42.139536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.205490 (+ 65954us) service_pool.cc:224] Handling call
0709 12:27:44.704259 (+2498769us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003723d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:44.704366 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2599963520}
I20260709 12:27:44.708761 13413 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:44.716063 12966 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.715906  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.715994 13346 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:44.717411 13346 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:44.718092  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.718623  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:44.600076 13410 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:44.690927  7654 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:44.691124  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:44.724159  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.725630  7654 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:44.696648  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:44.697880  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:44.697796  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:44.700449  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.726642  7731 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:44.672374  7653 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:44.705039 13338 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:44.708935  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:44.731869  7734 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:27:44.677402 13264 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:44.712706  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0355652630000804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 12:27:44.741912  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.719899  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 202909us
I20260709 12:27:44.713402  7733 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:44.661602  7656 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:44.720934  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:44.728860  7731 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:44.704658  7728 rpcz_store.cc:276] 0709 12:27:41.850805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.598436 (+1747631us) service_pool.cc:224] Handling call
0709 12:27:43.598844 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.607414 (+  8570us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:43.607816 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.699360 (+ 91544us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.702739 (+  3379us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.702892 (+   153us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:43.703154 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.703401 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.703479 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.580120 (+876641us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b180032b9c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:44.682521 (+102401us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:44.704026 (+ 21505us) raft_consensus.cc:1685] finished
0709 12:27:44.704052 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.704175 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5470,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":85,"raft.queue_time_us":11999,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":42381,"spinlock_wait_cycles":62242432,"thread_start_us":91071,"threads_started":1,"wal-append.queue_time_us":220276}
I20260709 12:27:44.744029  7733 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:44.745592  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1015) took 1014 ms. Trace:
I20260709 12:27:44.732648  7730 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:44.748932  7730 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:44.727378  7732 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:44.740698  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:44.741870  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0175652630000513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:27:44.756191  7702 connection.cc:256] RPC call timeout handler was delayed by -1.01666526300005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
I20260709 12:27:44.757764  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:44.743798  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1026) took 1008 ms. Trace:
I20260709 12:27:44.759016  7656 rpcz_store.cc:276] 0709 12:27:43.735120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.660872 (+925752us) service_pool.cc:224] Handling call
0709 12:27:44.661234 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.743426 (+ 82192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.743726 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:44.761734  7732 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:44.611129 13398 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:44.783329  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.740761  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:44.780584  7728 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:44.784919  7729 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:44.746008  7662 rpcz_store.cc:276] 0709 12:27:43.730921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.172287 (+441366us) service_pool.cc:224] Handling call
0709 12:27:44.173540 (+  1253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.173822 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:44.287638 (+113816us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.743848 (+456210us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.745294 (+  1446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.745345 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:44.745386 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.745446 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.745539 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146824,"prepare.run_cpu_time_us":5851,"prepare.run_wall_time_us":17942,"thread_start_us":501068,"threads_started":2}
I20260709 12:27:44.780927  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:44.785432  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:44.786654 13442 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:44.788708  7653 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:44.792738  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:44.794603  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1018) took 1061 ms. Trace:
I20260709 12:27:44.795396  7660 rpcz_store.cc:276] 0709 12:27:43.732732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.187915 (+455183us) service_pool.cc:224] Handling call
0709 12:27:44.188420 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.190549 (+  2129us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:44.314269 (+123720us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.794067 (+479798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.794152 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:44.794217 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.794321 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.794512 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42014,"prepare.queue_time_us":182844,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":1645,"thread_start_us":545296,"threads_started":2}
I20260709 12:27:44.800060  7658 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:44.800876  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:44.801549  7658 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:44.798444  7660 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:27:44.803061  7658 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 10. Current term is 11. Ops: []
W20260709 12:27:44.804241  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 284281us
I20260709 12:27:44.806563  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.806800  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.807426  7810 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:44.807660  7811 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:27:44.721002 13379 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:44.815428  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:44.817001  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:44.781173 13264 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.788008 13410 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:44.780826 13309 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:44.820138  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:44.832932  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.833981  7807 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:44.821233  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.780381 13453 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.821616  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.825042  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.828962  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.816883  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.838317  8408 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:44.827031  7659 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:44.840639  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:44.841718  7810 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:44.811152 13442 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:44.844547  7804 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:44.836630  7809 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:44.813715 13189 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:44.812163 13328 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:44.856956  7812 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:44.858901  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.860705  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.818765 13305 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:44.865206  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:44.873113  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.874074  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:44.874224  8405 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:44.875697  7804 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:44.877449  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.880862  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.824321 13425 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:44.884055  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:44.888523  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:44.891481  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.900779  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.901278  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.902282  8412 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:44.903053  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:44.904150  8415 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.906991  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.908388  8409 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:27:44.924171  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:44.925179 13457 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:27:44.926671 13463 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:44.928824  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.862659 13346 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:44.881337 13433 raft_consensus.cc:2764] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:44.880126 13376 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:44.849581 13355 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:27:44.827031 13379 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:44.957371  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 175153us
I20260709 12:27:44.963979  7810 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:44.964556  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:44.965103  8413 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:44.955703  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:44.971016  8415 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:27:44.980798  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.953035 13400 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.000s
W20260709 12:27:44.954046 13434 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:27:45.019100  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63531.8us
I20260709 12:27:45.020138  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.005190  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:44.978722  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:44.894378 13309 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:45.009711  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:44.848590 13453 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:45.016232  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.021440  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56965.2us
I20260709 12:27:45.032418  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:45.024996  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 224276us
W20260709 12:27:45.035146  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56642.1us
I20260709 12:27:45.036029  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:45.027945  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:45.037693  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:45.034034  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.044441  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:45.082778  8415 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:45.080946  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:44.906546 13425 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.085695  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:45.090067  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.090883  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53300.1us
I20260709 12:27:45.023980 13376 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.023743 13433 raft_consensus.cc:2764] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:45.091945  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:45.034449 12796 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.034767 13299 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:45.096463  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:45.099304  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:45.100455  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:45.085625 13393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.010s	sys 0.014s
I20260709 12:27:45.078647 13123 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:45.103298 13123 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:45.103930 13123 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:45.085902 12868 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.104738 13398 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:45.045495 13467 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.106521 13467 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.107585 13106 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:45.108893 13106 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:45.078588 13135 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:45.078718 13189 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:45.090988 13462 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:27:45.102419  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.124707  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:45.087142 13335 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:45.087491 13346 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:45.137099  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51481.6us
I20260709 12:27:45.090513 13442 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:45.138553  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.124013 12796 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:45.145610 13461 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:45.116719 13273 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:27:45.145155  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.122447 13189 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:45.147712 13189 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:45.157168 13379 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.157886 13379 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:45.158701 13379 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:45.123692 13295 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:45.162846 13425 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:45.164346 13468 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:27:45.171871 13459 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:27:45.172019 12796 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.173303  7805 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:27:45.136037 13328 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.175590 13456 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:45.097813 13428 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:45.176564 13428 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.176977 13428 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:45.172153  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2482) took 3095 ms (client timeout 1794 ms). Trace:
W20260709 12:27:45.181895  7803 rpcz_store.cc:269] 0709 12:27:42.076457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.077638 (+  1181us) service_pool.cc:224] Handling call
0709 12:27:45.171928 (+3094290us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003ea3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:45.172078 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3192394624}
W20260709 12:27:45.102382 13393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.011s	sys 0.015s
I20260709 12:27:45.194199  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2490) took 3047 ms. Trace:
I20260709 12:27:45.195298  7805 rpcz_store.cc:276] 0709 12:27:42.146888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.379866 (+232978us) service_pool.cc:224] Handling call
0709 12:27:45.172868 (+2793002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.173781 (+   913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.193862 (+ 20081us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b54003ea3d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:45.194105 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6422492032}
I20260709 12:27:45.201632  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:45.203148  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 59335.4us
I20260709 12:27:45.205502  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:45.205788  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:45.206183  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.206952  7803 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:45.209882  7806 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:27:45.215708  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.217804  7808 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:27:45.218722  7808 pending_rounds.cc:85] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 10
I20260709 12:27:45.125192 12650 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:45.222987  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:45.226146  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:45.228286  7807 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:45.228756  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:45.227748  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.219139  7808 pending_rounds.cc:107] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:27:45.235579  7808 raft_consensus.cc:2889] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:45.099108 13421 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:27:45.240592  7729 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:45.242069  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:45.230124 13295 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:27:45.247077  7733 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:45.245481  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:45.249066  7732 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:45.249879  7733 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:45.238416  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:45.226626 13470 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:27:45.271381 13413 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:45.253638  7733 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:45.254760  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.255755  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:45.240772 13421 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:45.190338 13442 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:45.230296 13335 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:45.219985 13425 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:45.137773 13309 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.274601  7808 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:45.275815  7803 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:45.220614 13398 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.271922 13413 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:45.280298  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2818) took 2759 ms. Trace:
I20260709 12:27:45.277664  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:45.299508  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 192039us
I20260709 12:27:45.299707  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:45.302841  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:45.303706  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:45.325162  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:45.331079  7662 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:27:45.331189  7654 raft_consensus.cc:2368] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:45.278560  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.329267  7736 rpcz_store.cc:276] 0709 12:27:42.521483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.929780 (+1408297us) service_pool.cc:224] Handling call
0709 12:27:43.951945 (+ 22165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.952461 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:44.710217 (+757756us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b18003b33e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:27:44.710498 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.938452 (+227954us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.006022 (+ 67570us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.006262 (+   240us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:45.006313 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.006415 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.272811 (+266396us) raft_consensus.cc:1685] finished
0709 12:27:45.280042 (+  7231us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.280226 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3898,"prepare.queue_time_us":755710,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":7084,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":25781,"spinlock_wait_cycles":7767296,"thread_start_us":220370,"threads_started":1,"wal-append.queue_time_us":373894}
I20260709 12:27:45.243522 13135 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:45.336225  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:45.336963  7656 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:45.338442  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60890.8us
I20260709 12:27:45.339668  7733 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:45.340524  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:45.342758  7733 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:45.277196 13421 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:45.334672 13413 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:45.355194 13421 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.356381  8411 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:45.356868 13421 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:45.344878 13400 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:45.359756 13400 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:45.344569 13308 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:45.346633 13410 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.003s
I20260709 12:27:45.347654 13442 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.336632 13425 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:45.349419  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:45.228820 13123 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.352733 13189 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.358966  7656 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:45.298674 12868 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.333123 13299 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.362198  7811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.008s
I20260709 12:27:45.298072 13335 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:45.326609 13346 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:45.365450  7806 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:45.375717  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:45.375401  7811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.005s	sys 0.008s
W20260709 12:27:45.380769  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122314us
W20260709 12:27:45.367877  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 222835us
I20260709 12:27:45.383132  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.360687 13400 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.387269  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:45.388948  7736 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:45.373538 13314 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:45.403445 13308 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:45.376832 13335 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:45.413151  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83345.2us
I20260709 12:27:45.419016  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.373821 13338 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:45.376631 13401 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:27:45.379038 13356 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:27:45.373598 13487 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:27:45.410238 13314 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.379988 13299 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.417192 13308 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.451126 13488 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:45.438849  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.479255  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.455854 13135 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.433965 13410 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:45.472301  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90079.1us
I20260709 12:27:45.421368 12868 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:27:45.466760 13314 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:45.483628 13308 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:45.499670 13398 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.483481  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:45.470165  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52585.5us
I20260709 12:27:45.504022 13189 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:45.556607  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73414.7us
I20260709 12:27:45.465138 12796 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 12:27:45.560712  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:45.560305  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:45.542029 13456 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:45.580080  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:45.558172 13488 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:27:45.594681  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 181670us
I20260709 12:27:45.595538 13438 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.596136 13489 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:45.596340 13399 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:45.596146 13395 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:27:45.594210 13398 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:45.598732  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.613440 13482 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.614498 13423 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.698s	user 0.006s	sys 0.000s
I20260709 12:27:45.618557  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:45.498530 13346 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.471158 13425 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.642614 13331 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:45.672937  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.643028 13490 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:27:45.650226  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51735.9us
I20260709 12:27:45.658160  8411 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:45.597675 13410 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:27:45.679281  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.657135 13123 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:45.633273 13299 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.664054 13398 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:45.655776 13399 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:45.676785  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 150963us
I20260709 12:27:45.673764 13485 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:45.671954 13488 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:45.712680  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 155335us
I20260709 12:27:45.672947 13331 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:45.654666 12868 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:27:45.714455  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:45.654603 13075 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:45.667894 13489 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:45.717567  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 886) took 2035 ms. Trace:
I20260709 12:27:45.633733 13438 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.770272 13484 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:45.778297 13507 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:27:45.727705 13474 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:45.676605 13482 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.659490 13428 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.836396  8379 leader_election.cc:400] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:45.862978  8379 leader_election.cc:403] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:27:45.841670  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 127314us
W20260709 12:27:45.832722 13264 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.957s	user 0.009s	sys 0.007s
I20260709 12:27:45.871392  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:45.846807  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:45.678632 13502 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
I20260709 12:27:45.834017 13399 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:45.844174  8407 rpcz_store.cc:276] 0709 12:27:43.682825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.694960 (+ 12135us) service_pool.cc:224] Handling call
0709 12:27:43.695343 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.695659 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.695692 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.696158 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.858877 (+162719us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:43.858939 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.859017 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.613024 (+1754007us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:45.717092 (+104068us) raft_consensus.cc:1685] finished
0709 12:27:45.717127 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.717479 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20432,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":34053376,"thread_start_us":124845,"threads_started":1,"wal-append.queue_time_us":203543}
I20260709 12:27:45.865355  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.886427 13503 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:45.887252 13503 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:45.887823  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:45.888674 13442 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.888371 13503 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.854110 13189 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:45.893631  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:45.897343 13442 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:45.899320 13488 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:45.915689 13453 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:45.874822 13474 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:45.859763 13421 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:45.921876  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.884732 13419 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:45.921430  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:45.923139  7807 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:45.924613  7808 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:45.839039 13485 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.937529  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:45.874537 13410 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:27:45.957131  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.968420  7559 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 3 to 4, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 4 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.970139 13148 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.971354  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:45.973206  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:45.975546 13295 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:45.976171 13295 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:45.976894 13338 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:45.977387 13338 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:45.977165 13295 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.978397 13338 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.982200 13489 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.982223 13338 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:45.971190  7559 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 5 to 7, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.990036  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:45.990943  8407 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:45.992456  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:45.993283  8407 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:45.912235 13331 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.996812 13331 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:45.921661 13398 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:45.872915 13264 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:45.940737 13438 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:45.940872 13499 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.000s	sys 0.003s
I20260709 12:27:46.033640 13264 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:45.941174 13510 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.003s	sys 0.000s
I20260709 12:27:45.943881 13428 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.945613 13491 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:27:45.947213 12650 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:45.897889 13442 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:45.941182 13500 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:46.056226 13500 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.056675 13500 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.062685 13467 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:45.953299 13482 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:45.955822 13474 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:45.940227 13505 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.052s	user 0.003s	sys 0.000s
W20260709 12:27:45.940570 13494 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.176s	user 0.000s	sys 0.004s
I20260709 12:27:46.016991  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.019418 13376 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:46.021266 13413 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.007s
W20260709 12:27:46.048471 13335 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 12:27:46.025849 13485 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.047818 12650 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:46.055614 13442 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:46.047358 13314 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.502s	user 0.004s	sys 0.010s
W20260709 12:27:46.090657  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 197151us
I20260709 12:27:46.093377  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:46.108520  7657 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:46.100069 13517 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:46.136682  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.138686  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:46.142227  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:46.089072 13335 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.114661 13485 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.088066 13376 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.088595 13413 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.099538 13314 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.158660  7657 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:46.149896 12990 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:46.154731 13498 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:46.150677 13517 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.180984  7807 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:27:46.184394  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.187361  7657 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:46.191546  7808 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
W20260709 12:27:46.216454  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 322873us
I20260709 12:27:46.155457 12650 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:46.211051 13308 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.652s	user 0.015s	sys 0.000s
I20260709 12:27:46.154464 13516 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:27:46.178146 13503 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:27:46.254685  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.271649  7806 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:46.196499 13517 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.255833  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.174074 13485 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.163848 13492 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:27:46.230731  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.263530  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.264379  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.278852  8407 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:46.253196 13525 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:46.266335  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.283931  7731 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:46.285056  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.285323  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:46.279925  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.285815  7731 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:46.256021 12966 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.290903  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.293848  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.292297  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.295188  7731 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:46.295861  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.297127  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.298305  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:46.299081  7655 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.299456  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.302109  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.304446  7736 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:46.305701  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.306505  7659 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:46.309394  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.311908  7661 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:27:46.312652  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.316663 13514 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:46.317006 13506 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:46.317286 13514 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:46.317785 13514 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.318473 13298 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.319024 13298 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.321066 13298 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.321584 13298 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.322062 13298 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.316418  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.324110 13314 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.272578 13438 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:27:46.326792  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.299333 12966 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.327521  7733 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:46.328676 13298 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.329460 12966 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:46.326787  7731 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:46.336018 13376 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.336930  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.338064  7731 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:46.339967  7731 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:46.340131 13485 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.342567  7733 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:46.277832 13308 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.355945 13308 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.355940  7731 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:46.353711 13514 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.345201 13442 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.348793 13346 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.360108 13346 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:46.361742  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.362457 13346 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:46.364141 13526 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:27:46.364487  7734 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:46.347970  8407 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:46.342967  7655 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:46.274154 13489 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:27:46.365607  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.343307 13335 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:46.368153  7810 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:46.372074  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.373610  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.374679  7736 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:46.375685  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.338449 13413 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:46.386804 13530 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:27:46.416807  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2542) took 1716 ms. Trace:
I20260709 12:27:46.417243  7653 rpcz_store.cc:276] 0709 12:27:44.714480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.789601 (+ 75121us) service_pool.cc:224] Handling call
0709 12:27:44.817571 (+ 27970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.817927 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.817960 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.822303 (+  4343us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.853031 (+1030728us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b180004f660. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:46.167091 (+314060us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:46.167134 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.167259 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.416478 (+249219us) raft_consensus.cc:1685] finished
0709 12:27:46.416499 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.416740 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":760,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8333,"spinlock_wait_cycles":3746560,"thread_start_us":313848,"threads_started":1,"wal-append.queue_time_us":1450399}
W20260709 12:27:46.392814  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 240325us
I20260709 12:27:46.378755 13438 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:46.421018 13413 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:46.424119  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.365574 13442 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:46.429042  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.434806  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.469466  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:46.470654  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.470932  7810 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:27:46.489113  7806 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:46.489523  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54811.1us
I20260709 12:27:46.491987  8379 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13.
I20260709 12:27:46.436503 13413 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:46.527140  7735 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:46.523346 13305 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:46.540658  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 283732us
I20260709 12:27:46.494692 13428 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:27:46.525874 13491 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:46.495790 13308 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:27:46.548705  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:46.553568  7806 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:46.535516 13413 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:46.546373 13305 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:46.551461 13399 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.003s
W20260709 12:27:46.563534  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70421.1us
I20260709 12:27:46.565793  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:46.568049  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:46.548226 13514 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.571360 13399 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.569890 13413 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:46.573138 13534 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:46.579954  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:46.569648 13305 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:46.569727 13540 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:27:46.546761 13500 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.585477  8408 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:27:46.586016  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.586843  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.591950  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:46.574865 13429 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:46.587404 13542 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:46.607092  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.607879  7806 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.613596 13492 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:46.614161 13492 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:46.615259 13492 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.616665 13492 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.618096 13500 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.620888  7806 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:46.621347 12898 raft_consensus.cc:993] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:27:46.622116 12898 raft_consensus.cc:1081] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
W20260709 12:27:46.625941 13538 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:27:46.644574 13228 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.605041 13542 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:46.647166  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.660586 13413 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:46.661514 13442 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:46.647799  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.662488  7728 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:46.662524 13399 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.664674  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.666306 13442 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:46.666399  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.666828 13442 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.667451 13410 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:46.668502 13410 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:46.669157 13410 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:46.671458 13346 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:46.671970 13346 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:46.672958 13346 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.674466 13346 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:46.650604 13412 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.675845 13412 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.678180 13412 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.679172 13412 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.679888 13412 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:46.651944 13536 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:27:46.683701  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:46.686594  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.687335  7735 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.688390  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.670228 13410 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.661031  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.693233  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.694521  7804 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:27:46.694742  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.696138  7810 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:46.696611  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.698182 13489 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:46.698704  8414 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:46.667208 13442 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.701084  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:46.656884 13542 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.708755  8413 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:46.712813  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:46.713752  8414 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:46.689363  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.717119  7734 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:46.722688  7728 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:46.716671  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:46.710804 13481 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:46.717198 13433 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
W20260709 12:27:46.750696  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52812.3us
I20260709 12:27:46.724989  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2850) took 1477 ms. Trace:
I20260709 12:27:46.799566  7728 rpcz_store.cc:276] 0709 12:27:45.247276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.247993 (+   717us) service_pool.cc:224] Handling call
0709 12:27:46.647662 (+1399669us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfdeee 000000000036eb39 00007fc006e711e0 00007fc006f3a0c4 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:27:46.724879 (+ 77217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5013760}
I20260709 12:27:46.755533  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:46.803488  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48071.8us
I20260709 12:27:46.805634  7728 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:46.806429  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:46.807468  7810 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.785120  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:46.731915  7735 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:46.800724  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:46.754120  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.822088  7656 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:46.823886  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:46.826512  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.827404  7655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:40408
I20260709 12:27:46.828194  7655 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:46.828712  7655 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:46.832196  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.833328  7656 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:46.841852  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:46.845561  7653 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:27:46.848883  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 132289us
I20260709 12:27:46.849880  7810 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:46.853556  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:46.765036 13426 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.755189 13481 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.754968 13438 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:46.784910 13412 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:46.725880 13485 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 12:27:46.862607  7655 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.882910  7662 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:46.814283 13377 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.826619 13433 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:46.888487  7655 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:46.889846  7661 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:46.850170 12990 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:46.896014  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.848961 13413 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:46.849989 13494 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:27:46.897935  8405 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:46.752192 13517 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.907490  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:46.910116 13552 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:27:46.910313  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2846) took 2277 ms (client timeout 1928 ms). Trace:
W20260709 12:27:46.910842  7737 rpcz_store.cc:269] 0709 12:27:44.632642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.910005 (+2277363us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:46.910195 (+   190us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:27:46.912081  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.912851  7737 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:46.913960  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.914969  7737 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:46.883258 12796 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.881548 13500 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:46.929385  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.891892 13433 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.936861  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.847683 12694 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:27:46.942446  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:46.945637 13514 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:46.954321  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:46.955092  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.956086  8412 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:46.956490  8412 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:46.957405  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.958590  7733 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:46.881456 13399 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:27:46.941239 13541 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
W20260709 12:27:46.933518 13534 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:46.905740 13489 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.948411  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.950258  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:46.951084  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:46.971302  7653 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:46.918653 13377 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:46.952862  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:46.957633 13495 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.395s	user 0.002s	sys 0.011s
I20260709 12:27:46.961737  8408 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:27:46.955129  7732 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.943259  7804 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:46.894394 13481 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:46.963268  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.938863 12796 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.940642 13433 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:46.966946  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:46.974658  7627 connection.cc:256] RPC call timeout handler was delayed by -1.171769470000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 12:27:46.977711  7662 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:46.890455 13426 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.981283 13421 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.988377 13421 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:46.993217  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 921) took 1786 ms. Trace:
I20260709 12:27:46.949965 13532 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:46.979280 13377 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:46.994474  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.982683  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:46.995599  7732 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:46.995898  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:46.993690  7812 rpcz_store.cc:276] 0709 12:27:45.206940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.208016 (+  1076us) service_pool.cc:224] Handling call
0709 12:27:45.208311 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:45.208566 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.208593 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.220360 (+ 11767us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.416423 (+196063us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:45.416461 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.416544 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.956840 (+1540296us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:46.992911 (+ 36071us) raft_consensus.cc:1685] finished
0709 12:27:46.992932 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.993145 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15200000,"thread_start_us":136499,"threads_started":1,"wal-append.queue_time_us":271651}
I20260709 12:27:47.002298  7733 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:47.003952  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.005236 13487 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:47.006048 13565 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:27:47.005954 13487 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.007107 13487 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:46.987324 13564 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:46.987001 13485 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.010325  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.012914 13517 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.013419 13426 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.017467 12796 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:47.018132  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2849) took 1771 ms (client timeout 1963 ms). Trace:
W20260709 12:27:47.018662 13553 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 12:27:47.023896  7729 rpcz_store.cc:269] 0709 12:27:45.246948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.247103 (+   155us) service_pool.cc:224] Handling call
0709 12:27:47.017871 (+1770768us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54000d2ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:47.018052 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181209984}
I20260709 12:27:46.978305 13489 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.028764  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.011159 13564 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:47.047144 13442 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.023633 13485 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.060958 13442 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.061129 13485 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.029316 13135 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:47.061859 13135 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:47.062884 13135 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.064051 13135 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.069136 13489 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.070282 13485 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:47.077668 13487 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.080650 13484 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.082998 13564 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.067019 13305 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.090286 13421 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.086520 13413 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:47.028708 13567 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:47.040884 13554 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.186s	user 0.000s	sys 0.002s
W20260709 12:27:47.087142  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 145071us
I20260709 12:27:47.088974 13377 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.091138  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:47.094338  8414 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
W20260709 12:27:47.095868  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 169721us
I20260709 12:27:47.085415 13500 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:47.087128 12694 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:47.123667 12694 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:47.125010 12694 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.126714 12694 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:47.087172 13419 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:47.108320 13484 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:47.128230  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.131168  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:47.133013  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104336us
I20260709 12:27:47.133468  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.133809  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.116608 13567 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.138583  7658 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:47.140096  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:47.136896  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:47.146801  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.148861  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.154455  7654 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:47.142876 13346 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.128980 12796 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:47.127777 13377 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.157091  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.160235  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:47.166661  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:47.168532  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:47.169164  7658 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.160127 13410 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.151517  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.141527 13489 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:47.133900 13564 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.143337 13567 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:47.178117  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.159510 13346 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.153565 13570 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:27:47.179768 13410 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.211691  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3031) took 1287 ms. Trace:
I20260709 12:27:47.212268  7807 rpcz_store.cc:276] 0709 12:27:45.924202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.938715 (+ 14513us) service_pool.cc:224] Handling call
0709 12:27:45.939056 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.186058 (+247002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.211364 (+1025306us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001347d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:47.211614 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555716864}
I20260709 12:27:47.213546 13410 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.159983 13442 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:47.189836 13559 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:27:47.165539 13399 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:27:47.191038 13500 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.191823 13398 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:47.192374 13413 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.194512 13305 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.166402 13228 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.198047 13426 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:47.203526 13542 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.205047 13570 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:47.205588 13356 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.205951  8412 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.184052 13489 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.184804 13024 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:47.199259 13346 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.253252  8412 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.176684 13377 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:47.254014  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95699.6us
I20260709 12:27:47.261363  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.262362  7807 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.220396 13572 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.256436 13570 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:47.248521 12796 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:47.269690  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:47.256182  7658 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:47.259473 13489 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:47.239620 13573 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:47.277676  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99892.9us
I20260709 12:27:47.275967  7662 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:47.272243  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.274940  7654 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:47.276877  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.276726  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:47.267424 13572 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:47.282486  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 125530us
I20260709 12:27:47.247457 13228 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.273483 12796 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.263784 13481 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.293377  7737 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:27:47.298243  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.293411  7733 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:47.257205 13356 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.279009 13487 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.250566 13426 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.302060  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:47.279036 13573 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.268211 13024 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.252100 13542 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.300174  7654 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:47.315604  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:47.301190  7778 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Timed out: Call waited in the queue past client deadline [suppressed 13 similar messages]
I20260709 12:27:47.300590  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.317832  7656 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:47.289129  7807 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:27:47.305974  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.311594 13570 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.312625 13377 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.313174 13024 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.299338 12796 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.296187 13572 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.304877 13425 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:47.303824 13412 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:47.326831 13228 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:27:47.329697  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.330785  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.332129  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:47.305111 13573 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:47.340545  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.320724 13578 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.350445  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:47.333776 13399 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:47.359882 13399 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:47.360322 13487 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:47.358045  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.333741 13412 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.370510  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:47.344586 13586 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:47.346611 13425 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:47.361042 13487 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:47.366647 13485 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:47.369907 13412 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:47.395809  8412 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:47.392659  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:47.333518 13580 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:27:47.394227  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.360940 13399 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.396752  7729 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:47.405313  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.397555 13570 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:47.362771 13578 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.447851 13487 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.449326 13578 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.452147 13299 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:47.453009 12650 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:47.432533 13577 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:47.433914  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:47.446971 12796 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.411924 13589 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.458555 13589 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.459055 13589 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.454536 13433 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:47.435773 13581 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:47.440243 13570 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.442900 13425 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.446306 13323 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.457993  8406 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:47.464099  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 162921us
I20260709 12:27:47.442703 13356 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.468816 13570 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.477417 13542 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.404922 13584 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:27:47.478611 13481 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.496146 13426 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.497478 13489 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.474462 13485 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:47.442703 13484 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.482972 13433 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.524751 13346 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.500945 13567 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:47.532843 13547 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.939s	user 0.005s	sys 0.000s
I20260709 12:27:47.532348 13412 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.534053 13567 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.536188 13484 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:47.537683 13570 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.540192 13412 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.766552 13412 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.767709  7728 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:47.767575 13412 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.455029  7735 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:47.659845  7732 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 12:27:47.660419  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 321398us
I20260709 12:27:47.661394  7731 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:27:47.662441  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:47.666144  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.780582  7661 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:27:47.667742 13604 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:47.783389  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:47.783118 13604 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.784674  7657 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:47.685189  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.670984  7737 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:47.716228  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 240185us
I20260709 12:27:47.726385  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:47.733592  7555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 269) took 28141 ms (client timeout 15000 ms). Trace:
W20260709 12:27:47.747454  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.749879  8406 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:27:47.752581 13290 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.544922 13295 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:47.754371 13305 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.805222 13305 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.806309  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:47.806529 13532 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.807368  7653 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.807528 13532 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:47.808262  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 402887us
I20260709 12:27:47.809613  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.811058 13487 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.543532 13346 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.761806 12796 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.548002 13489 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.546654 13517 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:47.543756 13533 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.166s	user 0.008s	sys 0.008s
I20260709 12:27:47.792946 13331 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:47.881927  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.797670  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:47.554596 13485 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.883852 13433 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:47.884425 13517 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.884768 13433 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.799283  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:47.887997  8405 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:47.811159 13135 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.812076 13532 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.889496 13135 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:47.807649  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 183093us
I20260709 12:27:47.891337  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:47.758813 13323 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.558056 13570 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.550776 13572 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.892678 13323 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.540362 13573 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.895097 13572 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.514252 13586 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:47.897945 13586 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.821282  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:47.898911 13573 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.899891  7654 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:47.821439  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:47.903278 13524 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.903682  8414 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:47.904237 13524 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.823565 13305 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.823817  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.824946 13346 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:47.907681 13529 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:47.907943 13346 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.546170 13589 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.787010 13604 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.912492 13604 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.913249 13604 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.913812 13589 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.833482  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.914713 13425 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 12:27:47.916442  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 83205.4us
I20260709 12:27:47.917454  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:47.918511 13309 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:47.834424 13567 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:47.838438  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.919471 13567 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.919363 13309 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.920055 13567 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:47.920521  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82382.2us
I20260709 12:27:47.921916  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:47.922194 13309 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.922742 13346 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.848948  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.923400 13594 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:47.831987 13290 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.923624 13305 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:47.924057 13594 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.925993 13290 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:47.926616 13594 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:47.790977  7555 rpcz_store.cc:269] 0709 12:27:19.592297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.592675 (+   378us) service_pool.cc:224] Handling call
0709 12:27:47.733202 (+28140527us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:36.323490 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:36.324387 (+   897us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:36.324453 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:27:36.324478 (+    25us) tablet.cc:662] Decoding operations
0709 12:27:36.326977 (+  2499us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:36.327094 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:36.327133 (+    39us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:27:36.397245 (+ 70112us) tablet.cc:701] Row locks acquired
0709 12:27:36.397301 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:27:36.397532 (+   231us) write_op.cc:270] Start()
0709 12:27:36.397715 (+   183us) write_op.cc:276] Timestamp: P: 1783600056397436 usec, L: 0
0709 12:27:36.397763 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:27:36.564664 (+166901us) log.cc:844] Serialized 10884 byte log entry
0709 12:27:37.578488 (+1013824us) op_driver.cc:464] REPLICATION: finished
0709 12:27:37.671758 (+ 93270us) write_op.cc:301] APPLY: starting
0709 12:27:37.671950 (+   192us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:38.136957 (+465007us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:38.137129 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:38.421653 (+284524us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:38.422212 (+   559us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 12:27:38.422287 (+    75us) write_op.cc:312] APPLY: finished
0709 12:27:38.423028 (+   741us) log.cc:844] Serialized 244 byte log entry
0709 12:27:38.441252 (+ 18224us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:38.441852 (+   600us) write_op.cc:454] Released schema lock
0709 12:27:38.441942 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:38.442635 (+   693us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:38.442673 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2650332,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":92708,"apply.run_cpu_time_us":57599,"apply.run_wall_time_us":770866,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121167,"num_ops":28,"prepare.queue_time_us":931147,"prepare.run_cpu_time_us":38875,"prepare.run_wall_time_us":640813,"raft.queue_time_us":454056,"raft.run_cpu_time_us":16943,"raft.run_wall_time_us":134454,"replication_time_us":1166320,"thread_start_us":1562420,"threads_started":4,"wal-append.queue_time_us":447015,"wal-append.run_cpu_time_us":29876,"wal-append.run_wall_time_us":2406401}]]}
I20260709 12:27:47.853313  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.927054 13531 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:47.855263 13412 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.855916  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:47.927928 13599 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:27:47.928694  7658 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.860092  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.860543  7804 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:47.861279  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.861563 13481 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:47.861310  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:47.812175 13487 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.862645  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.839057 13428 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:47.932179  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:47.886780  7810 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:27:47.882979  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:47.940501  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49469.3us
I20260709 12:27:47.947471  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.947671 13331 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:47.949553 13487 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:47.943346  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:47.951759 13599 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:47.952611 13599 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:47.953063 13599 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:47.953567  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 142445us
I20260709 12:27:47.954353 13599 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:47.955685 13299 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:47.958139 13489 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.923444 13532 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:47.970384 13481 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:47.966233  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:47.969388 13356 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:47.959336 13572 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:47.985819  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66579.9us
I20260709 12:27:47.974910 13356 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.985169  8409 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:47.992156  7657 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:47.982227 13573 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:48.003762  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:48.008414  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:47.994310 13356 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:47.995133 13604 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.021152  7806 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:48.024147  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:48.014489 13323 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.031913 13456 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.030447  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:48.035707  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:48.065407  7653 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:48.048877  7807 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:48.030398 12868 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:48.023030 13484 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.074856 13572 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:48.074791  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48391.8us
I20260709 12:27:48.078372 13572 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.081894  8412 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:48.082667  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:48.083619  8413 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:48.084719  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:48.086494 13484 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.087412  7803 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:48.088018  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123910us
I20260709 12:27:48.089659 13531 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:48.091485  7811 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:48.093771 13487 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.094561 13487 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.026532 13299 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:27:48.096832 13456 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.098585 13377 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:48.074687 13481 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.099090 13377 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:48.091763 13300 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:27:48.106369 13323 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:48.106750 13487 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:48.106957 13323 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:48.010187 13489 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.107928 13377 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.109391  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:48.109825 13517 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.111693 13524 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.118067  7654 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:48.096966  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:48.123281  8414 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:48.125941  7658 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:48.130966  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:48.131529  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:48.131994 13377 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:48.132071  8413 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:48.133848  8413 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:27:48.135900  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 134723us
I20260709 12:27:48.137703 13305 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:48.139678 13305 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:48.140513 13305 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:48.142378  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 115483us
I20260709 12:27:48.145656 13489 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:48.063140 13573 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:48.155776  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:48.147300  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:48.160917  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:48.167524 13517 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:48.168892 13517 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:48.169348 13517 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:48.158949  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:48.177933  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.124531 13599 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:48.187768  7803 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
W20260709 12:27:48.189071  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51295.9us
I20260709 12:27:48.154616 13572 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.173707  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 901) took 1332 ms. Trace:
I20260709 12:27:48.170547 13517 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.191128 13524 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.120971 12868 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.167526 13305 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:48.214627  7732 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:48.216701  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.219290  7737 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:48.199373  8407 rpcz_store.cc:276] 0709 12:27:46.841408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.841561 (+   153us) service_pool.cc:224] Handling call
0709 12:27:48.173177 (+1331616us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:48.173445 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195021184}
I20260709 12:27:48.198729 13572 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.226806  7731 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:48.196720 13487 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:48.229295  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:48.225426  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.221729  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:48.215453 13524 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.228977  7737 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:48.193586 13295 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:48.215477 13484 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.230957 13572 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:48.229133 13299 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:48.263063  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75898.2us
I20260709 12:27:48.240515 13524 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:48.266925 13517 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:48.267073  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 123816us
I20260709 12:27:48.217636 13356 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:48.268481  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.268649  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2856) took 1505 ms. Trace:
I20260709 12:27:48.244421 13487 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:48.284435  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:48.288712  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72872.9us
I20260709 12:27:48.279655 13529 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:48.265206 13299 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:48.293243  7730 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:48.301636  7809 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:48.244028 13567 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.302989  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.303428  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.275609 13433 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:48.374970  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 933) took 1680 ms (client timeout 1930 ms). Trace:
I20260709 12:27:48.309638  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:27:48.375424  7806 rpcz_store.cc:269] 0709 12:27:46.694369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.694489 (+   120us) service_pool.cc:224] Handling call
0709 12:27:48.374715 (+1680226us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001324d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:48.374885 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799148928}
I20260709 12:27:48.314466 12868 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.316771 13356 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:48.377530  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85445us
I20260709 12:27:48.379575  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:48.323983  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:48.324477 13135 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.326361 13567 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:48.329941  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 903) took 1486 ms. Trace:
I20260709 12:27:48.331157  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:48.332958  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.336970  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:48.337270  7559 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 181) took 54886 ms (client timeout 15000 ms). Trace:
I20260709 12:27:48.337503  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1048) took 1389 ms. Trace:
I20260709 12:27:48.404757  8408 rpcz_store.cc:276] 0709 12:27:46.843246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.845115 (+  1869us) service_pool.cc:224] Handling call
0709 12:27:46.845385 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.979895 (+134510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.329309 (+1349414us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:48.329701 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266329472}
I20260709 12:27:48.307762 13299 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:48.347016  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2869) took 1047 ms. Trace:
I20260709 12:27:48.346673 13487 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.373577 13456 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.376242  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:48.380977 12796 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:48.321234 13529 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:48.420787 13529 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:48.388231 13484 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:48.388293 13331 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.389370  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:48.392436  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:48.393647  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.275153  7734 rpcz_store.cc:276] 0709 12:27:46.692107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.719030 (+ 26923us) service_pool.cc:224] Handling call
0709 12:27:46.720221 (+  1191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.725694 (+  5473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.725723 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:47.677999 (+952276us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b1800425e00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:27:47.679200 (+  1201us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.865974 (+186774us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:47.868461 (+  2487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.875966 (+  7505us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.174118 (+298152us) raft_consensus.cc:1685] finished
0709 12:27:48.174141 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.197889 (+ 23748us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5010,"raft.queue_time_us":5124,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":26344,"spinlock_wait_cycles":51812480,"thread_start_us":166191,"threads_started":1,"wal-append.queue_time_us":236071}
I20260709 12:27:48.338349 13598 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:48.406016  7812 raft_consensus.cc:2376] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13. Re-sending same reply.
I20260709 12:27:48.407570  8409 rpcz_store.cc:276] 0709 12:27:46.947755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.947979 (+   224us) service_pool.cc:224] Handling call
0709 12:27:48.337227 (+1389248us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400780354. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:48.337421 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993985280}
I20260709 12:27:48.410243  7728 rpcz_store.cc:276] 0709 12:27:47.299721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.660076 (+360355us) service_pool.cc:224] Handling call
0709 12:27:47.661298 (+  1222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.768205 (+106907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.346699 (+578494us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540037ee54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:48.346946 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243443968}
W20260709 12:27:48.342770 13605 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.272s	user 0.000s	sys 0.002s
I20260709 12:27:48.430040  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:48.433233  7657 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:48.413163 13433 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:48.444367  7736 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:27:48.450834  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61934.4us
I20260709 12:27:48.453461 12990 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.454052  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.456939  7730 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:27:48.407245  7559 rpcz_store.cc:269] 0709 12:26:53.451077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.465654 (+ 14577us) service_pool.cc:224] Handling call
0709 12:27:48.336987 (+54871333us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:46.815828 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:46.816625 (+   797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:46.816684 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:27:46.816723 (+    39us) tablet.cc:662] Decoding operations
0709 12:27:46.817677 (+   954us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:46.817870 (+   193us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:46.817908 (+    38us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:27:46.818596 (+   688us) tablet.cc:701] Row locks acquired
0709 12:27:46.818676 (+    80us) write_op.cc:260] PREPARE: finished
0709 12:27:46.818911 (+   235us) write_op.cc:270] Start()
0709 12:27:46.819338 (+   427us) write_op.cc:276] Timestamp: P: 1783600066818810 usec, L: 0
0709 12:27:46.819362 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:46.820139 (+   777us) log.cc:844] Serialized 1736 byte log entry
0709 12:27:47.207682 (+387543us) op_driver.cc:464] REPLICATION: finished
0709 12:27:47.323203 (+115521us) write_op.cc:301] APPLY: starting
0709 12:27:47.325563 (+  2360us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:47.362032 (+ 36469us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:47.384287 (+ 22255us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:47.490127 (+105840us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:47.490385 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:27:47.490432 (+    47us) write_op.cc:312] APPLY: finished
0709 12:27:47.490723 (+   291us) log.cc:844] Serialized 49 byte log entry
0709 12:27:47.494377 (+  3654us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:47.494509 (+   132us) write_op.cc:454] Released schema lock
0709 12:27:47.494559 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:47.495067 (+   508us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:47.495100 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8374703,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":100680,"apply.run_cpu_time_us":9645,"apply.run_wall_time_us":177055,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20847,"num_ops":4,"prepare.queue_time_us":106599,"prepare.run_cpu_time_us":50969,"prepare.run_wall_time_us":170680,"raft.queue_time_us":138116,"raft.run_cpu_time_us":12548,"raft.run_wall_time_us":73947,"replication_time_us":388093,"thread_start_us":329823,"threads_started":4,"wal-append.queue_time_us":180795}]]}
I20260709 12:27:48.405439 13573 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.454233 12990 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.461624 12990 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:48.416090 13519 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.467931 13519 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:48.470335  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.472847  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:48.474509 13580 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:48.475711 13580 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:48.477016 13580 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.480247  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.481285  7728 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:48.482627  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:48.485591 13610 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:27:48.487468  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 107992us
I20260709 12:27:48.487924  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.488643  7734 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:48.489750  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:48.489993  7730 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:48.490362  7734 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:48.523006  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:48.515843  7728 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:48.528313  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 216951us
I20260709 12:27:48.414314 13323 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.519237 13489 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:48.564411  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 75596.8us
I20260709 12:27:48.572813  7730 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:48.580705  8408 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:48.584295  7730 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:48.590181  7730 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
W20260709 12:27:48.591856  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.594770  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.592861  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:48.519129 13598 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.598943 13504 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:48.675258 13377 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.543s	user 0.016s	sys 0.000s
I20260709 12:27:48.615520  7803 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:48.620042 13485 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:27:48.522607 13529 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.574596 13489 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:48.664922  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 498124us
I20260709 12:27:48.682898  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:48.611776  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52761.3us
W20260709 12:27:48.703354  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 134719us
I20260709 12:27:48.695894  7730 pending_rounds.cc:85] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 13
W20260709 12:27:48.667344 13531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.008s
W20260709 12:27:48.718183  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55826.9us
I20260709 12:27:48.722836 13584 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:48.720877 13504 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.723596 13584 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.725926 13504 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.710086  7730 pending_rounds.cc:107] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 14.14
W20260709 12:27:48.728211  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.715608 13531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.006s	sys 0.008s
I20260709 12:27:48.731329  7730 raft_consensus.cc:2889] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:48.708384 13519 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.735320 13290 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:48.753393  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:48.726536 13584 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.754052  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2868) took 1326 ms. Trace:
I20260709 12:27:48.743391 13481 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:48.767028  7728 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 11.
I20260709 12:27:48.760375 13290 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.746384 13309 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 12:27:48.784472  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.768281 13481 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:48.787662  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48348.9us
I20260709 12:27:48.781152 13290 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:48.758630  7733 rpcz_store.cc:276] 0709 12:27:47.297939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.463375 (+165436us) service_pool.cc:224] Handling call
0709 12:27:47.463761 (+   386us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:47.464240 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.464273 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:47.663279 (+199006us) log.cc:844] Serialized 78 byte log entry
0709 12:27:47.877313 (+214034us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:48.062350 (+185037us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.115378 (+ 53028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.153987 (+ 38609us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.341615 (+187628us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:48.617008 (+275393us) raft_consensus.cc:1685] finished
0709 12:27:48.623523 (+  6515us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.623805 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5874,"prepare.queue_time_us":306024,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":10145,"raft.queue_time_us":57021,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":24340,"spinlock_wait_cycles":18331904,"thread_start_us":294990,"threads_started":2,"wal-append.queue_time_us":319971}
I20260709 12:27:48.758137 13356 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 12:27:48.808408  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 102272us
I20260709 12:27:48.791949  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2522) took 1638 ms. Trace:
W20260709 12:27:48.808713  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68493.4us
I20260709 12:27:48.815933  7810 rpcz_store.cc:276] 0709 12:27:47.144891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.145067 (+   176us) service_pool.cc:224] Handling call
0709 12:27:47.168043 (+ 22976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.941895 (+773852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.770278 (+828383us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:48.782793 (+ 12515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651005312}
I20260709 12:27:48.774003 13567 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:48.792452 13517 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.002s
I20260709 12:27:48.825315 13492 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:48.827289  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76930.1us
I20260709 12:27:48.827896  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:48.793394 13481 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:48.787912 13456 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:48.844419  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:48.846967 13456 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.847522 13456 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:48.847939 13456 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:48.849937 13572 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.002s
I20260709 12:27:48.850492 13572 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.851528 13356 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:48.853170 13567 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.834779 13492 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:48.853199  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:48.867477 13305 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.860296 13489 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.860803 13598 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.812022 12990 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.863497 13584 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:48.864018 13456 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.937711 13572 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:48.866266 13299 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:48.938014 13456 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:48.842857 13599 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.004s
W20260709 12:27:48.868058  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.046s	sys 0.040s
W20260709 12:27:48.939488  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.046s	sys 0.040s
I20260709 12:27:48.842262  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:48.940184 13617 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:27:48.940706  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 117913us
I20260709 12:27:48.842360 13425 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:48.858675 13524 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.005s
W20260709 12:27:48.945492 13618 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
W20260709 12:27:48.881409  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.922696 13489 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.947317 13489 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.950413 13489 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:48.913537  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 131205us
I20260709 12:27:48.951465 13517 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:48.951961 13517 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.952354 13517 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:48.927464 13356 consensus_queue.cc:1243] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:48.954965  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:48.955411 13356 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:48.956022  7656 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:48.956234 13356 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.959167 13309 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:48.959985  7662 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:48.961004  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:48.931366  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 272659us
I20260709 12:27:48.923555 13492 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.935175 12990 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:48.931414 13305 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:48.944816 13524 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:48.913388  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60236.4us
I20260709 12:27:48.934123 13529 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:48.959282  7658 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:27:48.965065  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.959941 13309 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:48.967819  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:48.964702 13584 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:48.940218 13299 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.003783  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.005133  7658 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:48.982960 13331 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:48.983124 13489 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.967056 13290 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.025264  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:49.030795 13599 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:49.001085 13584 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:49.020215 13331 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.078311  7656 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 12:27:49.074821 13531 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:49.082115 13531 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:49.067575 13456 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:27:49.083485 13531 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.076305  7653 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:49.080624 13586 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:49.086194 13584 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.080998  7658 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:49.071651  7662 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:49.011166  7554 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 4 to 8, term changed from 8 to 11, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:49.071445  7659 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:49.076939  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 113176us
I20260709 12:27:49.094228  7655 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:49.101018  7661 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:49.045804 13599 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.086798 13586 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.053197 13492 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:49.080030 13625 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:49.081228 13305 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.085009 13531 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:49.081239 13299 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.077157 13290 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:49.092945 13331 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:49.076398 13524 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.084919 13567 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.146649  7654 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:49.120754 13599 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:49.053486 13519 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:27:49.032893 13489 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.156497  7660 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:49.155957  7554 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 11 to 13, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 13 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:49.157248 13299 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.158855  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.160230 13481 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.165041 13598 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:49.165663 13323 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:49.171586  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:49.173718 13635 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.173820 13504 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:49.174741  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.214151  8405 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:49.175326 13305 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:49.175977  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.177168  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.177644  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.216568  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:49.178670 13632 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.180588 12990 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:49.150671  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:49.212061  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 230923us
I20260709 12:27:49.211498  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.217288  8412 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.229514  7660 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:49.239392  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.195832 13567 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.216585 13504 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:49.145778 13309 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.253638  8408 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:49.254736  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2887) took 1009 ms. Trace:
I20260709 12:27:49.238947 13428 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:49.256074  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.258755  7729 rpcz_store.cc:276] 0709 12:27:48.245685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.261175 (+ 15490us) service_pool.cc:224] Handling call
0709 12:27:48.281938 (+ 20763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.312753 (+ 30815us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:48.440125 (+127372us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.101664 (+661539us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:49.137837 (+ 36173us) log.cc:844] Serialized 26 byte log entry
0709 12:27:49.138111 (+   274us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:49.138173 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.138306 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.254500 (+116194us) raft_consensus.cc:1685] finished
0709 12:27:49.254523 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.254670 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93209,"prepare.queue_time_us":183930,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":67980,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":225,"thread_start_us":640281,"threads_started":2,"wal-append.queue_time_us":614364}
W20260709 12:27:49.275419  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.274462  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:49.276716  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.277022  7660 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:49.264103 13428 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:49.252343 13636 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:49.277824  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 230816us
I20260709 12:27:49.242436 12990 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.246165 13635 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.321923  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.273543 13567 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.329187  7661 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:49.333637  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 207264us
I20260709 12:27:49.257448 13632 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.318181 13636 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:49.356834  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.336562 13305 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:49.349370 13135 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:49.369531  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2886) took 1130 ms. Trace:
I20260709 12:27:49.339949 13517 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:49.377429  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 198189us
I20260709 12:27:49.367718  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2884) took 1146 ms. Trace:
W20260709 12:27:49.383793  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 152536us
I20260709 12:27:49.384434  7661 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:49.383926  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62819.1us
I20260709 12:27:49.373709  7737 rpcz_store.cc:276] 0709 12:27:48.239096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.260282 (+ 21186us) service_pool.cc:224] Handling call
0709 12:27:48.280579 (+ 20297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.309561 (+ 28982us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:48.510385 (+200824us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.218925 (+708540us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.369197 (+150272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:49.369249 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:49.369298 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.369362 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.369471 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46566,"prepare.queue_time_us":158160,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":94,"raft.queue_time_us":190396,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":15218,"thread_start_us":784430,"threads_started":3,"wal-append.queue_time_us":507942}
I20260709 12:27:49.366508 13305 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:49.385515  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:49.367614 13604 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:49.368836 13647 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.368919 13135 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:49.399067  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:49.408193 13135 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.408943 13604 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.409824 13135 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:49.385826 13481 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.368714 13428 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.411077 13524 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:49.411782 13524 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:49.412533 13425 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:49.412850 13524 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.414350 13484 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.411149 13428 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:49.416271 13645 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:27:49.416853  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.366150 13641 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:27:49.417462 13647 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.418121 13641 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:49.418797 13641 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:49.420297  7661 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:27:49.404393  7554 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 7 to 12, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:49.382755  7732 rpcz_store.cc:276] 0709 12:27:48.221108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.228163 (+  7055us) service_pool.cc:224] Handling call
0709 12:27:48.228732 (+   569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.243722 (+ 14990us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:48.738529 (+494807us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.367321 (+628792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:49.367377 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:49.367433 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.367522 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.367656 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10035,"prepare.queue_time_us":33808,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":76,"raft.queue_time_us":124885,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":12053,"spinlock_wait_cycles":71296,"thread_start_us":1052344,"threads_started":3,"wal-append.queue_time_us":580590}
I20260709 12:27:49.356987 13504 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.392735 13489 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.436896  8412 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:49.450065  7554 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 6 to 9, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 9 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.453029 13647 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:49.452538  7554 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.454609 13632 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.429129 13481 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.455695 13481 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.458176 13481 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.414984 13484 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:49.485498  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 100055us
I20260709 12:27:49.493245  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:49.485963  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 99823.7us
I20260709 12:27:49.488080  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:49.385042 13643 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.195s
I20260709 12:27:49.488572  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:49.494190  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96580.8us
I20260709 12:27:49.394513 13584 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 12:27:49.497570  7661 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:49.500660  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:49.500059  8406 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:49.454443  7554 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 5 to 9, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 9 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:49.502810  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.420297 13641 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:49.504870 13649 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:27:49.505275  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:49.505625 13489 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:49.500064 13636 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.504884  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.508741  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.514446 13604 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.518414 13356 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.522879 13309 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:49.525483 13331 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:49.527275 13644 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:27:49.532959 13517 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:49.535409 13492 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:49.592530 13640 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.248s	user 0.001s	sys 0.001s
I20260709 12:27:49.484583 13658 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:49.567343  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.593282 13658 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:49.569168 13641 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:49.593823 13658 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:49.596123 13653 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:49.596397 13492 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:49.595801  7554 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.596963 13492 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:49.596835 13653 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.597959 13492 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.599849 13492 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.603964  8412 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:27:49.556763 13305 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.607195 13331 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:49.607314 13305 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:49.608878  7554 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 5 to 9, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 9 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.575870 12990 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.570734  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.571602  7657 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:49.571769  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.621577  7654 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:49.572304  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.572793  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.573495  8406 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:27:49.574005  7656 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:27:49.623662  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:49.623999  7656 pending_rounds.cc:85] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 9
I20260709 12:27:49.624294  7656 pending_rounds.cc:107] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:27:49.624579  7656 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.574323  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:49.576612  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83492.4us
W20260709 12:27:49.577611  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89617.1us
I20260709 12:27:49.578369  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:49.626554  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82066.5us
I20260709 12:27:49.501905 13648 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.584954 13647 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:49.591800  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.613404  7662 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:49.623147  7655 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:49.639243  7660 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:49.641629  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:27:49.718516  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76990.4us
W20260709 12:27:49.719991  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.720093  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.715858  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.720748  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.725128  7655 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:49.725296  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.725874  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.726342  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.727005 13584 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:49.725010  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.727977  7662 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:49.728351  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.559408 13481 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 12:27:49.611174 13635 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.725049  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.744061  7654 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:27:49.731597  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.732152  8408 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:49.735785  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:49.736796  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 94234.3us
I20260709 12:27:49.745590  7656 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.739059  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.746685  7660 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:49.748482  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:49.578040 13489 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:49.617100 13586 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:49.752942  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.755151  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.612128 13567 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:49.781522 13638 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.399s	user 0.004s	sys 0.001s
I20260709 12:27:49.770486  7659 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:27:49.881728  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 133499us
I20260709 12:27:49.882608  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:49.613633 13580 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:49.795099  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.775911  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.770082  7661 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:49.783463  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.887478  7655 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:49.812592  7656 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:49.774294 13661 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.891067  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.774662 13632 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:49.779474 13654 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:49.769102  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:49.900548  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 131577us
I20260709 12:27:49.738041 13425 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.723587 13659 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:49.757444 13599 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.578s	user 0.014s	sys 0.000s
I20260709 12:27:49.812269  7662 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:49.734083 13647 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:49.727670 13584 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:49.737874 12990 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:49.729974 13653 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.840392 13484 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:49.754181 13135 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 12:27:49.844030 13356 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.847728  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:49.859409 13323 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.005s
I20260709 12:27:49.862282  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:49.865586 13656 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.866662  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:49.867724  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:49.868628  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:49.869470  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:49.830791 13643 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:49.879590  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:49.948906  7731 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:49.885576  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:27:49.885962  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.894979  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.947777  7729 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.897722 13661 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:49.898738 13641 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:49.663643 13648 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.913482  8413 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:49.891670 13604 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:49.928714  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:49.928819  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 333334us
W20260709 12:27:49.935442  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52918.6us
I20260709 12:27:49.619259  7554 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.939659  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:49.915620 13492 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:49.946966  7732 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:49.959131  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.976387  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:49.992825 13661 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:49.993124 13668 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 12:27:49.994752  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.997572  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:49.998592  7662 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:49.918757 13425 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.000136  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.001338 13425 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:50.001698  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:49.943696 13567 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:49.923820 13659 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.927582 13599 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.895383 13580 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.029443  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:50.039897  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.008782  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.036053  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:50.062772  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72043.7us
I20260709 12:27:50.022223  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.032006  8413 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 12:27:50.033396  8405 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:27:50.029611  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:50.072294  8405 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 12
I20260709 12:27:50.072573  8405 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:27:50.072816  8405 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:50.036731  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:50.022786  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.073786  7728 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:50.017222  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:50.026063  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:50.075194  7732 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:50.076500  8409 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:27:50.019501  7806 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:50.079375  7729 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:50.082209  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:50.053582  7812 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:50.074209  7809 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:50.018646  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:49.955020 13529 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.009250  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:50.103075  7735 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:50.103652  7803 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:27:50.082729  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 110353us
I20260709 12:27:50.032546  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:50.078814  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:50.002913 13299 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:50.003651  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:50.003122  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:49.968936 13643 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:50.109889  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.110311  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.007351 13504 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:50.127527  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.128433  7806 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:50.127545  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:49.976603 13641 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:50.135780  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.125522  7731 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:50.003513 13642 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.988009 13492 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.049015 13135 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.146028  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.101636 13481 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.015630 12990 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:50.006917 13648 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.156553 13299 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:50.157051 12990 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:50.157577 13599 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.135664  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.157395 13299 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.158679  7803 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:50.093444 13567 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:49.964114 13356 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.115108  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.160372  7811 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:50.160984  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:50.161942 13641 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.166396  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.167362  7662 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:50.167886  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.168708  7810 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:49.960831 13656 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:50.143668  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 64986us
I20260709 12:27:50.130982  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.176097  7660 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
W20260709 12:27:50.116575  8379 connection.cc:256] RPC call timeout handler was delayed by -1.1961845809996703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
I20260709 12:27:50.155282  7728 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:27:49.989583 13604 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.156132 13290 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:50.177277  8379 connection.cc:256] RPC call timeout handler was delayed by -1.0863845809994928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
I20260709 12:27:50.155956 13481 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.146862  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:50.013518 13658 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:50.178877  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.182435  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.139761  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.129246 13650 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.184767  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:50.184677  7731 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:50.146181 13492 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:50.188907  7662 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:50.189209  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:50.146286 13331 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.149590 13487 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.204092  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.207003  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.010113 13584 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.003836 13489 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.128782 13529 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.181864 13290 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.216560  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:50.003687 13647 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.216945  7730 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:50.214735  7662 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:50.213377  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.231489  7810 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:50.231755  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.230922 13331 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.238963  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:50.239719 13331 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.239929 13481 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.240803 13481 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.241464 13481 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.204001 13654 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:50.244848  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55745.5us
I20260709 12:27:50.263350  7806 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:50.263685  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:50.262424  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 307294us
I20260709 12:27:50.273667  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.286652  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:50.182078 13135 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.274788  7812 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:27:50.217427 13664 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:50.290518  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.226395 13290 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:50.194165 13519 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:27:50.309870  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:50.294180  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 66960.7us
I20260709 12:27:50.290865  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:50.310951  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47148.3us
I20260709 12:27:50.298270  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:50.151324 13642 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.313884  7807 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:50.263326 13654 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.004954  7554 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 7 to 11, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 11 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:27:50.322829  7658 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:50.301273  7805 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
W20260709 12:27:50.318303  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.319816  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.213965 13487 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.309244 13331 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:50.331542  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:50.333400  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:50.347182  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:50.352916  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.212373 13650 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.389842  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.397825  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.379416  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:50.390997  7812 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:50.398806  7803 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:50.402786  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:50.404188  7653 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:50.408303  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:50.325627 13654 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:50.331132 13331 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.427100  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.376688 13428 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:50.376212 13323 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:50.330119 13356 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.409539  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:50.398896 13661 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:50.431209 13632 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.401957 13425 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:27:50.432989 13331 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:50.518661 13680 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:50.518249 13679 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:27:50.401546 13567 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.261897 13643 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:50.617202 13675 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.293s	user 0.001s	sys 0.002s
W20260709 12:27:50.484963 13305 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.877s	user 0.013s	sys 0.003s
I20260709 12:27:50.570823  7807 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:50.582563  7732 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:50.413146 13309 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:50.676810  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 355278us
W20260709 12:27:50.699519  7654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.038s	sys 0.079s
W20260709 12:27:50.702194  7654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.038s	sys 0.079s
I20260709 12:27:50.626256 13323 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:50.663681 13599 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.674500 13659 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.622879 13632 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:50.789763  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.014s	sys 0.019s
I20260709 12:27:50.747852 13481 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:50.747424 13580 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:50.800163 13683 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:27:50.802141 12650 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:50.802814 12650 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.771180 13658 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:50.660619 13680 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.758272 13309 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:50.814908  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.014s	sys 0.020s
I20260709 12:27:50.797271 13659 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.805806 13650 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:50.781102 13599 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.805804 12650 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:50.824484 13135 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:50.856890  7737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.016s	sys 0.017s
W20260709 12:27:50.859557  7737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.017s	sys 0.018s
I20260709 12:27:50.856654  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:50.825935 13604 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:50.847469 13647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.002s	sys 0.031s
I20260709 12:27:50.874393  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.845022 13487 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:50.843027 13693 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.874393  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.840934 13642 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:50.880127  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49637us
I20260709 12:27:50.905139  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:50.884811  7807 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
W20260709 12:27:50.875614 13657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.016s	sys 0.026s
W20260709 12:27:50.876179 13647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.003s	sys 0.031s
I20260709 12:27:50.848811 13659 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.910192  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:50.915552  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.916270  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.916945  7806 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:50.918460  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:50.920320  7806 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
W20260709 12:27:50.907428 13686 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 12:27:50.883958 13567 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:50.916426  7807 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:27:50.908809 13657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.016s	sys 0.027s
I20260709 12:27:50.885815 13696 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:27:50.935367 13695 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:27:50.936961 13689 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:27:50.940697  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 188288us
I20260709 12:27:50.943133  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:50.913470 13135 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:50.863719 13323 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.928203 13487 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:50.928823 13567 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:50.964299 13696 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:50.909317 13693 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:50.963290 13135 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:50.910055 13309 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:50.966737  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:50.971405 13696 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:51.035813 13696 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:51.037819 13290 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.002s
I20260709 12:27:50.967556 12650 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:50.975320 13135 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:51.040107 13705 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:27:51.039851 12650 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.042299  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1184) took 1089 ms. Trace:
I20260709 12:27:50.973047 13704 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.042788  7729 rpcz_store.cc:276] 0709 12:27:49.953147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.168030 (+214883us) service_pool.cc:224] Handling call
0709 12:27:50.168363 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.382238 (+213875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.382471 (+   233us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.383415 (+   944us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.710755 (+327340us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:50.710821 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.710923 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.042016 (+331093us) raft_consensus.cc:1685] finished
0709 12:27:51.042045 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.042224 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":487320,"threads_started":2,"wal-append.queue_time_us":442821}
I20260709 12:27:50.977722 13693 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:50.974988 13504 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.832s	user 0.017s	sys 0.001s
I20260709 12:27:51.043849 13704 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.046181 13586 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:50.969900 13567 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:51.047515  7808 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:51.048153 13704 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.038431 13290 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:51.061765 13682 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:27:51.065759  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.067703  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2917) took 1146 ms. Trace:
I20260709 12:27:51.068202  7735 rpcz_store.cc:276] 0709 12:27:49.921329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.165294 (+243965us) service_pool.cc:224] Handling call
0709 12:27:50.165711 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.303996 (+138285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.304029 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.339384 (+ 35355us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.778405 (+439021us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:50.778456 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.778543 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.067464 (+288921us) raft_consensus.cc:1685] finished
0709 12:27:51.067485 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.067639 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":488341,"threads_started":2,"wal-append.queue_time_us":559690}
I20260709 12:27:51.071794  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1059) took 1039 ms. Trace:
W20260709 12:27:51.072798 13674 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.781s	user 0.007s	sys 0.003s
I20260709 12:27:51.073374 13659 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:51.073951  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.043972 13599 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:51.071694 13632 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.072399  7654 rpcz_store.cc:276] 0709 12:27:50.032389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.032654 (+   265us) service_pool.cc:224] Handling call
0709 12:27:50.033001 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:50.033411 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.033462 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:50.034398 (+   936us) log.cc:844] Serialized 52 byte log entry
0709 12:27:50.747066 (+712668us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:50.747117 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.747199 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.063187 (+315988us) raft_consensus.cc:1685] finished
0709 12:27:51.071527 (+  8340us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.071721 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2605,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":108,"thread_start_us":667696,"threads_started":1,"wal-append.queue_time_us":881437}
W20260709 12:27:51.080720  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.042717 12650 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:51.051004 13492 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.842s	user 0.008s	sys 0.006s
I20260709 12:27:51.085532  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.043758 13299 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.098294  7807 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:51.049547 13650 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.103610  7803 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:51.104955  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.114068  7662 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:51.118032  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.120186  7654 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:27:51.085439 13573 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.004s	sys 0.013s
I20260709 12:27:51.130889  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.083269 13428 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:51.095063 13693 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.129112  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.133375  7658 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:27:51.131639 13573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.005s	sys 0.013s
I20260709 12:27:51.051620 13487 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:27:51.141789  7807 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:51.130829 13299 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:51.135668 12990 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.978s	user 0.015s	sys 0.000s
I20260709 12:27:51.131297 13680 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.157506  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.115962 13492 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.167099  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.131556 13650 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.183012  8377 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.007s	sys 0.010s
W20260709 12:27:51.193028  8377 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.007s	sys 0.010s
I20260709 12:27:51.184661  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.153627 13299 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:51.197131  7808 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:51.204762  8407 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:51.147673 13693 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.211311  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 226010us
I20260709 12:27:51.218787  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.218950 12650 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:27:51.219754  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.009s	sys 0.013s
I20260709 12:27:51.219558  8415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31230d31601445449fab8f0f9943db57"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:27:51.220614  8415 raft_consensus.cc:606] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:51.220187  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.009s	sys 0.013s
I20260709 12:27:51.221040  8415 raft_consensus.cc:614] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:51.222267  7804 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:51.222575  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:51.217258  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:51.223315  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.220964 13693 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.224390  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2745) took 1038 ms. Trace:
I20260709 12:27:51.224442  7662 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:51.223598  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.225409 12650 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:51.225028  7660 rpcz_store.cc:276] 0709 12:27:50.185744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.186122 (+   378us) service_pool.cc:224] Handling call
0709 12:27:50.203563 (+ 17441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.576420 (+372857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.576471 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.657699 (+ 81228us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.121055 (+463356us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:51.121099 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.121239 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.224157 (+102918us) raft_consensus.cc:1685] finished
0709 12:27:51.224181 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.224328 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":789,"raft.queue_time_us":196779,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":10588,"thread_start_us":693369,"threads_started":2,"wal-append.queue_time_us":464239}
I20260709 12:27:51.224992  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:51.233690 13697 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:51.235107 13290 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.235112 13697 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:51.237804 13697 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.240424 13697 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.162737 12990 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.206658 13299 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:51.211901 13331 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.010s
I20260709 12:27:51.244638  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.250517  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.255376  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:51.257798  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.257798  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.259588  7806 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:51.263399  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.263911  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.264632  7806 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:51.264920  8408 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:51.247236 12650 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:51.267887  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.268913  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.269354  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.270624  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.272212  7812 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:51.268774  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:51.273563  7804 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:51.253818 13714 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:51.276355  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.286370 13714 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.269357 12650 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.287094 13714 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:51.287462 12650 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.288403 13529 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.225196 13428 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.280936  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.267316 13693 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.283634 13719 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:51.283128 13135 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.285391 12990 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:27:51.256157 13331 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.313133  8412 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:51.298739 13632 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:51.313922 13632 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.316346 13632 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.317544 13650 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:51.322504 13492 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:51.298486 13719 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:51.305670 13654 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.000s
I20260709 12:27:51.301676 13693 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.331862 13685 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:51.332590 13685 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.333595  7812 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:51.334350 13693 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.334857 13685 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.335723 13661 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:51.336297 13661 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:51.337307 13661 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.337796  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2908) took 1474 ms. Trace:
I20260709 12:27:51.338703 13661 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:51.338891  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 925901us
I20260709 12:27:51.339308 13598 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:51.339350 13719 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:51.341866 13716 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:27:51.343641  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.344137  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.344614  7812 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:51.346520  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.346653  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.348951  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.349929  7732 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:51.352403  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.353387  7731 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:27:51.369407  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.382246  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.381876  7811 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
W20260709 12:27:51.380066  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.383175  8411 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:51.380375  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.338233  7736 rpcz_store.cc:276] 0709 12:27:49.877682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.880524 (+  2842us) service_pool.cc:224] Handling call
0709 12:27:49.884423 (+  3899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.884660 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:50.099171 (+214511us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.976800 (+877629us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:51.337429 (+360629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.337492 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:51.337541 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.337607 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.337734 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":928803,"prepare.queue_time_us":304291,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":910816,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"spinlock_wait_cycles":213583488,"thread_start_us":539563,"threads_started":2,"wal-append.queue_time_us":688852}
I20260709 12:27:51.394102  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.379145 13598 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:27:51.396440  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2909) took 1518 ms. Trace:
I20260709 12:27:51.395289  7728 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:51.393397  7807 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:51.393546  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:51.404812  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.401788  8412 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:51.404201  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.404237  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.399726  7733 rpcz_store.cc:276] 0709 12:27:49.878038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.878172 (+   134us) service_pool.cc:224] Handling call
0709 12:27:49.893032 (+ 14860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.893492 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.893535 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:49.914563 (+ 21028us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.208183 (+293620us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:50.288625 (+ 80442us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.324154 (+1035529us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b1800452ec0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:27:51.386729 (+ 62575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.387376 (+   647us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.387424 (+    48us) raft_consensus.cc:1685] finished
0709 12:27:51.396229 (+  8805us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.396361 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103070,"prepare.queue_time_us":81417,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":80,"raft.queue_time_us":1049582,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":194,"spinlock_wait_cycles":45340032,"thread_start_us":309206,"threads_started":2,"wal-append.queue_time_us":298637}
I20260709 12:27:51.415938  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.363612 13425 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:51.390836 13656 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.399313 13428 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:51.384313 13696 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.408291 13305 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:27:51.431365  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2916) took 1510 ms. Trace:
I20260709 12:27:51.433094 13567 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.414909 13658 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.359081 13654 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.438681  8412 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:27:51.442229  7732 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:27:51.442922  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.436170  7737 rpcz_store.cc:276] 0709 12:27:49.921024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.162324 (+241300us) service_pool.cc:224] Handling call
0709 12:27:50.162738 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.162978 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:50.860144 (+697166us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.095600 (+235456us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.099199 (+  3599us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.430795 (+331596us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.430981 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.431025 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:51.431077 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.431148 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.431301 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":383642,"prepare.queue_time_us":108037,"prepare.run_cpu_time_us":186,"prepare.run_wall_time_us":193,"raft.queue_time_us":333535,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":920654,"threads_started":2,"wal-append.queue_time_us":375038}
I20260709 12:27:51.448284  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2910) took 1569 ms. Trace:
I20260709 12:27:51.448686  7730 rpcz_store.cc:276] 0709 12:27:49.878554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.878715 (+   161us) service_pool.cc:224] Handling call
0709 12:27:49.900657 (+ 21942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.243355 (+342698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.448079 (+1204724us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54002490d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:51.448232 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721398016}
I20260709 12:27:51.429154 13693 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:51.428650 13428 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.439941 12650 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.456825 13567 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.457497 12650 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.425520 13692 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 12:27:51.457490  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2912) took 1567 ms. Trace:
I20260709 12:27:51.460922  7734 rpcz_store.cc:276] 0709 12:27:49.889988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.890169 (+   181us) service_pool.cc:224] Handling call
0709 12:27:49.916278 (+ 26109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.916520 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:50.098744 (+182224us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.445985 (+1347241us) spinlock_profiling.cc:243] Waited 875 us on lock 0x7b1800344b60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:27:51.456935 (+ 10950us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.457134 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.457183 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:51.457233 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.457299 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.457430 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80796,"prepare.queue_time_us":287024,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":95,"raft.queue_time_us":12653,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":148,"spinlock_wait_cycles":2013440,"thread_start_us":364244,"threads_started":2,"wal-append.queue_time_us":523516}
W20260709 12:27:51.462451 13694 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.661s	user 0.006s	sys 0.001s
I20260709 12:27:51.441172 13331 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:51.467633  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2536) took 1342 ms. Trace:
I20260709 12:27:51.468096  7809 rpcz_store.cc:276] 0709 12:27:50.125721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.136728 (+ 11007us) service_pool.cc:224] Handling call
0709 12:27:50.169213 (+ 32485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.210631 (+ 41418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.210699 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.233083 (+ 22384us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.475641 (+242558us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:50.475705 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.475825 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.461483 (+985658us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:51.467322 (+  5839us) raft_consensus.cc:1685] finished
0709 12:27:51.467345 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.467567 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11830,"raft.queue_time_us":17321,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":15878,"spinlock_wait_cycles":17465216,"thread_start_us":174494,"threads_started":1,"wal-append.queue_time_us":537771}
I20260709 12:27:51.452446 13685 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.471071 13685 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.472143 13691 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.623s	user 0.005s	sys 0.001s
I20260709 12:27:51.476569  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.477686  7732 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:51.487880  7730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:51.488647  7730 raft_consensus.cc:606] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:27:51.489070  7730 raft_consensus.cc:614] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:51.450328 13693 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:51.496508  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.497599  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.422719 13690 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:27:51.498322  7737 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:51.496497  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.463997 13428 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:51.503641  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.514345  7735 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:51.504958  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.506824  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:51.518208  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.519243 13299 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.520257 13727 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:27:51.521275 13733 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:27:51.446412 13658 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.523685  7729 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:51.525079  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64329.1us
I20260709 12:27:51.537272  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3070) took 1223 ms. Trace:
I20260709 12:27:51.497861 13693 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:51.537752  7810 rpcz_store.cc:276] 0709 12:27:50.313661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.314579 (+   918us) service_pool.cc:224] Handling call
0709 12:27:51.537061 (+1222482us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b54003c9e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:51.537202 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2231538944}
I20260709 12:27:51.541800  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:51.551813  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.553067  7732 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:51.554123 13734 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:27:51.554731 13729 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:51.556842 13299 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.558028  7653 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:51.561830  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3067) took 1299 ms. Trace:
I20260709 12:27:51.562515  7811 rpcz_store.cc:276] 0709 12:27:50.262891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.331044 (+ 68153us) service_pool.cc:224] Handling call
0709 12:27:51.561599 (+1230555us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400011654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:51.561769 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166037248}
I20260709 12:27:51.565204 13715 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:51.565562 13658 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.566838 13654 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.552876  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.569360  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:51.592684  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.533468 13642 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:51.531459 13135 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.574290  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:51.596268  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.596462  7732 raft_consensus.cc:606] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:51.596922  7733 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:51.581434  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:51.586895  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.599800  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:51.601073  8413 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20.
I20260709 12:27:51.602113  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.603721  8406 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 12:27:51.604176  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.605314  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.588119  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.590022  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.593487  7729 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:27:51.552758 13722 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:51.595366  7734 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:51.609613  8409 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:51.539274 13685 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.612108  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.612810  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.615983  7728 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:51.612884  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.619122  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.619895  7731 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:51.619805  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.582594 13715 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:51.629074  8411 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:51.611420  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:51.633016  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:51.595781 13642 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:51.639068  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.640215  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.645994  7728 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:51.647533  7736 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:51.650848  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.652664  7728 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:51.647290  7734 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:51.612005 13323 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:51.595206 13697 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:27:51.659307  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.631870 13492 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:51.658888  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:51.676783  7731 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:51.677829  7700 leader_election.cc:400] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20.
I20260709 12:27:51.681766  7660 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:51.657418 13484 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:51.686482  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.687767  7732 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:51.680119  7728 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:51.649886 13664 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:51.703235  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47301.3us
I20260709 12:27:51.686192 13654 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:51.710366 13654 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.711760  7728 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:51.712960  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.713840  7728 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:51.715894  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:51.716572 13719 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.719872 13712 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:51.720646 13712 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:51.721123 13712 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.715670 13309 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:51.644696 13517 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:27:51.723603 13309 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:51.724013 13309 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:51.680917 13632 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:51.724962  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.698800 13697 consensus_queue.cc:1243] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:51.686858 13323 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:51.724953 13632 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.727919 13642 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.727684  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.706203 13664 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.729592  8414 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:51.729820 13664 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:51.730858 13725 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.337s	user 0.001s	sys 0.001s
I20260709 12:27:51.732900  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.733151  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.734313  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.736126 13598 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.737064 13598 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.737872  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:51.734962  7736 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:51.742875  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:51.734318  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.746709  7730 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:51.752420  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:51.753322  7732 raft_consensus.cc:606] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:51.753829  7732 raft_consensus.cc:614] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:51.679543 13492 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:51.718930 13719 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.762452  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.765707  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.666652 13685 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:27:51.757254  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:51.767941  8415 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:51.768612  7736 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 11: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:51.768889  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:51.768714  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:51.780303  8414 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:51.779376  8406 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:51.727725 13323 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:51.791414  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.727953 13696 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:51.671000 13735 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 12:27:51.797374  7728 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:51.762686 13492 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:51.841020  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.841508  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.841977  7732 raft_consensus.cc:2376] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14. Re-sending same reply.
I20260709 12:27:51.769902 13642 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.845429  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2547) took 1529 ms. Trace:
I20260709 12:27:51.847668  7803 rpcz_store.cc:276] 0709 12:27:50.316529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.410759 (+ 94230us) service_pool.cc:224] Handling call
0709 12:27:50.411206 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.104194 (+692988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.844887 (+740693us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:51.845321 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591795968}
I20260709 12:27:51.848120  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:51.848628  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:51.849673  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.841876  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.849715 13425 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:51.850698  7734 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:51.850956 13425 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:51.851253  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.852077  7734 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:51.853075  7728 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:51.852365 13425 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.842943  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.432s	user 0.025s	sys 0.064s
I20260709 12:27:51.756366 13641 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:51.863793  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.864091  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:51.860817  7728 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:51.864662  7732 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:51.865409  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:51.867554  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.869413  7733 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
W20260709 12:27:51.875241  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.465s	user 0.025s	sys 0.065s
I20260709 12:27:51.875456  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:51.875804  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.46545e+06us
W20260709 12:27:51.886039  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 128882us
I20260709 12:27:51.698457 13715 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:51.840585 13658 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.860173 13682 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:27:51.894713  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.900279  7732 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:51.903563  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1070) took 2060 ms. Trace:
I20260709 12:27:51.904011  8407 rpcz_store.cc:276] 0709 12:27:49.843112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.843328 (+   216us) service_pool.cc:224] Handling call
0709 12:27:49.912063 (+ 68735us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:27:49.917167 (+  5104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.205430 (+1288263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.902895 (+697465us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400780354. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:51.903465 (+   570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2951141376}
I20260709 12:27:51.862908 13697 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:51.925632  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:51.926787  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:51.927485  7806 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.910657 13598 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:51.912443  7737 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:51.887938 13735 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:51.916019  7732 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:51.916816 13719 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:51.847265 13309 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:51.936385  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61014.2us
I20260709 12:27:51.920230  7733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:51.929458  7730 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:51.939091  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:51.930037  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64717.9us
I20260709 12:27:51.940145  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:51.939468  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.941797  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.875528 13484 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:51.932359  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.888350 13636 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:51.944618 13688 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:51.951995  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.955691  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:51.960767  7804 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:51.863953 13731 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:27:51.900110 13481 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:27:51.984525  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48695.3us
I20260709 12:27:51.985914  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:51.981176  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.954443 13484 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:51.938138  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.638s (SENT)
I20260709 12:27:52.032183  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:51.928309 13740 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:51.877417 13524 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:27:51.990904  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.908083 13658 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.043597  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.044749  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:51.937683 13735 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:52.050737  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.039363  7737 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:51.950800 13492 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:51.920928 13650 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:27:52.061204  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.068356  7808 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:51.979892 13688 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:51.932122 13529 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:27:51.985127 13481 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:52.071889  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:51.994397 13697 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:27:52.085489  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:52.090080  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.097927  7732 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:52.098663  7733 raft_consensus.cc:606] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:52.099279  7733 raft_consensus.cc:614] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:52.091861  7737 raft_consensus.cc:2376] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:27:52.109543  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.110569  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.119202  7735 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.006s
I20260709 12:27:52.119940  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:52.061806 13632 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:52.075119 13309 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:51.983248 13751 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:52.131973  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 174656us
I20260709 12:27:52.127490 13735 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:52.128394  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.129978  7806 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:27:52.140240  7737 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:52.126597  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:51.978950 13598 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:27:52.109383 13720 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:52.144708  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 206576us
I20260709 12:27:52.156255  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.149801  7730 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:52.157415  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.117449 13636 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:52.031401 13323 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:52.167456  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.167531  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.145023 13567 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:52.143589 13697 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:52.170954  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.174310  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.177024  7737 raft_consensus.cc:2410] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:27:52.177176  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.154511 13720 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:52.192245  7730 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:52.122781 13750 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:27:52.187366  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.169903 13751 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:52.171818 13567 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:52.199528 13748 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:52.189703  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:52.191090 13720 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:52.143288 13749 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:27:52.181298 13740 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:52.288447  7737 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:27:52.295149 13753 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:27:52.362196  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.071s (SENT) [suppressed 40 similar messages]
I20260709 12:27:52.301610  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.305827 13752 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:52.305413 13356 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:52.313683 13760 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:52.286103 13761 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:52.379716  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 221853us
I20260709 12:27:52.314962 13740 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.316318 13751 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.156030 13481 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.324545  7729 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
W20260709 12:27:52.332814 13712 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
W20260709 12:27:52.334004 13756 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:52.270958 13524 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:27:52.345391  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.352946  7737 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:52.230711 13750 consensus_queue.cc:1243] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:52.375535  7808 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:52.188074 13632 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.113s
W20260709 12:27:52.410856  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.414357  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.378333  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.381460  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 200062us
I20260709 12:27:52.318754 13636 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.388212  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.413169  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:52.392468  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.391983 13428 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.863s	user 0.013s	sys 0.001s
W20260709 12:27:52.394059 13714 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.106s	user 0.013s	sys 0.004s
W20260709 12:27:52.426795  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 434198us
W20260709 12:27:52.398828 13741 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.544s	user 0.000s	sys 0.008s
I20260709 12:27:52.420215  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:27:52.423727  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 338333us
I20260709 12:27:52.435995  7626 leader_election.cc:400] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:52.436431  7626 leader_election.cc:403] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:52.438167  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:52.440088  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.440227  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.431629  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:52.418140 13481 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:52.443116  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.448199  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.450173  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:27:52.451191  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.685s (SENT)
I20260709 12:27:52.451862  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:52.452469  7733 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:52.454806  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:52.461323  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.448985  7735 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:52.426859 13484 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:52.426894 13761 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:52.429522 13650 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:52.475836  7804 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:52.447316 13697 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:52.417507 13712 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.447796 13492 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:52.511385 13704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.020s
I20260709 12:27:52.429250 13428 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.516861  8413 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:52.434648 13714 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.521953  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.518110 13704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.004s	sys 0.021s
W20260709 12:27:52.520061 13664 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.007s
I20260709 12:27:52.531946  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.533813  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95712.6us
I20260709 12:27:52.443555 13567 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:52.537433  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86280.4us
I20260709 12:27:52.539369 13766 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:27:52.549460  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.536919 13661 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.189s	user 0.018s	sys 0.004s
I20260709 12:27:52.564699  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.565857  7733 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:27:52.547051 13693 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.951s	user 0.009s	sys 0.012s
I20260709 12:27:52.531162 13650 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:52.570827  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55374.4us
I20260709 12:27:52.575138  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:27:52.574501  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.544342 13772 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:52.570143 13763 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:27:52.577261  7735 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:52.540532 13664 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.574205 13693 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.537494 13425 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:27:52.585415 13650 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.574220  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:52.593456  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.571023 13567 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:52.573832 13766 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:52.597191  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.618391  7735 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:27:52.574014  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 93679.5us
W20260709 12:27:52.606462  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71866.9us
I20260709 12:27:52.606372  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:52.588167 13524 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:52.591549 13661 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:52.595356 13765 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:52.622380  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:52.622916  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:52.573139 13761 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.650105  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.651568  7733 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 9.
I20260709 12:27:52.653937  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.654923  7735 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:52.661839  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.662992  7734 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:52.622696 13766 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.666774 13766 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:52.667865 13428 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:52.668545 13693 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:52.650662 13484 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:27:52.672070 13754 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.480s	user 0.000s	sys 0.003s
I20260709 12:27:52.624889 13524 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:52.654528 13425 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:52.692065  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.622093 13780 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:52.669930 13776 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:27:52.695374  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 161170us
I20260709 12:27:52.699628  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.706063  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.706892  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.707777  7735 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:52.712122  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:52.717314  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.649459 13661 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:52.721141  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98889.7us
I20260709 12:27:52.722263  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:52.723886  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101074us
I20260709 12:27:52.694455 13425 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:27:52.698825 13309 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
I20260709 12:27:52.734393  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:52.733197  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.735085  8413 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:52.696839 13776 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:52.744341  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.749876  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:52.750164  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.751151  7654 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:52.775599  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:52.775983  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.777171  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.777880  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.731408 13693 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:52.720913 13712 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:52.770829  8413 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:52.793614  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.794281  7735 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:52.731635 13714 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:52.796783  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 81539.5us
I20260709 12:27:52.798256  7808 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:52.799109  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.752179 13135 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:52.807020  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:52.809754  7806 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:52.807165  7662 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:52.801503  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:52.807154  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.815488  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:52.815851  7808 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:27:52.816556  7808 pending_rounds.cc:85] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 8
I20260709 12:27:52.816946  7808 pending_rounds.cc:107] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:27:52.817317  7808 raft_consensus.cc:2889] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.819010  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.820176  7656 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 11.
I20260709 12:27:52.795557 13309 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:52.825003  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.825876  7805 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:52.827648  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.827965 13766 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:52.828415 13766 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:52.829660  7812 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:52.829484 13766 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.784224 13524 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.832618 13524 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:52.832774  7661 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:52.831195  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:52.835412  7810 raft_consensus.cc:2368] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
W20260709 12:27:52.836346  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89914.1us
I20260709 12:27:52.709750 13771 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260709 12:27:52.838456  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.809145  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.811494  7653 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:52.811499  7803 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
W20260709 12:27:52.823545  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96714.7us
I20260709 12:27:52.830876 13425 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.842680  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:52.841185  8412 pending_rounds.cc:85] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 9
I20260709 12:27:52.843891  8412 pending_rounds.cc:107] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:27:52.844188  8412 raft_consensus.cc:2889] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.846768  7654 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:52.848296  7660 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:52.849773  7806 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:52.786345 13428 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:52.856361  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:52.857069  7808 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:52.860981  8415 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:52.862272  7804 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:52.836673 13661 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:52.867622  7808 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:52.830982 13766 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:52.874435  7660 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:52.878589 13654 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:52.838655 13650 consensus_queue.cc:1175] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "992d6df95ef74f7f8b12534534a45501" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 15 index: 13 } }
I20260709 12:27:52.881531 13688 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.852511 13428 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:52.883486  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:52.852473 13750 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.895805  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:52.896624  8415 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:52.893385  7659 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
W20260709 12:27:52.897969  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 167307us
W20260709 12:27:52.899493  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.885021  7654 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:52.858136 13693 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:52.858464 13771 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:27:52.883984 13664 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:52.903721  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:52.904243  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61665.6us
I20260709 12:27:52.950299  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:52.888872 13425 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:52.953475  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.954972  7737 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:52.896122 13642 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.903208  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:52.896124 13428 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:52.905997 13720 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.002s
W20260709 12:27:52.966104  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62991.7us
I20260709 12:27:52.892766 13586 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:27:52.903726 13685 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:52.967384  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.965492  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.969871  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:52.973990  7731 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:52.974805  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.975427  7735 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:52.975383  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:52.976732  7731 raft_consensus.cc:2368] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:52.901953 13688 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.981004  8413 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:52.901479 13793 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:52.991663 13728 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:52.992164 13728 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:52.994009 13752 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:52.994560 13752 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:52.982335 13735 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.823s	user 0.000s	sys 0.014s
W20260709 12:27:53.000425  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50235.4us
I20260709 12:27:52.998358  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:52.969947 13697 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:52.979640 13661 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:53.001916  8380 leader_election.cc:400] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:53.004245  8380 leader_election.cc:403] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:27:53.001840  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:52.876474 13776 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.005913 13776 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:52.995532 13752 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.008423 13752 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.019896 13799 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.020615 13799 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.986861 13685 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:52.980088 13664 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.027652 13664 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.003407 13693 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.030563 13664 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:53.032722 13481 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.003s
I20260709 12:27:53.033478 13735 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.961100 13642 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.036130 13735 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.038441 13762 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:53.038972 13762 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:53.039428 13762 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:53.044049 13661 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:53.045280 13661 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.901564 13750 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.049293 13750 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:52.993014 13728 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.052050 13728 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:53.057507  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:53.059065  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53389.3us
I20260709 12:27:53.059998  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.063372  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.068806  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.072013  7735 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
W20260709 12:27:53.072322  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70552.1us
I20260709 12:27:52.972579 13323 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:53.074156  7805 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:53.075826  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.082744  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.061017 13650 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:53.094283  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:53.100255  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2941) took 1469 ms. Trace:
I20260709 12:27:53.116003  7729 rpcz_store.cc:276] 0709 12:27:51.631048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.631673 (+   625us) service_pool.cc:224] Handling call
0709 12:27:53.099853 (+1468180us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:53.100121 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496679296}
I20260709 12:27:53.095321 12650 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:53.093684  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:53.117477 12650 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:53.118067  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.117997  7735 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:53.073419 13719 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
W20260709 12:27:53.121050  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.124155  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.073419 13766 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:53.073670 13792 raft_consensus.cc:2764] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:27:53.099457 13790 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.004s
I20260709 12:27:53.081527 13799 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:53.136296 13794 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.000s	sys 0.003s
W20260709 12:27:53.175621  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 2966) took 1263 ms (client timeout 1638 ms). Trace:
I20260709 12:27:53.106236 13747 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:53.096005 13331 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:53.081782 13786 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:53.104036 13796 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:53.135418 13777 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:27:53.128990 13323 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:53.130506 13793 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:53.176132  7732 rpcz_store.cc:269] 0709 12:27:51.912376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.951660 (+ 39284us) service_pool.cc:224] Handling call
0709 12:27:53.134847 (+1183187us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001c4dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:53.175556 (+ 40709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314898816}
I20260709 12:27:53.106603 13656 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:53.182188  7734 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:53.108361 13759 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:27:53.197014  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121315us
I20260709 12:27:53.186213 13799 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.174831 12650 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:52.981674 13720 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.203342 13331 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:53.227972 12650 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.179103  7737 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:53.195200 13747 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:53.182411 13688 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:53.189385  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65317.4us
I20260709 12:27:53.235699  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.209275 13712 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:53.209064 13760 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:53.214246 13656 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:53.217917 13759 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:53.224243 13484 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.008s	sys 0.009s
I20260709 12:27:53.249754  7805 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:27:53.250206  7658 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 11: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:53.253856  7805 pending_rounds.cc:85] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 10
I20260709 12:27:53.254169  7805 pending_rounds.cc:107] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 16.11
W20260709 12:27:53.241089 13529 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.015s	sys 0.010s
I20260709 12:27:53.254627  7805 raft_consensus.cc:2889] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:53.256445  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3132535960003224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:27:53.256935  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.258775  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3132535960003224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:27:53.259155  7702 connection.cc:256] RPC call timeout handler was delayed by -1.171953596000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 12:27:53.238257 13712 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.263907 13712 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.241259 13760 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:53.266801 13760 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:53.273733 13331 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.274410  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:53.252976 13484 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:53.277915 13769 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.685s	user 0.000s	sys 0.004s
I20260709 12:27:53.277669 13484 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.228240 13799 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.284133 13759 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.285995 13759 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:53.287732 13747 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.289754 13747 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.291615 13656 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.293044 13656 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.293231  7806 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:27:53.294948  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:53.240787 13688 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.300565  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:53.300661  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:53.301657  7803 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:53.302160  7812 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 12:27:53.293291  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.314280  8415 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 12:27:53.259763  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1548535960000663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
W20260709 12:27:53.325737  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1044535960004396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 12:27:53.326309  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1044535960004396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 12:27:53.326721  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0126535960002911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:27:53.327068  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0117535960002897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 12:27:53.327448  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0117535960002897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 12:27:53.328749  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.260663 13529 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.015s	sys 0.010s
I20260709 12:27:53.332150  7654 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:53.333544  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.334138  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.336598  7660 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:53.337149  7662 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:27:53.337797  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:53.340893  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.341524  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.342147  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.342985  7656 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:53.345182  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.345854  7656 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:53.346715  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.347856  8415 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:53.349012  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.348937  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.352387  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:53.353685  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.355993  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.357440  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:53.395669  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.371706  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:53.386102  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.384860  7654 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:53.372239  7656 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:53.397464  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:53.400727  8412 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:53.375874  8407 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 12:27:53.401016  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.401557  8407 pending_rounds.cc:85] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 12
I20260709 12:27:53.401899  8407 pending_rounds.cc:107] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:27:53.387434  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:53.402303  8407 raft_consensus.cc:2889] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:53.361157  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.363581  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.403942  7655 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:53.330235 13524 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:53.406634  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.407308  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.414808  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:53.355029 13801 raft_consensus.cc:2764] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:53.417404  7657 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:27:53.418629  7662 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:27:53.448570  7737 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:53.419374  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:53.420444  7656 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:53.352620 13750 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.380232 13720 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.394809 13305 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:53.444199  8379 leader_election.cc:400] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15.
I20260709 12:27:53.381803 13800 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:53.439510  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.455415  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.379127 13688 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:53.431958 13696 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.444s	user 0.016s	sys 0.032s
I20260709 12:27:53.448623  7657 pending_rounds.cc:85] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 6
I20260709 12:27:53.458997  7657 pending_rounds.cc:107] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:53.459462  7657 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.457729 12650 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.461833 12650 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.450060  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:53.387319 13697 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:27:53.470459  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:53.375352 13777 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:27:53.480090 13777 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:53.474287  8412 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:53.475507 13722 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:53.468411  7656 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:53.480609 13777 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:53.481560 13722 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.482074 13722 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:53.490186  7653 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:53.456998 13773 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:53.501118 13773 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:27:53.464903 13696 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.481s	user 0.016s	sys 0.032s
I20260709 12:27:53.503337  7659 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:27:53.456609 13750 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.510480 12650 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:53.485493 13290 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:53.513288  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:53.517391  7660 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:27:53.510198  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60371.8us
I20260709 12:27:53.519959  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:53.522235 12650 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:53.522866 12650 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:53.523777  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:53.522979  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:53.526374  7737 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 17 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 12:27:53.529681  7656 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 17 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 12:27:53.481765 13777 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:53.511531 13428 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.525s	user 0.018s	sys 0.000s
I20260709 12:27:53.512537 13720 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:53.543814  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.546345  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.512187 13719 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:27:53.549520  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.482519 13760 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.530143 13697 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:53.552747  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 200438us
W20260709 12:27:53.554203  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.554409  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:53.534746 13799 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.536875  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:53.512495 13290 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:53.502707 13773 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.560997  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.536247 12650 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.548843 13796 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:53.569311 13762 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.000s
I20260709 12:27:53.561862 13761 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:53.563280 13777 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.570598 13425 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:53.589157  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.586843  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.571952 13775 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:53.575474 13800 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:53.571522 13697 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:53.601424  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.553431 13428 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.583943 13529 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:53.617370  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:53.594511 13761 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.606452 13775 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.629477 13775 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:53.629510 13809 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:27:53.632737 13688 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:53.633186 13688 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:53.634125 13688 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.636794 13642 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.639858  7657 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:27:53.640481  7657 pending_rounds.cc:85] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 10
I20260709 12:27:53.640803  7657 pending_rounds.cc:107] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:27:53.641124  7657 raft_consensus.cc:2889] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.641559  8414 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:53.606139 13800 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:53.600040 13305 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:53.627600 13664 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.628479 13656 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:53.609284 13323 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:53.648396  7811 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:53.645071  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:53.591647 13776 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:27:53.598259 13762 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.643867 13761 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:53.625268 13801 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:53.643353 13813 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:53.668033  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.648823 13800 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:53.615125 13793 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:27:53.673537  7811 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:27:53.619805 13771 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:27:53.685954  8406 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:53.656878 13323 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.693042 13323 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.699923 13813 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:53.699787 13642 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:53.700716 13813 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:53.701246 13813 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:53.702579 13814 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:27:53.710081 13798 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.490s	user 0.002s	sys 0.000s
I20260709 12:27:53.719589 13715 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:53.655820 13664 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.724573 13664 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.725464  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:53.736517  7734 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:53.728787  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:53.669977 13801 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:53.651571 13290 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.748325 13656 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:53.745606 13750 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:53.768335  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 123448us
I20260709 12:27:53.748068 13810 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:53.756958 13801 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:53.776780  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 126777us
I20260709 12:27:53.774896  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:53.771580 13656 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:53.762368 13712 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.785326  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.777619 13810 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:53.792034 13810 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.792649 13658 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.795102 13481 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:53.795701 13481 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:53.787246 13799 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:27:53.801573  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.796627 13481 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.802516  7661 raft_consensus.cc:2368] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14.
W20260709 12:27:53.788529 13815 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:27:53.784219 13826 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:53.807565  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.810181  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1078) took 1370 ms (client timeout 1605 ms). Trace:
I20260709 12:27:53.813580  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:53.817732  7731 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:53.815261  7658 rpcz_store.cc:269] 0709 12:27:52.439341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.439811 (+   470us) service_pool.cc:224] Handling call
0709 12:27:53.809905 (+1370094us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54000f3554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:53.810072 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858855680}
I20260709 12:27:53.748608 13697 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.812897  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:53.833160  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:53.833477  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49768.2us
I20260709 12:27:53.836525 13823 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:53.837134 13824 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:53.839206  7661 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:53.821713 13428 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.822067 13826 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:53.844771 13826 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.804366 13762 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:53.826092 13760 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.826208 13656 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.831341  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.858368 13775 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.870012 13331 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:53.870381 13775 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.816490  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:53.803130 13481 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.828778 13722 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.887158  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:53.887172  7730 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:27:53.888242  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71797us
I20260709 12:27:53.849273 13807 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:53.830096 13688 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:53.849325  7660 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:53.890887  7660 pending_rounds.cc:85] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 8
I20260709 12:27:53.891340  7660 pending_rounds.cc:107] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:53.891651  7660 raft_consensus.cc:2889] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.854310 13685 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:53.855855 13810 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.819057 13658 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.857080  7728 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:27:53.857681  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.895659 13693 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.877378  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.840653 13664 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:53.893631 13656 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:53.893748 13757 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:53.894306 13323 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:53.901122  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.923904  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.901597 12650 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:53.902371  8412 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:53.874365 12990 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:53.903463 13807 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:53.904098  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:53.905332  7736 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:27:53.930770  7736 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 12
I20260709 12:27:53.931130  7736 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:53.896605 13760 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:53.931455  7736 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:53.909767  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.912725  7737 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:27:53.919334  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.920854  7731 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:27:53.922289  7661 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:53.937664 13722 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:53.908864 13658 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.910723 13693 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:53.865659 13800 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.943171 13800 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.944881 13810 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:53.945291 13664 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:53.945808 13323 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:53.949136  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.946430 13664 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:53.953975  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:53.934686  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:53.958354  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 123934us
I20260709 12:27:53.959575  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:53.961773  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.962667  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:53.972493  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.974678  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63113.9us
I20260709 12:27:54.069916  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:53.968212 13823 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:53.967002 13760 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:53.974426 13740 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:54.067508 13826 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:54.068603 13757 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:54.092752 13757 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:54.078121 13803 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:27:53.966429 13813 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.080626 13823 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.110812  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:27:54.115799  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.066735 13800 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:27:54.030471 13779 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:27:54.121347 13813 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.115937 13323 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:27:54.123379  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.123554  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.124199  7734 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:54.124321  7661 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:54.124105  7658 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.130327  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:54.133880  7728 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:54.089653 13796 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:54.136804  7777 leader_election.cc:400] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14.
I20260709 12:27:54.150105  7777 leader_election.cc:403] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:54.144670  7661 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:27:54.149186  7728 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:27:54.152521  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:54.161793  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:54.166846 13658 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.092020 13740 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:54.083323 13777 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:27:54.108289 13712 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:54.187126  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50678.1us
I20260709 12:27:54.126907 13823 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:54.190368  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 227800us
I20260709 12:27:54.191319  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.190403  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:54.193976  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.194149  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.194938  7661 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.195854  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:54.181789 13722 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:54.195123  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:54.181823 13779 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.133579 13763 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:27:54.164924 13760 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.189321 13517 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14.
I20260709 12:27:54.203081  8406 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:54.209553  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.233722 13658 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:54.262893 13722 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:54.267926 13658 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:54.268508 13658 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.268914 13722 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.196455 13800 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:54.271461 13839 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:27:54.263562 13794 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:54.192114 13766 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:27:54.277637 13800 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:54.278302 13658 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.240232 13786 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:27:54.288692  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 79318.2us
W20260709 12:27:54.289510 13836 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:54.290118  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.288874 13658 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.292375 13800 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:54.293169 13305 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:54.295002 13835 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:27:54.301235  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.302099  7656 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:54.303784  7655 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:54.308393 13524 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.309290 13524 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.312474  8407 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:27:54.318838 13524 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.320987 13749 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:54.298244 13740 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.309259 13305 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:54.295439  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:54.324329  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:54.325970  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.322472 13524 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:54.329453  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.330468  7656 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.250283  7658 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:27:54.212831  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44313.2us
I20260709 12:27:54.332089  7661 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:54.342171  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:54.342171  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.197332 13529 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:54.197608 13813 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:54.359609  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:54.342830 13800 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:54.376255 13800 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:54.376847 13800 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:54.348635 13656 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:54.379667 13656 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:54.380141 13656 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:54.383340 13776 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:54.384246 13766 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.384958 13766 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.232595 13704 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.234790 13779 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.387705 13704 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.390137 13779 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.390375 13704 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.390846 13529 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:54.197597 13801 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.392064  7732 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:54.392717 13524 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:54.393436 13524 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:54.393977 13751 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:54.392060 13529 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.233825 13323 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.138s
I20260709 12:27:54.407485 13763 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:54.356227 13658 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:54.405069 13837 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.412264  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.420775  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:54.393316 13801 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.371621 13688 raft_consensus.cc:2764] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:54.434274  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:54.435459  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.436913  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.432139  7656 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:27:54.437685  7655 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:54.430969  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.399816 13776 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:54.447160 13776 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.450264  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1094) took 1055 ms. Trace:
I20260709 12:27:54.394594 13524 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.442018 13779 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:54.455395 13777 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.428685 13813 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:27:54.432977 13685 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:27:54.430737 13837 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.459553  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:54.481561  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:54.482401  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.491313  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.491313  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.492291  7655 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:54.492458  7656 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:54.494611  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.494781  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:54.450771  8415 rpcz_store.cc:276] 0709 12:27:53.394591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.396515 (+  1924us) service_pool.cc:224] Handling call
0709 12:27:53.396864 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.397092 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:53.564140 (+167048us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.564782 (+   642us) log.cc:844] Serialized 14 byte log entry
0709 12:27:54.449510 (+884728us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b1800375ee0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fc001ad8d70 00007fc001acc293 00007fc001ac1cea 00007fc001ac223a 00007fc001f2ca2a 00007fc001f3c0bf 00007fc001f3c038
0709 12:27:54.449839 (+   329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:54.449902 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:54.449957 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.450032 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:54.450179 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4477,"prepare.queue_time_us":436715,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":94,"raft.queue_time_us":4765,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":7335,"spinlock_wait_cycles":11128064,"thread_start_us":166606,"threads_started":1}
I20260709 12:27:54.496783  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:54.456234 13777 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.498198  7808 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:54.499282  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.500300  7658 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:54.440557 13763 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.394604 13751 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.503016  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:54.509140  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74968.1us
W20260709 12:27:54.503326  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 279723us
W20260709 12:27:54.505036  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45582.2us
I20260709 12:27:54.507448  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:54.509951  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.510198  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:54.511878  7658 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:54.461089 13779 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:54.460641 13801 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:54.514110  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:54.537909  7806 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.432103 13823 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.517006  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.517557 13697 raft_consensus.cc:2764] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:54.517712  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.518916  7657 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [12.8-15.9]
I20260709 12:27:54.548338  7661 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:54.517048 13771 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:54.517009  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:54.553211  7805 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:54.517737  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:54.521626 13837 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.521953  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1084) took 1188 ms. Trace:
I20260709 12:27:54.496333 13760 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.483647 13658 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:27:54.530673 13800 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.484184 13773 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.503676 13832 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:54.515623  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.542656  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:54.429960 13846 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:27:54.546036  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:54.499374 13715 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:54.551543  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:54.513381 13766 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:54.556525  7653 rpcz_store.cc:276] 0709 12:27:53.333449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.334050 (+   601us) service_pool.cc:224] Handling call
0709 12:27:53.334348 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.504323 (+169975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.521582 (+1017259us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:54.521821 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349006720}
W20260709 12:27:54.524690 13818 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.874s	user 0.001s	sys 0.001s
I20260709 12:27:54.512614 13356 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:54.527012 13775 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:54.588903  7653 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:27:54.572544  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57023.8us
I20260709 12:27:54.596239  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.597946  7806 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:54.598170  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.552374 13771 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.514379 13763 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.599884  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.601480  7805 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:54.607847  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.587322 13697 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:54.608626 13761 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.923s	user 0.016s	sys 0.000s
I20260709 12:27:54.568868 13722 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:27:54.591806 13775 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:54.605738 13771 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:54.607023 13481 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:54.610662 13715 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:54.652603 13775 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.653859 13775 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:54.632027 13751 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.633720 13800 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.635093  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2620) took 1289 ms. Trace:
I20260709 12:27:54.583768 13773 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.636776  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:54.564463 13760 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.660653  7654 rpcz_store.cc:276] 0709 12:27:53.346008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.409231 (+ 63223us) service_pool.cc:224] Handling call
0709 12:27:53.409780 (+   549us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:53.415880 (+  6100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.415929 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:53.446744 (+ 30815us) log.cc:844] Serialized 52 byte log entry
0709 12:27:53.572165 (+125421us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:53.749618 (+177453us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.811015 (+ 61397us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.811308 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.811408 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:54.524032 (+712624us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:54.634807 (+110775us) raft_consensus.cc:1685] finished
0709 12:27:54.634834 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:54.635022 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65782,"prepare.queue_time_us":289673,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":3841,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":11213,"spinlock_wait_cycles":5954176,"thread_start_us":233678,"threads_started":2,"wal-append.queue_time_us":181828}
I20260709 12:27:54.663465  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.645964 13763 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.569669 13838 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:27:54.628772 13356 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:54.655488 13779 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.640300 13761 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.691824  7803 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:54.689899  7658 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:54.672262 13356 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.698459 13823 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.707739  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1088) took 1868 ms. Trace:
I20260709 12:27:54.708228  8413 rpcz_store.cc:276] 0709 12:27:52.839698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.860040 (+ 20342us) service_pool.cc:224] Handling call
0709 12:27:52.861954 (+  1914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.981450 (+119496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.707421 (+1725971us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400780354. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:54.707670 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273081088}
I20260709 12:27:54.722406 13773 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.746546 13760 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:54.767895  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.769240  7658 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:54.707609 13656 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:54.770471  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 172420us
I20260709 12:27:54.771173  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:54.767809 13802 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.387s	user 0.016s	sys 0.016s
I20260709 12:27:54.769860  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:54.781802  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:54.783692  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.784551 13761 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:54.785975 13823 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:54.786520 13823 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.786885 13823 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.787283  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:54.788703  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.789640 13813 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:54.790405 13802 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.410s	user 0.016s	sys 0.016s
I20260709 12:27:54.790649 13688 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:54.772811  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.791244 13809 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:54.792418  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.779080 13747 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:54.792754 13809 consensus_queue.cc:1243] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:54.793349 13658 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:54.793473 13747 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:54.779016  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:54.791431 13846 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:27:54.795387 13846 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:54.795763 13846 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:54.796589 13779 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:54.797402  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.759922 13800 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.796643 13846 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.799772 13800 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.800896 13846 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:54.801031  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.792263 13685 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:54.802263 13685 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:54.803022  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:54.802975 13685 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:54.765646 13664 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.790320 13813 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.804988  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:54.828801  8413 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:54.824746  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:54.819398  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.827553  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:54.822201  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.805399  7658 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:54.822568  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.832494  7654 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:54.831259  7653 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:27:54.806531  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.800305  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.834679  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:54.811941  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 935) took 1275 ms. Trace:
I20260709 12:27:54.835974  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.835999  8412 rpcz_store.cc:276] 0709 12:27:53.536189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.536524 (+   335us) service_pool.cc:224] Handling call
0709 12:27:54.811744 (+1275220us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400469e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:54.811879 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763514752}
I20260709 12:27:54.839109  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:54.840849  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.840951  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.841777  7654 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:54.841871  7653 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:54.845448 12650 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:54.845999 12650 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.846400 12650 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.845784  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:54.793617 13809 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:54.849242  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.847157 13864 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:54.851585  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:54.844590  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:54.804466 13688 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:54.800944 13779 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:54.848058  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.813377 13757 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.860610  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:54.861455  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.859620  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:54.804791 13685 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:54.841845 13813 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:54.803171 13800 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:54.853907 13812 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.490s	user 0.013s	sys 0.000s
W20260709 12:27:54.867620  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.870608  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.847242 13529 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.849853 13715 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:54.847733 13860 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:54.872354 13685 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.865293 13356 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:54.855963 13632 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:54.862795 13658 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:54.824841 13664 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.850075 13826 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:27:54.873097 13688 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:54.893327  7728 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:27:54.963083  7728 pending_rounds.cc:85] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:27:54.894673  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:54.963438  7728 pending_rounds.cc:107] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:27:54.822248 13823 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.968559 13766 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:54.968564  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:54.914149 13656 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:54.858584 13517 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:27:54.970758  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121277us
I20260709 12:27:54.965273  7653 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [14.10-17.11]
W20260709 12:27:54.975811 13866 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:27:54.976133 13524 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:54.907720  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.978579 13664 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:54.854867 13864 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:54.965862  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71312.4us
I20260709 12:27:54.938696  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:54.976905 13524 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.892081  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:54.989969  7737 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:54.963778  7728 raft_consensus.cc:2889] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:54.969610 13766 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.981673  7803 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:54.874351 13773 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:54.966769  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:54.984295  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:54.984328  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:54.894896  7729 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:54.998152  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:54.982514  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:54.999527  7654 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:54.987198  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.000869  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:54.886978 13685 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:54.881636 13808 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.510s	user 0.007s	sys 0.000s
I20260709 12:27:54.876718 13786 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:54.990558  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:54.981050 13779 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:54.931655  7730 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:54.956372  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:27:54.995489  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44993.5us
I20260709 12:27:54.914700 13862 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:54.906865 13425 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:55.013124  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.005688  7728 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:54.909125 13712 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:54.883270 13801 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:54.983036  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:55.008340  7810 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:55.009207  8407 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:54.879284 13771 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:54.901554 13356 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.907066 13763 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.914609 13658 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.914387 13632 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.014366  8415 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:54.989440 12650 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:54.989352 13517 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.028398  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.070130  7809 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.020663  7807 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:54.988680 13864 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:54.879532 13529 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:55.035840  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79625.8us
I20260709 12:27:55.038302  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.077080  7812 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.014004 13751 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:55.010064 13860 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:55.043433  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:55.086928  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43634.9us
I20260709 12:27:55.088212  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:55.014015 13786 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.109807  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:55.111402  7659 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.009917  7662 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:27:55.050065  7656 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:55.023695 13832 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:55.059516 13862 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:55.054937 13356 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:55.075392  7737 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:55.046125 13801 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:55.084533 13875 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:55.051856 13290 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:55.106348 12650 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.069931 13658 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:55.061363  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48375.4us
I20260709 12:27:55.072927 13632 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:55.036634 13425 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.105949 13824 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.173799  7805 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:55.177496  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.117007 13800 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:55.085839 13751 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.048470 13688 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:55.187376  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.120098 13786 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:55.049162 13712 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.068512 13763 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.117924 13832 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:55.053607 13846 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:27:55.175341 13875 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.196194  7809 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:55.085552 13837 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 12:27:55.073670 13517 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.085392 13731 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 12:27:55.198513 13751 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.109037 13719 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:27:55.117233 13757 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:27:55.199966 13800 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:55.209591 13862 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:55.195914 13809 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:55.173151 13799 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:55.217118  7803 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.200322 13801 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.206604  7808 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:55.210472  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:55.229038  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.211498 13776 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.211036 13878 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.215317 13823 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.195578 13824 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.215083  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.239856 13771 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:55.240387 13869 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:27:55.240815 13517 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:55.210418 13875 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:55.231599 13809 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:55.246148 13763 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:55.248258  7812 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:55.252746 13801 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.258179 13813 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.268893  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.271860  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:55.296178  7660 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:27:55.327180  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 150674us
I20260709 12:27:55.333468  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.244596 13712 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:55.318392  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103384us
I20260709 12:27:55.334287  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.335328  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.316885 13777 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:55.321241 13862 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:55.325834 13809 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.347018  8406 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:55.344588  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 256430us
W20260709 12:27:55.348950  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 155282us
I20260709 12:27:55.325656 13524 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.356230  7803 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:55.374065  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:55.377666  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.276280 13776 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:55.369470  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 969) took 2077 ms (client timeout 2473 ms). Trace:
I20260709 12:27:55.356801  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:55.379031  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:55.384250  7811 rpcz_store.cc:269] 0709 12:27:53.291861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.294585 (+  2724us) service_pool.cc:224] Handling call
0709 12:27:55.369180 (+2074595us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54003e9ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:55.369372 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810822016}
I20260709 12:27:55.384987  7807 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:55.382332  8407 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:55.280840 13876 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 12:27:55.388545  7654 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.385643  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.281057 13878 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.267288 13832 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.394106  7627 leader_election.cc:400] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:27:55.391782  7811 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:27:55.341979 13837 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:55.408267  7807 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:55.399832  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.405691  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:55.417799  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 118751us
I20260709 12:27:55.387509 13877 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:55.392683 13529 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:55.429379  7809 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:55.393350 13874 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:55.428103  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.406885 13860 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:27:55.436463  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 86062.7us
W20260709 12:27:55.442664  7658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.013s
W20260709 12:27:55.443092  7658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.011s	sys 0.013s
I20260709 12:27:55.445701  7729 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:55.446468  7728 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:55.454129  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.456984 12650 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:55.460337  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:55.458810  7659 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:27:55.370703 13800 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.462487  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.416805 13823 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:27:55.470377  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.473589  7807 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:27:55.441076 13860 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:55.421991 13837 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.452482 13779 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:55.455498  7654 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:55.483605 13860 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:55.491154  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:55.491122  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115233us
I20260709 12:27:55.503890  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:55.460461 13664 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:27:55.450608 13874 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.566255  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:55.559739  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55954.1us
I20260709 12:27:55.512280 13777 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.520737 13776 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.523770 13428 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:55.532585 13837 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.535457 13693 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.537736 13878 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:55.517172  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.559433  7812 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:55.567432  7729 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:55.620983  7811 raft_consensus.cc:2376] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15. Re-sending same reply.
W20260709 12:27:55.571318  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178633us
I20260709 12:27:55.564574 13824 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.577313  7730 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:55.565399  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.580399  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.483661 13887 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 12:27:55.633138  7805 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.633994  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.581620  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.636294  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2960) took 1743 ms. Trace:
I20260709 12:27:55.636796  7732 rpcz_store.cc:276] 0709 12:27:53.892730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.892901 (+   171us) service_pool.cc:224] Handling call
0709 12:27:53.894525 (+  1624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.392726 (+498201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.635958 (+1243232us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b5400072bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:55.636223 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143545088}
W20260709 12:27:55.556458 13873 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 12:27:55.586433  7659 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:55.562175  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.543135 13425 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.613102  7660 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:55.550958 13731 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:55.572634  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 167046us
I20260709 12:27:55.674908  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.625708  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.576685 13792 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:55.602782  7657 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:55.572341 13775 raft_consensus.cc:2764] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:55.667148  7804 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:55.667596  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:55.682009  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:55.682947  7732 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.619504 13813 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.690935  7807 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:55.690577 13792 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.692476  7807 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:55.693259 13792 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.694010  7807 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:27:55.695729 13685 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 12:27:55.697343 13685 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:55.697690  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.698117 13685 consensus_queue.cc:1243] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:55.698457  7811 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:27:55.699609 13356 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.520s	user 0.002s	sys 0.010s
I20260709 12:27:55.700042 13356 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.700940  7732 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:55.701078  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.702852 13779 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:55.708187  7809 raft_consensus.cc:2368] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17.
I20260709 12:27:55.586463 13862 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:27:55.686139  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.717821  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:55.702720 13356 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.728257  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.724407  7730 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:55.587189 13777 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:55.732056  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65660.9us
I20260709 12:27:55.732183  7734 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:55.677847 13887 consensus_queue.cc:1243] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:55.735883  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.584385 13796 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 12:27:55.723973 13824 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:55.685621 13880 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:55.716079 13870 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.087s	user 0.000s	sys 0.003s
I20260709 12:27:55.687908 13731 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:55.742066  7805 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
W20260709 12:27:55.720646 13658 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.001s
I20260709 12:27:55.765137 13658 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.765990 13425 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:55.770200  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.749271  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.772557  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.773003  7730 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:55.774238  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:55.727988 13779 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:55.775616 13779 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.776270 13779 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.776896 13766 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.768361 13838 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:55.779430 13838 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:55.781641  8407 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:55.780599 13838 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.783603  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:55.768283 13356 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:55.729669 13775 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:55.694895 13751 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.764523  7734 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:27:55.788904  7734 pending_rounds.cc:85] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:27:55.789202  7734 pending_rounds.cc:107] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:55.789513  7734 raft_consensus.cc:2889] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:55.786314  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:55.790558  8412 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:55.787060  7732 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:55.762625 13731 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.801885  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:55.776597 13796 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:55.793113  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:27:55.794059  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.795392  7730 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:55.812153 12650 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:55.813072 12650 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:55.811934  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71081.9us
I20260709 12:27:55.814141 12650 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.815013 13650 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:55.831574  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.747673 13824 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:55.857983  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.785413 13719 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.807492  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.859824  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:55.862406  7656 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:55.873536  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:55.873688  7659 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:55.882903  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:55.883649  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76357.3us
I20260709 12:27:55.884874  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:55.888019  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:55.893373  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 270456us
I20260709 12:27:55.896421  7730 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:27:55.896965  7730 pending_rounds.cc:85] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:27:55.897220  7730 pending_rounds.cc:107] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:27:55.897464  7730 raft_consensus.cc:2889] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:55.899856  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.900633  7805 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:55.805656 13796 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:55.826495 13893 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.906353  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.902866  7730 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:27:55.907521  7659 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:27:55.914374  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:55.915273  7655 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:55.913550  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:55.918104  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:55.801093 13775 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.913465 12650 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:27:55.760968 13880 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:55.827062 13664 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 12:27:55.886200 13428 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:55.911324 13877 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:55.924549 12650 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:55.931252  7805 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:27:55.936554  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.891261 13773 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:55.965297 13773 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.934036  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:55.885641 13719 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:55.948770  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:55.968254  7811 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:55.949587  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:55.969362  7655 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:55.924250  7734 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:55.891173 13425 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:27:55.922937 13323 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:55.921868 13658 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:55.932456 13761 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:55.969532  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.911427 13894 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:27:56.025171  7805 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:56.092388  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.093624  7732 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:56.094704  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.096530  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:55.934659 13766 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:27:55.994432 13323 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.971724 13779 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:55.993181 13877 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:56.104274  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:56.128196  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.120846  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:56.137581  7732 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:55.910837 13893 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:56.165828 13898 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:27:56.171432 13911 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:27:55.946111 13824 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.178012 13773 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.130228 13323 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:56.209981  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.100432 13800 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.220983  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:56.236078  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 131910us
I20260709 12:27:56.279713  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:56.279922 13896 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:56.280864 13773 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:56.282029  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.283092  7732 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:56.246904 13864 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.137s	user 0.007s	sys 0.008s
I20260709 12:27:56.013797 13796 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.296646  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.289009  7732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
W20260709 12:27:56.248134 13632 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.049s	user 0.015s	sys 0.005s
W20260709 12:27:56.271335 13801 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.936s	user 0.012s	sys 0.006s
I20260709 12:27:56.297801  7734 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:56.298592  7732 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:56.292443  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:56.258939 13902 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:56.310236 13916 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:27:56.300262 13922 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:27:56.301025 13906 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:27:56.313591  7736 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:27:56.314329 13779 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:27:56.316442  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2572) took 1320 ms (client timeout 1625 ms). Trace:
W20260709 12:27:56.349537  7803 rpcz_store.cc:269] 0709 12:27:54.996387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.997800 (+  1413us) service_pool.cc:224] Handling call
0709 12:27:56.316200 (+1318400us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.316364 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500933376}
I20260709 12:27:56.318298  7810 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:56.352484  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1230) took 1030 ms. Trace:
I20260709 12:27:56.303323 13864 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.354225  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:56.353165  7737 rpcz_store.cc:276] 0709 12:27:55.322307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.322477 (+   170us) service_pool.cc:224] Handling call
0709 12:27:55.335062 (+ 12585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.335486 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:55.768873 (+433387us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18004d8720. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:27:55.836999 (+ 68126us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.239201 (+402202us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.239900 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.240085 (+   185us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.240135 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.240237 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.352102 (+111865us) raft_consensus.cc:1685] finished
0709 12:27:56.352219 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.352400 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15641,"prepare.queue_time_us":221116,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":49,"raft.queue_time_us":590,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":32918,"spinlock_wait_cycles":6195072,"thread_start_us":406817,"threads_started":2,"wal-append.queue_time_us":477424}
I20260709 12:27:56.354969  7656 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:56.359387 13918 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.360189  7728 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:56.360067 13918 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:56.335347 13809 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.948s	user 0.011s	sys 0.004s
I20260709 12:27:56.362490 13809 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.363327 13779 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.290328 13719 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.364089 13779 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.364739  7737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:56.365554  7737 raft_consensus.cc:606] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:27:56.328004 13907 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:56.366314  7729 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:56.306540 13824 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.289934 13878 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:56.344352 13775 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.368312  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:56.368628 13775 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:56.369077  7657 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:56.369313 13775 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.308204 13632 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.347899 13893 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.328015 13796 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:56.289644 13800 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.311077 13877 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.372046 13632 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.372375  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2570) took 1387 ms. Trace:
I20260709 12:27:56.372761  7808 rpcz_store.cc:276] 0709 12:27:54.984830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.984988 (+   158us) service_pool.cc:224] Handling call
0709 12:27:54.985394 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.238392 (+252998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.372078 (+1133686us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:56.372308 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507972480}
W20260709 12:27:56.373418  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2559) took 1877 ms (client timeout 1558 ms). Trace:
I20260709 12:27:56.366178 13907 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.371027  8413 raft_consensus.cc:2368] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18.
I20260709 12:27:56.375341 13809 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.364483 13922 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.377858 13864 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.378373  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1111) took 1465 ms. Trace:
I20260709 12:27:56.366439 13894 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.378871  7662 rpcz_store.cc:276] 0709 12:27:54.913041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.915570 (+  2529us) service_pool.cc:224] Handling call
0709 12:27:54.915921 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.112353 (+196432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.378097 (+1265744us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b540028df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:56.378304 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214829952}
I20260709 12:27:56.379375 13894 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:56.303695 13860 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.817s	user 0.006s	sys 0.007s
I20260709 12:27:56.380584  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:56.380508 13860 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.381363  7662 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:56.382544  7660 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:56.389133 13786 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.179s	user 0.010s	sys 0.005s
I20260709 12:27:56.389739 13786 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.391736  7807 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:56.362839 13918 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.392480  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.391435  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.396461 13918 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:56.397248 13918 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.370915  7737 raft_consensus.cc:614] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.403721  7737 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:56.404886  7737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691f0d885861432f91375d5f288af312"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:56.422554  7737 raft_consensus.cc:606] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:56.426064  7737 raft_consensus.cc:614] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.409235 13731 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:56.373839  7804 rpcz_store.cc:269] 0709 12:27:54.496277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.496440 (+   163us) service_pool.cc:224] Handling call
0709 12:27:56.373225 (+1876785us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400331654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.373345 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2689562368}
W20260709 12:27:56.428357  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.429785  7660 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:27:56.435566  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.003s	sys 0.018s
W20260709 12:27:56.436152  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.018s
W20260709 12:27:56.436542  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 517444us
I20260709 12:27:56.444826  7730 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 11.
I20260709 12:27:56.449894  7655 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:56.334589 13751 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.422721 13864 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.476323 13923 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:56.477169 13922 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:56.481488 13875 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.149s	user 0.019s	sys 0.000s
I20260709 12:27:56.482103 13875 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.462512 13688 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.489022 13894 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.489508 13688 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.490654 13779 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.436755 13809 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.498317  7657 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:56.471342 13685 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.469115 13801 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:56.496065 13786 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:56.519498  7662 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:56.521294 13875 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:56.518570  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:56.532644  7660 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:56.527319  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 2655) took 1256 ms (client timeout 1511 ms). Trace:
W20260709 12:27:56.533414  7654 rpcz_store.cc:269] 0709 12:27:55.271025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.271460 (+   435us) service_pool.cc:224] Handling call
0709 12:27:56.527058 (+1255598us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.527199 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129193728}
I20260709 12:27:56.530332 13832 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:56.497283 13864 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:56.542371  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.008s	sys 0.019s
I20260709 12:27:56.500262 13894 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:27:56.504606 13923 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:27:56.503189 13809 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:56.543100  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.008s	sys 0.019s
W20260709 12:27:56.558207  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 640126us
I20260709 12:27:56.562364  7654 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:56.578784  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:56.543740  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 251103us
W20260709 12:27:56.540846  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.005s	sys 0.021s
W20260709 12:27:56.585886  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.006s	sys 0.022s
I20260709 12:27:56.530387  7728 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:27:56.531793  7658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.020s	sys 0.009s
W20260709 12:27:56.594215  7658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.020s	sys 0.009s
I20260709 12:27:56.596263  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:56.517143 13801 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.601991  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:56.610737 13786 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:56.614284 13896 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:56.615155 13896 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:56.514914 13685 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.619760  7737 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:56.609743  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:56.619798  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:56.623504  7662 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.616523 13896 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.655042  7809 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:56.627815 13918 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.630106 13907 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.631067 13685 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.634389 13801 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.598780 13305 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.636598 13903 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:56.643713  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 976) took 1248 ms. Trace:
I20260709 12:27:56.645375  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.646186  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.625571 13846 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.649115 13905 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:56.650976 13751 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:56.651929  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:27:56.787027  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136898us
I20260709 12:27:56.789942  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:56.656831  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.791760  7803 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:56.793390  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.664544  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:56.664404  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.795908  8409 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:56.607539 13800 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.609110 13902 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.675799  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.683459  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.683197  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.696946  7810 rpcz_store.cc:276] 0709 12:27:55.394941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.413450 (+ 18509us) service_pool.cc:224] Handling call
0709 12:27:56.318054 (+904604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.351904 (+ 33850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.643429 (+291525us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54003732d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.643618 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049382784}
I20260709 12:27:56.809015  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:56.700512  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:56.625927 13837 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.681561  7660 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:27:56.666887 13757 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:27:56.812927 13685 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:56.752278  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:56.666453 13918 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:56.696717 13903 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.721753  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.796726  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.794895  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:56.800155  7662 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
W20260709 12:27:56.698067  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 269021us
I20260709 12:27:56.839749  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.842095  7803 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.810214  7807 raft_consensus.cc:2376] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14. Re-sending same reply.
I20260709 12:27:56.811489 13661 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:27:56.811753 13908 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:27:56.835690  7728 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:56.813055  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:56.717689 13305 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.819597  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:56.819502 13766 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:56.848495  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:56.850180 13838 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:56.850386 13773 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.008s
I20260709 12:27:56.851791 13837 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.847529  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:56.895402  7662 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:56.838711  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:56.843855  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:56.784559 13356 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.827333 13786 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:56.824965 13895 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:56.813747 13846 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.864058 13751 consensus_queue.cc:1243] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:56.897506  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.820425 13797 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:56.842037 13323 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.545s	user 0.015s	sys 0.006s
I20260709 12:27:56.902098  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2672) took 1044 ms. Trace:
I20260709 12:27:56.868485  8409 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:27:56.861083  7804 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:56.837282 13874 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:56.870325 13893 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:56.908458 13786 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:56.914510  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.910405 13356 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:56.919409  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:56.836545 13918 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:56.921905  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1268) took 1940 ms (client timeout 1765 ms). Trace:
W20260709 12:27:56.923658  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2983) took 1299 ms (client timeout 1717 ms). Trace:
I20260709 12:27:56.924305  8407 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:56.992870  7729 rpcz_store.cc:269] 0709 12:27:55.624631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.625314 (+   683us) service_pool.cc:224] Handling call
0709 12:27:56.923462 (+1298148us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b5400419454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.923588 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586664320}
I20260709 12:27:56.938215 13481 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.894613 13907 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:56.875705 13685 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:56.939162  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:56.905117 13773 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:56.945194  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:56.822932 13902 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.946177 13797 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:56.836877 13903 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:56.934825 13895 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.930131 13881 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:56.994899  8413 rpcz_store.cc:269] 0709 12:27:54.981560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.981777 (+   217us) service_pool.cc:224] Handling call
0709 12:27:56.921649 (+1939872us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:56.921825 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3045544192}
I20260709 12:27:56.950515  7653 rpcz_store.cc:276] 0709 12:27:55.857712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.860562 (+  2850us) service_pool.cc:224] Handling call
0709 12:27:55.861085 (+   523us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:56.019731 (+158646us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:56.141000 (+121269us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.425266 (+284266us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:56.460359 (+ 35093us) log.cc:844] Serialized 52 byte log entry
0709 12:27:56.460575 (+   216us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:56.807952 (+347377us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.824611 (+ 16659us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b4800080358. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:27:56.883832 (+ 59221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.883937 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.901795 (+ 17858us) raft_consensus.cc:1685] finished
0709 12:27:56.901828 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.902016 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54339,"prepare.queue_time_us":140887,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":88,"raft.queue_time_us":112519,"raft.run_cpu_time_us":12080,"raft.run_wall_time_us":922080,"spinlock_wait_cycles":6166400,"thread_start_us":764319,"threads_started":4,"wal-append.queue_time_us":474256}
I20260709 12:27:57.042768  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:57.043833  7657 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
W20260709 12:27:56.950393 13731 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.005s
W20260709 12:27:57.019078 13824 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.652s	user 0.005s	sys 0.014s
I20260709 12:27:57.005100 13864 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:56.949345 13751 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:56.909565 13897 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:57.020066  7811 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
W20260709 12:27:57.024726  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 374895us
W20260709 12:27:57.029323  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 357281us
I20260709 12:27:57.001252 13786 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:56.960840  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:56.989631  7803 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:27:57.065860  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:57.002106 13656 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.043694  7653 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:27:57.067759 13656 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.068733  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:57.067070  7803 raft_consensus.cc:2368] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15.
W20260709 12:27:57.058192  7656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.039s	sys 0.049s
W20260709 12:27:57.070659  7656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.040s	sys 0.049s
I20260709 12:27:57.058701  7735 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:27:57.071808  7735 pending_rounds.cc:85] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:27:57.072121  7735 pending_rounds.cc:107] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 17.11
W20260709 12:27:57.071928  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 258942us
I20260709 12:27:57.059759  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:57.028937 13809 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:56.964277 13323 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:56.959789 13712 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.072661  7735 raft_consensus.cc:2889] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:57.075433  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.080638  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:57.040027 13800 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.999166 13893 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.054257 13751 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:57.012598 13481 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.081753  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.109359  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.082190  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:57.111961  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:57.112955  8414 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.084514  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.043350 13907 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:57.085276  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 124538us
I20260709 12:27:57.086143  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.115906  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.116240  7808 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:27:57.086885  8379 leader_election.cc:336] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.541s (SENT)
I20260709 12:27:57.087510  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:57.117058  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.117687  8413 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.069940 13881 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:57.106053  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.080914 13877 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:57.120213  8379 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.578s (SENT)
I20260709 12:27:57.108801  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:57.078946  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:57.130203  7803 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:57.118057  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:57.136906  8379 leader_election.cc:400] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17.
I20260709 12:27:57.137259  8379 leader_election.cc:403] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:57.086342 13874 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:27:57.089322 13712 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.125501 13907 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.160634  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:57.104511 13800 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.145396  7702 leader_election.cc:400] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15.
I20260709 12:27:57.166332  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.178717  7702 leader_election.cc:403] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:27:57.178987  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 70288.2us
I20260709 12:27:57.166692  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:57.149928  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2973) took 2731 ms. Trace:
I20260709 12:27:57.061759 13897 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.177592  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:57.179602  7729 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:27:57.189543  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.140383  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.196833  7803 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:57.198163  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.119757 13809 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:57.199059  7803 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:57.179986  7654 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:27:57.180163  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101299us
W20260709 12:27:57.177091  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 165948us
I20260709 12:27:57.192744  7653 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.179989  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:57.150122  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68120.8us
I20260709 12:27:57.206255  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:57.132699 13824 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:57.152406 13934 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 12:27:57.139489 13943 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:27:57.151348 13938 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:27:57.220366  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.187134  7731 rpcz_store.cc:276] 0709 12:27:54.418786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.421621 (+  2835us) service_pool.cc:224] Handling call
0709 12:27:54.422081 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.431658 (+  9577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.431712 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.025730 (+1594018us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b18001ffd80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:27:56.521847 (+496117us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:27:56.523946 (+  2099us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.999943 (+475997us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:56.999989 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.000074 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.149224 (+149150us) raft_consensus.cc:1685] finished
0709 12:27:57.149299 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.149634 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2786,"raft.queue_time_us":3043,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5289,"spinlock_wait_cycles":25008768,"thread_start_us":474515,"threads_started":1,"wal-append.queue_time_us":623088}
I20260709 12:27:57.221855  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:27:57.166973 13924 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.847s	user 0.006s	sys 0.000s
I20260709 12:27:57.107788 13685 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.171725 13948 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17.
I20260709 12:27:57.229056 13751 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.168524 13846 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.229085  7805 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:57.137740 13955 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:57.155927 13901 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.054s	user 0.003s	sys 0.007s
I20260709 12:27:57.280977  7653 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:57.153644 13860 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.232432  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3101) took 1857 ms. Trace:
I20260709 12:27:57.252615  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2579) took 1349 ms. Trace:
I20260709 12:27:57.143541 13801 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:57.143463 13950 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:57.153987 13952 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:57.144665 13864 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 12:27:57.308957 13529 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:57.315115 13529 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:57.315446  7660 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:57.316145  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2676) took 1457 ms. Trace:
W20260709 12:27:57.155838 13942 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:27:57.238783  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:57.137856 13881 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.157959 13323 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:57.233701 13837 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.147338 13963 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 12:27:57.191932 13792 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:57.168900 13951 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 12:27:57.319487 13801 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.329411  7729 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:27:57.257494 13751 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.330036 13801 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.312561 13950 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.325341 13881 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:57.331463 13950 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:57.220363 13776 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:57.174718 13877 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.177124 13773 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.346562 13773 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.348138  8406 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:57.349768 13773 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.231488 13917 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:57.299057  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 119130us
W20260709 12:27:57.256397  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.227618  8413 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:27:57.227650 13826 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.228657 13356 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.300690  7812 rpcz_store.cc:276] 0709 12:27:55.375319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.377344 (+  2025us) service_pool.cc:224] Handling call
0709 12:27:55.558905 (+181561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.620996 (+ 62091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.232198 (+1611202us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400296650. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.232363 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416796032}
I20260709 12:27:57.301028  7806 rpcz_store.cc:276] 0709 12:27:55.903669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.905534 (+  1865us) service_pool.cc:224] Handling call
0709 12:27:55.905964 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.916240 (+ 10276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.916276 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.923593 (+  7317us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.289307 (+365714us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:56.289359 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.289507 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.962898 (+673391us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:27:57.252263 (+289365us) raft_consensus.cc:1685] finished
0709 12:27:57.252290 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.252539 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12746,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11457,"spinlock_wait_cycles":26973440,"thread_start_us":268512,"threads_started":1,"wal-append.queue_time_us":348261}
I20260709 12:27:57.255854 13907 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.255910 13893 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:57.262547 13632 consensus_queue.cc:1175] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 16 status { last_received { term: 15 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:57.258019 13712 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.260030 13800 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.317488  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:57.316788  7656 rpcz_store.cc:276] 0709 12:27:55.859119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.381334 (+522215us) service_pool.cc:224] Handling call
0709 12:27:56.381664 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.423346 (+ 41682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.423381 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.429998 (+  6617us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.526476 (+ 96478us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b180045ac00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:57.071348 (+544872us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:57.071407 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.071488 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.315823 (+244335us) raft_consensus.cc:1685] finished
0709 12:27:57.315846 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.316059 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":591,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":15055,"spinlock_wait_cycles":10291072,"thread_start_us":544386,"threads_started":1,"wal-append.queue_time_us":830606}
I20260709 12:27:57.305576  8414 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:57.244122 13949 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.363268 13949 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.365860 13305 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.366334 13949 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.322381 13529 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.304306 13908 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:57.325137 13918 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.327046 13897 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.245050 13902 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.310115 13922 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:57.368515 13922 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:27:57.368607  7736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 2993) took 1606 ms (client timeout 2084 ms). Trace:
I20260709 12:27:57.368924  7734 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:57.369571 13922 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.311782 13860 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:57.371207 13860 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:57.316393 13832 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.773s	user 0.013s	sys 0.000s
I20260709 12:27:57.243209 13481 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.164384 13895 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.351769  7804 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:27:57.352499  7654 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:57.371704 13860 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.246358 13642 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.356037 13776 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:57.314539 13903 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:57.369413  7736 rpcz_store.cc:269] 0709 12:27:55.762647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.769724 (+  7077us) service_pool.cc:224] Handling call
0709 12:27:57.368407 (+1598683us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54001c5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.368549 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246706432}
I20260709 12:27:57.319907 13864 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.345743 13877 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.371870  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3008) took 1280 ms. Trace:
I20260709 12:27:57.396488  7734 rpcz_store.cc:276] 0709 12:27:56.091199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.305639 (+214440us) service_pool.cc:224] Handling call
0709 12:27:57.368562 (+1062923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.369504 (+   942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.371015 (+  1511us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001c5cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.371173 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375001472}
I20260709 12:27:57.399145 13776 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.400866 13656 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.404397  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3006) took 1313 ms. Trace:
I20260709 12:27:57.406038  7732 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:27:57.405030  7730 rpcz_store.cc:276] 0709 12:27:56.090590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.096207 (+  5617us) service_pool.cc:224] Handling call
0709 12:27:57.404127 (+1307920us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.404307 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391035136}
I20260709 12:27:57.388233  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2785) took 1456 ms. Trace:
I20260709 12:27:57.409011  7655 rpcz_store.cc:276] 0709 12:27:55.932780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.466963 (+534183us) service_pool.cc:224] Handling call
0709 12:27:56.468983 (+  2020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.469317 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:56.503052 (+ 33735us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.385250 (+882198us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b180025fba0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:27:57.386885 (+  1635us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.387584 (+   699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:57.387706 (+   122us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:57.387790 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.387916 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.388132 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23502,"prepare.queue_time_us":33918,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":124,"raft.queue_time_us":29486,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":30716,"spinlock_wait_cycles":9915392,"thread_start_us":494410,"threads_started":2,"wal-append.queue_time_us":460742}
I20260709 12:27:57.430321  7728 raft_consensus.cc:2368] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16.
I20260709 12:27:57.430948  7730 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:57.359529 13826 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.464066 13880 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:57.476409  7729 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:57.480378  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 2994) took 1717 ms. Trace:
I20260709 12:27:57.493227  7735 rpcz_store.cc:276] 0709 12:27:55.763047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.771347 (+  8300us) service_pool.cc:224] Handling call
0709 12:27:55.771777 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.080538 (+1308761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.479968 (+399430us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001c57d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:57.480276 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958423808}
I20260709 12:27:57.483970 13903 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.485370 13688 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.487092  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3010) took 1391 ms. Trace:
I20260709 12:27:57.498981  7732 rpcz_store.cc:276] 0709 12:27:56.097272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.312660 (+215388us) service_pool.cc:224] Handling call
0709 12:27:57.405519 (+1092859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.406601 (+  1082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.486741 (+ 80140us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540007f8d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.486989 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512192256}
I20260709 12:27:57.489194 13826 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.405170 13864 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.498781 13832 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.502562 13832 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.504248 13864 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.505048 13832 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.505587  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1274) took 1718 ms. Trace:
I20260709 12:27:57.499580 13797 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.507787 13797 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:57.506287  8407 rpcz_store.cc:276] 0709 12:27:55.787474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.788450 (+   976us) service_pool.cc:224] Handling call
0709 12:27:56.924089 (+1135639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.997490 (+ 73401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.505243 (+507753us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540059ecd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.505485 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508507008}
I20260709 12:27:57.490574 13922 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:57.564891  7658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.011s	sys 0.038s
W20260709 12:27:57.567829  7658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.038s
W20260709 12:27:57.572788  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 376816us
I20260709 12:27:57.571382  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:57.574472  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:57.567094 13719 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:57.602890 13786 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.006s
I20260709 12:27:57.598764 13796 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:57.609972  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.592116 13714 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:27:57.626312  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 477478us
I20260709 12:27:57.605187 13908 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:57.626655  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:57.630448  7733 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.608035 13947 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:57.565541 13950 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.628192  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.637266  7730 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:57.628372 13809 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:57.646541  7730 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:57.625422 13751 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:57.607887 13763 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:27:57.615113 13731 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:57.668215  7731 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:57.624398 13712 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:27:57.654280  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:57.657815  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.652231  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.674877  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:27:57.629135 13826 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:57.664878  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.678364  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:57.678934  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 533817us
I20260709 12:27:57.667510  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.675658  7659 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:57.615060 13889 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:27:57.621254 13786 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.686311  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.678992  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:57.656147 13824 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:57.651073 13947 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:57.627585 13796 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.696517  7700 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:57.679946  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:57.654105  7733 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:27:57.708849  7735 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:57.677716 13751 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:27:57.667521 13763 consensus_queue.cc:1243] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:57.698063  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:57.713898  7775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.223s (SENT) [suppressed 11 similar messages]
I20260709 12:27:57.670548 13777 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.672667 13895 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:57.706513  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.722378  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:57.706646 13967 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:27:57.731550  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:57.732800  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.728710  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.687019 13860 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.736469  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:57.735599  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:27:57.736658  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.013s	sys 0.027s
I20260709 12:27:57.729570 13824 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.733017 13685 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:27:57.745364 13896 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:57.764921 13973 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:57.765162 13896 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.765558 13973 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.765839 13896 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.766081 13973 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.767561  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.768962  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.770205  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.689510 13826 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.733214 13982 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:57.776440 13860 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:57.778223 13968 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:27:57.780514 13950 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.748878 13749 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:57.782202 13749 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.782734 13947 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.735643 13895 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.791010 13978 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:57.752691  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.017s	sys 0.027s
I20260709 12:27:57.792122 13926 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:27:57.792436  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 586297us
I20260709 12:27:57.792935 13926 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.793913  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.796041 13918 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:57.754922  8411 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:27:57.798179  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.804394  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 999) took 1103 ms. Trace:
I20260709 12:27:57.758188  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:57.668859 13809 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.728955 13889 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:57.764458 13984 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:27:57.790871  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54534.3us
I20260709 12:27:57.754241  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.806306 13826 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.781273 13786 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:57.806905  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71386.6us
I20260709 12:27:57.808641 13832 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:57.810710  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3003) took 1248 ms. Trace:
I20260709 12:27:57.878540  7731 rpcz_store.cc:276] 0709 12:27:56.562655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.273068 (+710413us) service_pool.cc:224] Handling call
0709 12:27:57.273566 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.668797 (+395231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.810411 (+141614us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54002aba54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:57.810636 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907108608}
I20260709 12:27:57.723605 13903 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:57.812599  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2674) took 1954 ms. Trace:
I20260709 12:27:57.816823 13977 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.760504 13642 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15.
I20260709 12:27:57.859656  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:57.798933  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:57.881384  7658 rpcz_store.cc:276] 0709 12:27:55.858457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.859643 (+  1186us) service_pool.cc:224] Handling call
0709 12:27:55.860174 (+   531us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:56.594888 (+734714us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:56.882519 (+287631us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.569644 (+687125us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:57.576093 (+  6449us) log.cc:844] Serialized 78 byte log entry
0709 12:27:57.576497 (+   404us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:57.576945 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.577356 (+   411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.577486 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.812269 (+234783us) raft_consensus.cc:1685] finished
0709 12:27:57.812293 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.812508 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44582,"prepare.queue_time_us":176067,"prepare.run_cpu_time_us":350,"prepare.run_wall_time_us":2725,"raft.queue_time_us":39919,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":28101,"spinlock_wait_cycles":4654976,"thread_start_us":1589842,"threads_started":3,"wal-append.queue_time_us":762763}
I20260709 12:27:57.799923 13751 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.860581 13982 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:27:57.872378  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 114278us
I20260709 12:27:57.872486  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.920447  8407 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:57.873826  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:57.878023  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149395us
W20260709 12:27:57.860306  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.867856  7807 rpcz_store.cc:276] 0709 12:27:56.701306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.861800 (+160494us) service_pool.cc:224] Handling call
0709 12:27:56.862327 (+   527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:57.052889 (+190562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.052942 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:57.076196 (+ 23254us) log.cc:844] Serialized 52 byte log entry
0709 12:27:57.405243 (+329047us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:57.464856 (+ 59613us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.465509 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.465625 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.804002 (+338377us) raft_consensus.cc:1685] finished
0709 12:27:57.804037 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.804281 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58015,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":69864,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":3662,"thread_start_us":463966,"threads_started":2,"wal-append.queue_time_us":500316}
I20260709 12:27:57.871990 13777 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.924989  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:57.926643  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.927403  7735 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:57.928467  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.904906 13980 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:57.905112 13656 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:57.904959 13977 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:57.881124 13903 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.910043  7730 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:27:57.910601  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3008) took 1288 ms. Trace:
W20260709 12:27:57.911406  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51862.9us
I20260709 12:27:57.872570 13846 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:57.913720  7731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:27:57.873023 13889 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.873204 13749 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:57.929438  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.929726  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.929641  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.930604  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:57.930495  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:57.932628  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58926.7us
I20260709 12:27:57.887739 13897 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:57.934703  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.931653  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:57.934203 13985 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:58.049321  7729 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:27:57.936014  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:58.047405  7807 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:57.916275 13860 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:57.935799  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:57.948094  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:57.953576  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.954826  7731 raft_consensus.cc:606] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:27:58.068617  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 132678us
I20260709 12:27:58.087397  7731 raft_consensus.cc:614] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:57.957530  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1002) took 1202 ms. Trace:
I20260709 12:27:58.046370  7733 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:27:58.088354  7811 rpcz_store.cc:276] 0709 12:27:56.754962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.843537 (+ 88575us) service_pool.cc:224] Handling call
0709 12:27:57.957350 (+1113813us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:57.957464 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283374336}
I20260709 12:27:58.089367  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:58.090277  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.910604 13895 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.069343  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.090596  7733 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:57.940711  7734 rpcz_store.cc:276] 0709 12:27:56.622263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.413496 (+791233us) service_pool.cc:224] Handling call
0709 12:27:57.436064 (+ 22568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.453934 (+ 17870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.453987 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.454740 (+   753us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.733747 (+279007us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:57.733798 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.733893 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.910346 (+176453us) raft_consensus.cc:1685] finished
0709 12:27:57.910370 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.910532 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11233,"raft.queue_time_us":18210,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":14184,"thread_start_us":235204,"threads_started":1,"wal-append.queue_time_us":436984}
I20260709 12:27:58.070640  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:58.085933  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 150224us
I20260709 12:27:58.098038  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:27:58.086464  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 138514us
I20260709 12:27:58.099494  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:27:58.099967  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.932456 13926 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:57.925218 13823 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:58.097004  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:27:58.108541  7811 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:27:57.935494 13799 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:58.119400  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:57.938912 13832 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 12:27:58.069046 13949 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:27:58.128326  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.954779 13903 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.954522 13846 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:58.132145  7658 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:27:58.047302 13826 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:58.133060  7656 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:58.093349 13864 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:58.134191  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.066895 13935 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:57.932763 13982 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.095475 13881 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:58.075358 13889 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.142783 13881 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:58.143435 13881 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:58.144088 13982 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.144491 13881 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.105026 13712 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:57.936210 13777 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.093436 13897 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:27:57.960343 13907 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.162s
I20260709 12:27:58.139321  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:58.152745  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.160324  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:58.161629  7658 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:58.117312 13985 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:58.132606 13925 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:58.068032 13981 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.207s
I20260709 12:27:58.139108 13786 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:58.175585  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:27:58.179071  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:58.180413  7653 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:27:58.169790  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:58.183163  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:27:58.123368 13824 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:27:58.132208 13799 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.202308  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.130249 13823 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:57.959662 13980 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.168079 13712 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:58.172418 13925 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.206518  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:27:58.211112  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:58.305802 13987 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 12:27:58.478871  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 373174us
W20260709 12:27:58.485594  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 386320us
I20260709 12:27:58.214136 13777 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:58.488580  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.492321  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 285794us
I20260709 12:27:58.179868 13786 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.496003 13786 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:58.212597 13797 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.006s
I20260709 12:27:58.216430 13712 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:58.327884 13973 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.561s	user 0.016s	sys 0.000s
I20260709 12:27:58.503959 13973 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.504395  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:27:58.173650 13749 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:27:58.154649 13826 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.204252 13832 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:58.509243 13832 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.510047 13832 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:58.257275 13838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.024s	sys 0.006s
I20260709 12:27:58.219820 13823 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.519785  7655 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:58.520165  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 467822us
I20260709 12:27:58.291247 13799 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.197582 13846 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.332929 13881 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.523512 13838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.025s	sys 0.006s
I20260709 12:27:58.524058 13881 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.525123 13950 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:58.344971 13889 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.255721 13977 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.240109 13897 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.544533  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:27:58.545574 13978 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:58.355669 13903 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.547731 13918 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:58.548367 13918 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:58.549135 13918 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.549562 13903 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:58.212822 13922 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.689s	user 0.015s	sys 0.000s
I20260709 12:27:58.157575 13935 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.552762 13826 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.136963 13949 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.210294 13925 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.196269 13930 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:58.563930 13930 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:58.564394  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3002) took 2002 ms. Trace:
I20260709 12:27:58.564719 13930 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:58.565068  7728 rpcz_store.cc:276] 0709 12:27:56.561830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.912380 (+350550us) service_pool.cc:224] Handling call
0709 12:27:58.563791 (+1651411us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:58.564082 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950816512}
I20260709 12:27:58.192426 13959 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:58.511317 13973 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.570717 13959 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.575412  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.179162 13985 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.564774 13881 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:58.580736 13889 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.582382 13985 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.584623 14003 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.585529 14003 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.215524 13980 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.587664 13995 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.587982 13949 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.554399 13935 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:27:58.455700  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 357584us
I20260709 12:27:58.452527 13998 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:58.519049  7732 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:58.518186  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2793) took 1661 ms (client timeout 1641 ms). Trace:
W20260709 12:27:58.321854 13896 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.004s
W20260709 12:27:58.595206  7657 rpcz_store.cc:269] 0709 12:27:56.856909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.910117 (+ 53208us) service_pool.cc:224] Handling call
0709 12:27:58.516754 (+1606637us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:58.517862 (+  1108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79400320}
I20260709 12:27:58.527187  7812 raft_consensus.cc:2368] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:27:58.596695 13896 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.598052  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2587) took 1538 ms. Trace:
I20260709 12:27:58.525936 13997 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.473683  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:58.598861 13777 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:58.599721  7811 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:58.599135 13997 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.536085 13846 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:58.588655 13995 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:27:58.607664  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1272) took 2826 ms (client timeout 2223 ms). Trace:
I20260709 12:27:58.590286 13998 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:58.608222  8406 rpcz_store.cc:269] 0709 12:27:55.780980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.783330 (+  2350us) service_pool.cc:224] Handling call
0709 12:27:58.607081 (+2823751us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:58.607368 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3596271488}
I20260709 12:27:58.610042  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.610836 13998 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:27:58.612846  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68750.8us
I20260709 12:27:58.614758  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.598580  7804 rpcz_store.cc:276] 0709 12:27:57.059493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.104648 (+ 45155us) service_pool.cc:224] Handling call
0709 12:27:57.107623 (+  2975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.382248 (+274625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.597477 (+1215229us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400067cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:58.597853 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560057600}
W20260709 12:27:58.623415  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 136362us
I20260709 12:27:58.574929 13959 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:58.654688  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:58.655934  7805 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.003s	sys 0.043s
W20260709 12:27:58.658952  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.662343 13922 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:58.663147 13922 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.668262 13356 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:58.668833 13356 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:58.673552  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.676162  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.679538  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.678043 13949 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:27:58.681594  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.655813  7804 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:27:58.684211  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.675505 13935 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.687970 13935 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.684943 14003 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:58.694226 13997 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.695735  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3005) took 2130 ms. Trace:
I20260709 12:27:58.695942  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:58.697154 13985 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:58.696300  7736 rpcz_store.cc:276] 0709 12:27:56.564889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.413610 (+848721us) service_pool.cc:224] Handling call
0709 12:27:57.414361 (+   751us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:57.428385 (+ 14024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.428419 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:57.436067 (+  7648us) log.cc:844] Serialized 52 byte log entry
0709 12:27:58.578867 (+1142800us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800394ae0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:27:58.676711 (+ 97844us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:58.676808 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.676953 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.695112 (+ 18159us) raft_consensus.cc:1685] finished
0709 12:27:58.695135 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.695487 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2404,"raft.queue_time_us":18804,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":10507,"spinlock_wait_cycles":5685888,"thread_start_us":44902,"threads_started":1,"wal-append.queue_time_us":1198364}
I20260709 12:27:58.670433 13356 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.685930  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.680562  8409 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:58.682862 13995 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:58.678365  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.666834 13922 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.699589 13985 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:58.716463  7811 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:27:58.717190  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.725517 14016 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:58.727262 14018 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:58.737735 13997 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.739049  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.738633 13997 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:27:58.751770  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56007.4us
I20260709 12:27:58.754462  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:27:58.755358  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.760501  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:27:58.761494  7804 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:58.764358 14010 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:27:58.719024 13985 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:58.765856  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:58.773870 13777 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.774811 13777 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.775566 13985 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:58.779841  7734 pending_rounds.cc:85] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:27:58.780411  7734 pending_rounds.cc:107] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:27:58.780813  7734 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:58.795763  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:27:58.758675 13896 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:58.756740 13832 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.796062  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:58.822127 13995 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.796078  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:58.923400  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.049176 13954 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.726s	user 0.002s	sys 0.002s
I20260709 12:27:58.799826  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:58.852259  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.877121  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:58.955340  7732 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:59.062698  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:59.066215  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 311911us
I20260709 12:27:58.847793 13656 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:58.820394  8413 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 10.
I20260709 12:27:59.072204  7804 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:27:58.851886 13935 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.074736  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:27:58.967342 13930 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:59.071615 13656 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:27:59.081097  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 347561us
I20260709 12:27:59.058991 13995 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.987005 13786 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:59.085659 13995 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.086272 13786 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:59.086639 13995 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:58.922624 14011 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:58.851855 14016 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.825918 13949 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:59.082235 13656 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.913960 13775 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.088078 13995 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:59.110998 13481 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:59.111503 13995 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:59.095867  7811 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:27:59.098258  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1116) took 1419 ms. Trace:
I20260709 12:27:59.100360  7808 raft_consensus.cc:2368] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16.
I20260709 12:27:58.832365 14019 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:59.099017 14016 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.102358 13949 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:59.127310 13949 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:59.129850  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.102682 13935 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:27:59.131878 13935 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:59.133466  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1132) took 1478 ms. Trace:
I20260709 12:27:59.134069  7655 rpcz_store.cc:276] 0709 12:27:57.655189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.655436 (+   247us) service_pool.cc:224] Handling call
0709 12:27:58.519098 (+863662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.520574 (+  1476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.133141 (+612567us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540028da50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:59.133340 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946962816}
I20260709 12:27:59.090256 14011 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:59.135787 13776 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:59.136682 14011 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:59.137048 13776 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.138629 14011 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:59.110642 13766 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:27:59.082984 13930 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:27:59.139631 13766 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.142771 13766 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:59.145824 14016 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:27:59.081220  7806 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:27:59.113178  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2599) took 1179 ms. Trace:
I20260709 12:27:59.110855 13656 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:59.113189  8413 rpcz_store.cc:276] 0709 12:27:57.678318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.678736 (+   418us) service_pool.cc:224] Handling call
0709 12:27:59.097922 (+1419186us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:59.098150 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944904960}
I20260709 12:27:59.137580  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3001) took 2577 ms. Trace:
I20260709 12:27:59.150395  7737 rpcz_store.cc:276] 0709 12:27:56.560069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.560304 (+   235us) service_pool.cc:224] Handling call
0709 12:27:56.560728 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.622393 (+ 61665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.136793 (+2514400us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b54002f1554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:27:59.137288 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135173504}
I20260709 12:27:59.152297  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:27:59.116918 14019 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:27:59.154116 13996 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:59.155453 14019 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.154834 13996 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:27:59.157789 14019 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:59.159161 13996 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.162735 13775 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:59.163422  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.122262  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2596) took 1193 ms. Trace:
I20260709 12:27:59.148701  7811 rpcz_store.cc:276] 0709 12:27:57.933338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.747959 (+814621us) service_pool.cc:224] Handling call
0709 12:27:59.113115 (+365156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.913949 13918 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:58.920095 13925 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:58.868281 13981 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:59.163776 13775 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.709093  7735 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:27:59.636000  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.675480  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.011s	sys 0.019s
I20260709 12:27:59.331763  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:59.474727  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:27:59.285701  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.190501  8412 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:27:59.165277  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2595) took 1237 ms (client timeout 1593 ms). Trace:
I20260709 12:27:59.721728  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:59.761142  7806 rpcz_store.cc:269] 0709 12:27:57.928182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.929101 (+   919us) service_pool.cc:224] Handling call
0709 12:27:59.148168 (+1219067us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:59.148356 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314147712}
I20260709 12:27:59.760928  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:59.311993  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:27:59.764122  8413 raft_consensus.cc:606] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:27:59.764751  8413 raft_consensus.cc:614] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:59.764798  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:59.760563 13656 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
W20260709 12:27:59.264249  7778 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.717s (SENT)
W20260709 12:27:59.766928  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 518090us
I20260709 12:27:59.341845  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:59.635429  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:59.189468  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:27:59.760622  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.759198  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.011s	sys 0.023s
I20260709 12:27:59.187736  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:59.106371 13846 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:59.635459 13481 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:27:59.179986  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:27:59.349495  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:27:59.845994  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.189149 13529 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:59.164367  7812 rpcz_store.cc:276] 0709 12:27:57.928572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.928736 (+   164us) service_pool.cc:224] Handling call
0709 12:27:59.121551 (+1192815us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:27:59.121993 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011577600}
I20260709 12:27:59.842387  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3027) took 1233 ms. Trace:
I20260709 12:27:59.770422  7806 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:27:59.867182  8406 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 10.
I20260709 12:27:59.134940 13935 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.274014 13719 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.870563 13719 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.873853 13719 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.284662  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:27:59.880601  7729 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:59.907675 13981 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.232584 13985 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.933287  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2618) took 1035 ms. Trace:
I20260709 12:27:59.934198  7806 rpcz_store.cc:276] 0709 12:27:58.831066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.764470 (+933404us) service_pool.cc:224] Handling call
0709 12:27:59.866357 (+101887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.937258  8411 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:27:59.938437  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3129) took 1909 ms. Trace:
I20260709 12:27:59.189366 13930 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.940945  7804 rpcz_store.cc:276] 0709 12:27:57.957233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.115421 (+1158188us) service_pool.cc:224] Handling call
0709 12:27:59.866765 (+751344us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.939414  7702 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.022s	sys 0.059s
I20260709 12:27:59.849852 13776 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:27:59.941936  7702 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.023s	sys 0.059s
W20260709 12:27:59.945629  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 854444us
I20260709 12:27:59.945766  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:27:59.944473  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.939275  8380 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.036s	sys 0.048s
I20260709 12:27:59.946444  7811 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
W20260709 12:27:59.946694  8380 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.036s	sys 0.048s
W20260709 12:27:59.947121  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 840247us
W20260709 12:27:59.860230  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.942334  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:59.952477  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:27:59.953665  8413 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:27:59.863931 13766 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:27:59.864749  7734 rpcz_store.cc:276] 0709 12:27:58.609083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.644081 (+ 34998us) service_pool.cc:224] Handling call
0709 12:27:58.745663 (+101582us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:58.781955 (+ 36292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:58.781997 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:58.782865 (+   868us) log.cc:844] Serialized 52 byte log entry
0709 12:27:58.920615 (+137750us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b180040dec0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:27:59.116340 (+195725us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:59.116392 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.116475 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.474526 (+358051us) raft_consensus.cc:1685] finished
0709 12:27:59.474555 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.842249 (+367694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157004,"raft.queue_time_us":4443,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":246,"spinlock_wait_cycles":11310976,"thread_start_us":328514,"threads_started":1,"wal-append.queue_time_us":499977}
I20260709 12:27:59.965131  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:59.958617  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:27:59.972339  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2610) took 1294 ms. Trace:
I20260709 12:27:59.973027  7806 rpcz_store.cc:276] 0709 12:27:58.678359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.940225 (+1261866us) service_pool.cc:224] Handling call
0709 12:27:59.972216 (+ 31991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.953222 13832 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:27:59.983352  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:27:59.984266  7804 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:27:59.984768  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:27:59.944888 13846 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:27:59.993891 13656 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:59.980604  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:00.000677  7812 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:27:59.993106 14019 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.835s	user 0.016s	sys 0.001s
I20260709 12:28:00.012590  7655 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:28:00.007108  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.014564  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:00.016549  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:00.018045  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.014519  7657 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:00.017159  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:00.020107  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 545649us
I20260709 12:28:00.020596  7804 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:59.975365 13481 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.989131 13766 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:27:59.997711 13930 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.033762  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3122) took 2083 ms. Trace:
I20260709 12:28:00.034288  7812 rpcz_store.cc:276] 0709 12:27:57.950398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.016178 (+2065780us) service_pool.cc:224] Handling call
0709 12:28:00.033125 (+ 16947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.033357 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.033389 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.033454 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:00.033488 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.033544 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.033664 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.035327  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.035351  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.036504  7661 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:00.037990  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.043489  7659 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:00.043298  8408 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:00.038405 14024 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:00.050633  7654 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:00.052831  7658 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:00.051668  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.052980  8407 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:28:00.036538  7662 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:00.044960  7810 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:00.061709  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.064066 13775 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.065006 13935 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.028697 13981 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.061292  8414 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:28:00.043897 13995 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.932s	user 0.011s	sys 0.007s
I20260709 12:28:00.040050 13925 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.042937 13930 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.069303  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:00.068387  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.052135  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:00.076375 13918 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.052505 14024 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.098049 13832 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:00.099001 13832 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:00.100550 13832 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.096753  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:00.098129  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 996) took 3395 ms. Trace:
I20260709 12:28:00.108810  7812 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:28:00.096706  8412 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:28:00.113644 14027 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:28:00.115490 13846 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.116241 13356 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:00.120752  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:00.120925 13766 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.123401 14024 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.076920 13995 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.131070  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2602) took 2157 ms. Trace:
I20260709 12:28:00.131930 13656 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:00.132122 13481 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.139456  7731 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:00.141980  7812 rpcz_store.cc:276] 0709 12:27:57.952456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.079453 (+2126997us) service_pool.cc:224] Handling call
0709 12:28:00.105098 (+ 25645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.109252 (+  4154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.109491 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.145908  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3123) took 2179 ms. Trace:
I20260709 12:28:00.146535  7804 rpcz_store.cc:276] 0709 12:27:57.950708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.022007 (+2071299us) service_pool.cc:224] Handling call
0709 12:28:00.078018 (+ 56011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.129254 (+ 51236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.129285 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.129343 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:00.129376 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.129440 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.129565 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.130476  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 964) took 1057 ms. Trace:
W20260709 12:28:00.149134  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55567.1us
I20260709 12:28:00.157447  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.158411  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:00.160101  7731 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:00.163371 13846 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.164348 13846 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.165278 13995 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.166652  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:00.167413  7735 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.172297  7734 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:00.184177  7654 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:00.184757  7654 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:00.186218  7654 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:00.186928  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3020) took 2260 ms (client timeout 2200 ms). Trace:
W20260709 12:28:00.187470  7732 rpcz_store.cc:269] 0709 12:27:57.926929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.930296 (+  3367us) service_pool.cc:224] Handling call
0709 12:28:00.186591 (+2256295us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540039b2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.186820 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063835904}
W20260709 12:28:00.189504  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3120) took 2239 ms (client timeout 1582 ms). Trace:
W20260709 12:28:00.189939  7807 rpcz_store.cc:269] 0709 12:27:57.949770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.068968 (+119198us) service_pool.cc:224] Handling call
0709 12:28:00.189012 (+2120044us) spinlock_profiling.cc:243] Waited 79.9 ms on lock 0x7b54003723d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.189204 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183723264}
I20260709 12:28:00.191784  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.192859  7655 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:00.196324 13969 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 2.450s	user 0.026s	sys 0.010s
I20260709 12:28:00.199656  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1281) took 3071 ms. Trace:
I20260709 12:28:00.199802  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.200980  7662 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:00.179877 13656 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:00.202495 13656 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.108286  7809 rpcz_store.cc:276] 0709 12:27:56.700539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.848270 (+147731us) service_pool.cc:224] Handling call
0709 12:27:56.861428 (+ 13158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.082344 (+220916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.082375 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.082843 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.597048 (+514205us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:57.597111 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.599729 (+  2618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.704274 (+104545us) raft_consensus.cc:1685] finished
0709 12:27:57.704297 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.095446 (+2391149us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b54003e9ed0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.095800 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55853,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":8189,"spinlock_wait_cycles":29367552,"thread_start_us":640964,"threads_started":2,"wal-append.queue_time_us":611431,"wal-append.run_cpu_time_us":2383,"wal-append.run_wall_time_us":1067739}
W20260709 12:28:00.187923 13949 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.060s	user 0.009s	sys 0.007s
I20260709 12:28:00.149729  8409 rpcz_store.cc:276] 0709 12:27:59.088703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.100713 (+ 12010us) service_pool.cc:224] Handling call
0709 12:27:59.169934 (+ 69221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.285988 (+116054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.286085 (+    97us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.417787 (+131702us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.012923 (+595136us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:00.012995 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.013098 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:00.130052 (+116954us) raft_consensus.cc:1685] finished
0709 12:28:00.130088 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.130373 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5260,"raft.queue_time_us":24477,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":334532,"threads_started":1,"wal-append.queue_time_us":552518}
I20260709 12:28:00.213460  7654 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.107717 13935 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.217175  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.221509  7732 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:28:00.222581  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.214417 13481 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.231439 13481 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.243084  7735 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:00.246163  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.220255 13981 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.249647  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2609) took 2067 ms. Trace:
I20260709 12:28:00.252400 13995 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:00.253412 13995 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.254321  7731 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:00.254623 14009 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.200132  8415 rpcz_store.cc:276] 0709 12:27:57.128630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.128838 (+   208us) service_pool.cc:224] Handling call
0709 12:27:57.129446 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.133507 (+  4061us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:57.138975 (+  5468us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.542939 (+403964us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.545195 (+  2256us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.565527 (+ 20332us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:57.565885 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.566207 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.566315 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:00.195329 (+2629014us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:00.199207 (+  3878us) raft_consensus.cc:1685] finished
0709 12:28:00.199253 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.199444 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9549,"prepare.queue_time_us":6634,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":102,"raft.queue_time_us":5842,"raft.run_cpu_time_us":577,"raft.run_wall_time_us":11639,"spinlock_wait_cycles":8503552,"thread_start_us":402681,"threads_started":1,"wal-append.queue_time_us":557491}
I20260709 12:28:00.255384 14009 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.256399 13995 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.257710 13935 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.258323 13766 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.259084 13766 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.259549  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.260408  7662 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:00.264382  7803 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:00.265443 13786 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.269634 13786 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.270134 13985 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.271680  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.272680  7662 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:00.268996  7812 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:00.274120  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2605) took 2159 ms. Trace:
I20260709 12:28:00.274448  7812 rpcz_store.cc:276] 0709 12:27:58.114810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.210259 (+2095449us) service_pool.cc:224] Handling call
0709 12:28:00.250783 (+ 40524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.273850 (+ 23067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.274062 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.275004  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2619) took 1443 ms (client timeout 1556 ms). Trace:
I20260709 12:28:00.215188 13949 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:00.279394  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.275440  7808 rpcz_store.cc:269] 0709 12:27:58.831476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.062067 (+230591us) service_pool.cc:224] Handling call
0709 12:28:00.266031 (+1203964us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5400331654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.274938 (+  8907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48583936}
I20260709 12:28:00.279551  7804 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:00.262763 13766 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:00.284469  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.285496  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.243108 13997 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.247043 14024 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:00.285523  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3128) took 2323 ms. Trace:
W20260709 12:28:00.292861  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.251535  7809 rpcz_store.cc:276] 0709 12:27:58.182288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.213292 (+2031004us) service_pool.cc:224] Handling call
0709 12:28:00.243492 (+ 30200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.243700 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.243725 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.249302 (+  5577us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:00.249362 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.249433 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.249583 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.292861  7804 rpcz_store.cc:276] 0709 12:27:57.956964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.148881 (+2191917us) service_pool.cc:224] Handling call
0709 12:28:00.235127 (+ 86246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.280097 (+ 44970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.280325 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.294063  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.296486  7734 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:00.295957 13796 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.294795  8409 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:00.302556  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.303306  7731 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:28:00.303709  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:00.306473  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.305243  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.291575 13985 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.307439  7658 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:00.257716 14009 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.309930  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.316463  7805 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 10. Current term is 11. Ops: []
I20260709 12:28:00.317289  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2614) took 1637 ms. Trace:
I20260709 12:28:00.317589  7805 rpcz_store.cc:276] 0709 12:27:58.680311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.315856 (+1635545us) service_pool.cc:224] Handling call
0709 12:28:00.316286 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.316973 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.317207 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.291298 13981 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.321635  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1282) took 1037 ms. Trace:
W20260709 12:28:00.325434  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.319008  7805 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:28:00.338701  7809 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:28:00.352993  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.354420  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2616) took 1609 ms. Trace:
I20260709 12:28:00.354468  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1006) took 1894 ms. Trace:
I20260709 12:28:00.355082  7805 rpcz_store.cc:276] 0709 12:27:58.745225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.318332 (+1573107us) service_pool.cc:224] Handling call
0709 12:28:00.318656 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.332811 (+ 14155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.354355 (+ 21544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.355367  7809 rpcz_store.cc:276] 0709 12:27:58.460072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.301053 (+1840981us) service_pool.cc:224] Handling call
0709 12:28:00.309982 (+  8929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.339994 (+ 30012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.354405 (+ 14411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.323987  8411 rpcz_store.cc:276] 0709 12:27:59.283913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.313266 (+ 29353us) service_pool.cc:224] Handling call
0709 12:28:00.321410 (+1008144us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.321565 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167832960}
W20260709 12:28:00.361753  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60963.8us
I20260709 12:28:00.359110 13981 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.364363  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:00.365469 13981 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:00.365664 13786 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:00.366384  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:00.367764  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:00.366552  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 269881us
W20260709 12:28:00.377285  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68627.3us
W20260709 12:28:00.376901  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.378433  7655 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:28:00.371899  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.380877  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3125) took 2425 ms. Trace:
I20260709 12:28:00.381201 13981 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:00.381759 13981 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:00.382300 13981 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:00.380939  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.383989  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.385092  7662 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:28:00.384951  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.386049  7656 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.386286  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.387023  7662 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:00.381428  7806 rpcz_store.cc:276] 0709 12:27:57.951559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.027569 (+2076010us) service_pool.cc:224] Handling call
0709 12:28:00.065945 (+ 38376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.074540 (+  8595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.074575 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.083389 (+  8814us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.367328 (+283939us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:00.367384 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.367455 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:00.376688 (+  9233us) raft_consensus.cc:1685] finished
0709 12:28:00.376711 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.376922 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3641,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":11249,"thread_start_us":281686,"threads_started":1,"wal-append.queue_time_us":290623}
W20260709 12:28:00.388082  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3024) took 2710 ms (client timeout 1717 ms). Trace:
I20260709 12:28:00.392953  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:00.393303 13981 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:00.393370  7730 rpcz_store.cc:269] 0709 12:27:57.677934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.678085 (+   151us) service_pool.cc:224] Handling call
0709 12:28:00.387727 (+2709642us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400419454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.387905 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530968320}
I20260709 12:28:00.394057  7736 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:00.394886  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.396442  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.397615  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.397980  7809 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:28:00.398618  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.391645  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.399981  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.400375  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.309170 13481 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:00.359725 13897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.663s	user 0.023s	sys 0.039s
I20260709 12:28:00.406795  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:00.367470 14011 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.201s	user 0.015s	sys 0.000s
I20260709 12:28:00.311993 13997 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.415417  7654 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:28:00.398252  7730 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:00.445516  7734 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:00.400678  7655 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:00.381389  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:00.447139  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 76679.7us
I20260709 12:28:00.448539  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.528800 13762 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.533666  8412 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:00.533478 13762 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:00.534664  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.535250  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.536363 13981 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:00.537149 13846 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:00.538033  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.512748  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.538880  7736 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:28:00.539889 13881 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.540601 13881 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.543606 13881 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:00.543689  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:00.530447  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.545444  7700 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:00.545738  7653 raft_consensus.cc:2368] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16.
W20260709 12:28:00.546446  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.546324 14037 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:00.547062 14037 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:00.547154  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.547612 14037 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.548796  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:00.549497  7653 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.551426  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:00.513439 13777 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:00.552707  7654 pending_rounds.cc:85] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 9
I20260709 12:28:00.553054  7654 pending_rounds.cc:107] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:28:00.553316  7654 raft_consensus.cc:2889] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:00.528173  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.543493  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:00.483676  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.484249  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:00.556710  8405 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
W20260709 12:28:00.561586  8406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.007s
W20260709 12:28:00.562247  8406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.012s	sys 0.007s
W20260709 12:28:00.563288  7728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.011s	sys 0.007s
W20260709 12:28:00.563927  7728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.011s	sys 0.007s
W20260709 12:28:00.566962  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.455778  7659 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:00.454265  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43708us
W20260709 12:28:00.445551  7729 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.001s
I20260709 12:28:00.571087  7729 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:00.572118  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3036) took 1350 ms (client timeout 1718 ms). Trace:
W20260709 12:28:00.572665  7729 rpcz_store.cc:269] 0709 12:27:59.222130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.283894 (+ 61764us) service_pool.cc:224] Handling call
0709 12:28:00.572016 (+1288122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.522802  7812 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:00.585331  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:00.582724  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2613) took 1902 ms. Trace:
I20260709 12:28:00.587877  7812 rpcz_store.cc:276] 0709 12:27:58.679965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.310667 (+1630702us) service_pool.cc:224] Handling call
0709 12:28:00.446202 (+135535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.582331 (+136129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.582646 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.590636  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:00.453446 13897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.900s	user 0.024s	sys 0.040s
W20260709 12:28:00.563102 13996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.009s
I20260709 12:28:00.598865  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:28:00.627753  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50902.4us
I20260709 12:28:00.628295  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.447929 13935 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:00.632871  7658 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:28:00.636420  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.531530 13949 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:00.638037  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2597) took 2709 ms (client timeout 2592 ms). Trace:
I20260709 12:28:00.585506 13356 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.360571 14024 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:00.638724  7803 rpcz_store.cc:269] 0709 12:27:57.928957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.930172 (+  1215us) service_pool.cc:224] Handling call
0709 12:28:00.597899 (+2667727us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54003ebcd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:00.637951 (+ 40052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5044764928}
W20260709 12:28:00.627825 13996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.009s	sys 0.010s
I20260709 12:28:00.407649 13985 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.622319 13881 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:00.635320 13935 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.655809  7737 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:00.660446 13880 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:28:00.662463  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.663468  7659 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:00.664448 13935 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:00.672791  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1007) took 1572 ms. Trace:
I20260709 12:28:00.673456  7805 rpcz_store.cc:276] 0709 12:27:59.100643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.356592 (+1255949us) service_pool.cc:224] Handling call
0709 12:28:00.395571 (+ 38979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.395860 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:00.396624 (+   764us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.672229 (+275605us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.672321 (+    92us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:00.672390 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.672493 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.672699 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42961,"prepare.queue_time_us":1154,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"thread_start_us":231958,"threads_started":1}
I20260709 12:28:00.681680 13749 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.682430 13749 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.685925 13749 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:00.691350  7775 leader_election.cc:336] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.765s (SENT)
W20260709 12:28:00.692617  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48310.7us
I20260709 12:28:00.640784 13832 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:28:00.666939 13860 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:28:00.653038  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 107676us
W20260709 12:28:00.655861  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 112473us
I20260709 12:28:00.592159 13997 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.730907  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:00.707592  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.730891  7657 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:00.665894  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:00.748129  7658 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
W20260709 12:28:00.772168  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 318781us
I20260709 12:28:00.732201 13832 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:00.789357  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.789516  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.794929  7731 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:00.797626  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.798470  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.800261  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.787714  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:00.650168 13356 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:00.805763  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.808892  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:00.815477  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:00.780267 13897 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:00.779243 13972 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:00.827813  7702 leader_election.cc:400] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:00.801705 13832 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:00.830227  7702 leader_election.cc:403] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:28:00.821651  7626 connection.cc:256] RPC call timeout handler was delayed by -1.3868719479996798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:28:00.762406 13881 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.858522  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2608) took 2710 ms. Trace:
I20260709 12:28:00.793370 13656 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:28:00.884859  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3030) took 1731 ms. Trace:
I20260709 12:28:00.885456  7737 rpcz_store.cc:276] 0709 12:27:59.153884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.159478 (+  5594us) service_pool.cc:224] Handling call
0709 12:27:59.159914 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.663348 (+1503434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.871561 (+208213us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:00.884767 (+ 13206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3426311936}
I20260709 12:28:00.871361  7653 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:00.893432 13524 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:00.895210 13985 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:00.896281  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 230514us
I20260709 12:28:00.897549  8380 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:00.890774  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.848084 13529 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:00.853047 13786 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:28:00.915386  7656 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.003s
I20260709 12:28:00.916054  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:00.931807  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 116405us
I20260709 12:28:00.842504 13972 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:00.947568  7657 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:00.953414  7656 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:00.992030  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.992794  7656 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:28:00.996552  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 187765us
I20260709 12:28:00.994717  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.951005 13985 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:00.953975 13749 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.054884  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.998673 13796 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.016287  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:00.964320 13529 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:00.976177  7659 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:01.021314 13771 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:28:01.021840 14032 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:01.022280 14048 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
W20260709 12:28:01.024523 14042 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.238s	user 0.003s	sys 0.000s
W20260709 12:28:01.024986 14045 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:01.025214 13997 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.044911 13524 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:01.052069  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 251810us
I20260709 12:28:00.859702  7807 rpcz_store.cc:276] 0709 12:27:58.148088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.191085 (+2042997us) service_pool.cc:224] Handling call
0709 12:28:00.196178 (+  5093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.196469 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:00.369337 (+172868us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.857794 (+488457us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.857912 (+   118us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:00.857989 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.858245 (+   256us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.858453 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165632,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":10936,"thread_start_us":434112,"threads_started":2}
I20260709 12:28:01.074199  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.085086 14024 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.039844 13905 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:28:01.099399  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.108884 13881 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.109033  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:01.110411  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:00.915032 13981 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:01.102806  7657 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:01.110248  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.154340  7737 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:01.111778  7658 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:00.962117 13762 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:01.129699  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:01.108930 14043 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.221s	user 0.003s	sys 0.000s
I20260709 12:28:01.144418  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:01.130935 14040 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.263s	user 0.000s	sys 0.003s
I20260709 12:28:01.151989  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:01.157352  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:01.140390 14052 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
W20260709 12:28:01.169822  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 272235us
I20260709 12:28:01.171037  7731 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:01.171186  8405 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:28:01.172802  7662 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.001s
I20260709 12:28:01.163177  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:01.174054  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44470.3us
I20260709 12:28:01.174896  7662 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:01.205166  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:01.152575 13524 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:01.207695  7803 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:01.206882  7730 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:28:01.151628 13905 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:01.211625  7662 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:01.213546  7654 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:01.210479  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.211388  7731 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17. Re-sending same reply.
I20260709 12:28:01.151177 13978 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:01.221521  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.169236 13712 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:28:01.140334 13985 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.197623 13846 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:01.140522 14009 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:28:01.230334  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:01.208477 13881 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:01.136632 13796 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.208775 13481 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:01.236871  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:01.238534  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 101197us
W20260709 12:28:01.241065  7627 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.003s
I20260709 12:28:01.246843  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.251169  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:01.165391 13749 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.228000 13905 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:01.172972 13981 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:01.245627  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:01.267766  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.265985  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:01.254149  7627 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.004s
W20260709 12:28:01.265138  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 254096us
W20260709 12:28:01.425119  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.413249  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:01.311903  7803 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:28:01.313426  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2680) took 4123 ms. Trace:
W20260709 12:28:01.422073  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156183us
I20260709 12:28:01.320755  7662 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 12:28:01.445875  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 744206us
W20260709 12:28:01.413659 14053 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:28:01.401093  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104821us
I20260709 12:28:01.296288  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:01.413578 14038 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:28:01.457926  7812 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:01.412806 14037 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.864s	user 0.008s	sys 0.007s
I20260709 12:28:01.269730  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:01.399376 13935 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.731s	user 0.016s	sys 0.003s
I20260709 12:28:01.300149 13978 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.455636  7660 rpcz_store.cc:276] 0709 12:27:57.053711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.332890 (+279179us) service_pool.cc:224] Handling call
0709 12:27:57.333421 (+   531us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:57.334239 (+   818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.334290 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:57.349670 (+ 15380us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.519702 (+3170032us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b180048f9a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1
0709 12:28:00.651840 (+132138us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:00.870963 (+219123us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.141220 (+270257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.146024 (+  4804us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.146095 (+    71us) raft_consensus.cc:1685] finished
0709 12:28:01.157583 (+ 11488us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.177530 (+ 19947us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7492,"prepare.queue_time_us":35890,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":8631,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":24827,"spinlock_wait_cycles":19903232,"thread_start_us":735254,"threads_started":3,"wal-append.queue_time_us":817452,"wal-append.run_cpu_time_us":4176,"wal-append.run_wall_time_us":1479944}
I20260709 12:28:01.493242  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.494509 14070 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:28:01.455374  7803 pending_rounds.cc:85] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 12
W20260709 12:28:01.413367 14035 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 12:28:01.500211  8415 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 12:28:01.306099 14049 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.100s	user 0.000s	sys 0.002s
I20260709 12:28:01.447988 13762 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:01.314790 13972 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:01.504223  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50568us
I20260709 12:28:01.310680 13632 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:01.508152  7803 pending_rounds.cc:107] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:28:01.508584  7803 raft_consensus.cc:2889] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:01.266774 13881 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:01.313547 13749 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.523335 13838 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:01.505115 13762 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.524636 13762 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:01.518348 13777 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.528170 13777 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:01.523861 14046 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.381s	user 0.000s	sys 0.002s
I20260709 12:28:01.506274  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:01.532857  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:01.384342 14011 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:28:01.495491 14066 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:01.540076  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2815) took 1154 ms. Trace:
I20260709 12:28:01.546985  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:01.547715  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1137) took 1023 ms. Trace:
I20260709 12:28:01.548327  7653 rpcz_store.cc:276] 0709 12:28:00.524506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.534476 (+1009970us) service_pool.cc:224] Handling call
0709 12:28:01.547127 (+ 12651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.547379 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.547407 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.547451 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:01.547481 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.547534 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.547648 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.271024 13529 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.561877  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:01.499362 14037 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.563055  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1138) took 1038 ms. Trace:
I20260709 12:28:01.566602 14009 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:01.565860  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:01.567968  7660 rpcz_store.cc:276] 0709 12:28:00.524787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.535769 (+1010982us) service_pool.cc:224] Handling call
0709 12:28:01.552116 (+ 16347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.562694 (+ 10578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.562725 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.562782 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:01.562814 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.562882 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.562998 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.561158 14032 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:01.570350 13796 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:01.560089 14066 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.552948 13880 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:01.585253 13880 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:01.585490 13832 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.005s
I20260709 12:28:01.585750 13880 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:01.585968 13832 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.586551 13978 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.591501 14064 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:01.592155 14064 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.592514 14064 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:01.593750 13524 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.597291 13762 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.601912 13832 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.603402 14066 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.564632 14070 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:01.605476 13762 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.575623 14009 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:01.612310 14009 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:01.612969  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.615307 13978 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:01.571674  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:01.574803  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3133) took 2985 ms. Trace:
I20260709 12:28:01.616015 13978 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.616366  8415 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:01.575951  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1139) took 1050 ms. Trace:
I20260709 12:28:01.616264  7804 rpcz_store.cc:276] 0709 12:27:58.589197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.304088 (+1714891us) service_pool.cc:224] Handling call
0709 12:28:00.305776 (+  1688us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.337452 (+ 31676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.337506 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.369478 (+ 31972us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.647455 (+277977us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:28:00.647518 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.647639 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.574486 (+926847us) raft_consensus.cc:1685] finished
0709 12:28:01.574511 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.574727 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36123,"raft.queue_time_us":41180,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":58605,"thread_start_us":86992,"threads_started":1,"wal-append.queue_time_us":466100}
I20260709 12:28:01.617498  7653 rpcz_store.cc:276] 0709 12:28:00.525060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.555318 (+1030258us) service_pool.cc:224] Handling call
0709 12:28:01.567648 (+ 12330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.575486 (+  7838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.575527 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.575589 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:01.575626 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.575702 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.575877 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.618451 13978 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.619401  7653 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:01.620159  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1141) took 1094 ms. Trace:
I20260709 12:28:01.620503  7653 rpcz_store.cc:276] 0709 12:28:00.525645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.618570 (+1092925us) service_pool.cc:224] Handling call
0709 12:28:01.618953 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.619843 (+   890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.620068 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.576913 13777 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:01.577389 14055 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:28:01.577855 14058 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:28:01.579667  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3028) took 2934 ms. Trace:
I20260709 12:28:01.500756 13935 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.573115  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:01.583978  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:01.533942 13632 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.622058  7653 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:01.632490  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1142) took 1106 ms. Trace:
I20260709 12:28:01.632881  7653 rpcz_store.cc:276] 0709 12:28:00.525915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.621342 (+1095427us) service_pool.cc:224] Handling call
0709 12:28:01.621684 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.632131 (+ 10447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.632388 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.634001  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:01.623875  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1146) took 1096 ms. Trace:
I20260709 12:28:01.635528  7656 rpcz_store.cc:276] 0709 12:28:00.527012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.622926 (+1095914us) service_pool.cc:224] Handling call
0709 12:28:01.623291 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.623498 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.623523 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.623565 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:01.623595 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.623654 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.623772 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.636097  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2606) took 3504 ms. Trace:
I20260709 12:28:01.636770  7810 rpcz_store.cc:276] 0709 12:27:58.131872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.184174 (+2052302us) service_pool.cc:224] Handling call
0709 12:28:00.194916 (+ 10742us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.195518 (+   602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.195572 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.221481 (+ 25909us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.754158 (+532677us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:00.754208 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.754295 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.271608 (+517313us) raft_consensus.cc:1685] finished
0709 12:28:01.271630 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.635794 (+364164us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54003723d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:01.636025 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9566,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":23384448,"thread_start_us":501798,"threads_started":1,"wal-append.queue_time_us":606217}
I20260709 12:28:01.624943  7734 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:01.576234 14032 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:01.657359 14032 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.658877 14032 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:01.544394  7658 rpcz_store.cc:276] 0709 12:28:00.385653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.206705 (+821052us) service_pool.cc:224] Handling call
0709 12:28:01.228454 (+ 21749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.246124 (+ 17670us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:01.315524 (+ 69400us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.539726 (+224202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.539783 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:01.539830 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.539896 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.540008 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2189,"prepare.queue_time_us":68299,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":7351,"thread_start_us":148613,"threads_started":1}
I20260709 12:28:01.624364  7655 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:01.653661 14060 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:01.654229 13481 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:01.663782  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2812) took 1279 ms. Trace:
I20260709 12:28:01.640383  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2821) took 1117 ms. Trace:
I20260709 12:28:01.675791  7653 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:01.626421  7657 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:01.640404  7660 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:01.151590  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:01.630177  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:01.678483  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48397.8us
I20260709 12:28:01.625479 13935 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.630370 13978 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:28:01.628051 13632 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:01.681552  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121034us
I20260709 12:28:01.681759  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1140) took 1156 ms. Trace:
I20260709 12:28:01.682624  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:01.682181  7557 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:01.682235  7660 rpcz_store.cc:276] 0709 12:28:00.525356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.573288 (+1047932us) service_pool.cc:224] Handling call
0709 12:28:01.640224 (+ 66936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.681279 (+ 41055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.681691 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.685729  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1147) took 1158 ms. Trace:
I20260709 12:28:01.686182  7657 rpcz_store.cc:276] 0709 12:28:00.577656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.625772 (+1048116us) service_pool.cc:224] Handling call
0709 12:28:01.626213 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.685461 (+ 59248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.685668 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.676400  7662 rpcz_store.cc:276] 0709 12:28:00.522858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.250847 (+727989us) service_pool.cc:224] Handling call
0709 12:28:01.279837 (+ 28990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.456100 (+176263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.640130 (+184030us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b54001a3bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:01.640319 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92443520}
I20260709 12:28:01.684875  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:01.691058  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 147624us
W20260709 12:28:01.693652  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 109764us
I20260709 12:28:01.692889  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2706) took 1637 ms. Trace:
I20260709 12:28:01.626106 14037 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.694979  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:01.622944  7733 rpcz_store.cc:276] 0709 12:27:58.645134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.667530 (+ 22396us) service_pool.cc:224] Handling call
0709 12:28:00.512730 (+1845200us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:28:00.512936 (+   206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.513335 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.513366 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.514001 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.628642 (+114641us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:00.628691 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.628768 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.571593 (+942825us) raft_consensus.cc:1685] finished
0709 12:28:01.571617 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.579391 (+  7774us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16223,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":119,"spinlock_wait_cycles":78114560,"thread_start_us":114181,"threads_started":1,"wal-append.queue_time_us":390763}
I20260709 12:28:01.689304 13749 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:01.675377  7659 rpcz_store.cc:276] 0709 12:28:00.384654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.166962 (+782308us) service_pool.cc:224] Handling call
0709 12:28:01.207463 (+ 40501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.215763 (+  8300us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:28:01.215797 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.261210 (+ 45413us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.516030 (+254820us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:01.516071 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.516150 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.663409 (+147259us) raft_consensus.cc:1685] finished
0709 12:28:01.663444 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.663673 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8136,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":14350,"thread_start_us":194585,"threads_started":1,"wal-append.queue_time_us":358945}
I20260709 12:28:01.697842 13481 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.700183 13749 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.702903 13749 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:01.704185 13786 raft_consensus.cc:2764] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:28:01.704838  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:01.705814  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:01.674342 14060 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:01.708278 14060 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.709632 14060 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:01.710494 14070 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:01.711143 14070 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:01.711719 14070 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:01.712075 13356 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:01.712749 13356 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:01.713414 13356 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:01.713780 13356 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:01.714148 13356 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:01.714072 13786 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:01.715284 13796 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:01.704288 13481 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.715588 13978 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:01.715832 13796 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.716342 13796 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:01.716279 13978 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.718811 13978 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.692766 14058 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:01.725975  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.696668 13712 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:28:01.735615 13712 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.736104 13712 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:28:01.737100 14034 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
I20260709 12:28:01.695173  7661 rpcz_store.cc:276] 0709 12:28:00.055686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.055914 (+   228us) service_pool.cc:224] Handling call
0709 12:28:00.079592 (+ 23678us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.089483 (+  9891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.089526 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.140148 (+ 50622us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.589939 (+449791us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:00.649346 (+ 59407us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.649683 (+   337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.649795 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.064566 (+414771us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b180043a740. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:01.685221 (+620655us) raft_consensus.cc:1685] finished
0709 12:28:01.692622 (+  7401us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.692818 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":446,"prepare.queue_time_us":59141,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":7054,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":22227,"spinlock_wait_cycles":7012224,"thread_start_us":395945,"threads_started":1,"wal-append.queue_time_us":959373}
I20260709 12:28:01.745178  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1292) took 1709 ms. Trace:
I20260709 12:28:01.746382 14064 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.747779  8408 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:01.748627 13981 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.751817  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:01.753715 14065 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:28:01.745620  8406 rpcz_store.cc:276] 0709 12:28:00.035796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.039385 (+  3589us) service_pool.cc:224] Handling call
0709 12:28:00.039867 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.040482 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.040522 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.041235 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.562856 (+521621us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:00.644183 (+ 81327us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.650525 (+  6342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.650632 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.736381 (+1085749us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:01.744840 (+  8459us) raft_consensus.cc:1685] finished
0709 12:28:01.744935 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.745074 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4524,"prepare.queue_time_us":581527,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":6221,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":4262,"spinlock_wait_cycles":7838592,"thread_start_us":566675,"threads_started":2,"wal-append.queue_time_us":1229955}
I20260709 12:28:01.762074 14064 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:01.723809 13881 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.764004 13985 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.765038  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:01.766187  7662 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:01.767477  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1144) took 1241 ms. Trace:
I20260709 12:28:01.767927  7655 rpcz_store.cc:276] 0709 12:28:00.526464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.844856 (+318392us) service_pool.cc:224] Handling call
0709 12:28:01.767209 (+922353us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:01.767403 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660747136}
W20260709 12:28:01.771438  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.774140  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:01.774922 14047 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.775583 14047 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.778019 14047 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.775167  7659 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:01.697178 14071 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:01.779431  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:01.795936  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.796316  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.798075  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:01.799953  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1261) took 1256 ms. Trace:
I20260709 12:28:01.800084  7735 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:01.800482  7734 rpcz_store.cc:276] 0709 12:28:00.543975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.575840 (+ 31865us) service_pool.cc:224] Handling call
0709 12:28:01.799703 (+1223863us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400283054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:01.799887 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289746816}
I20260709 12:28:01.802098  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.803021  7653 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:01.800925  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:01.805847  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.807678  7661 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:01.625541 14009 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.806720  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.810231  7731 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:01.811187  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.778613 14047 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.812997  7657 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:01.816221  7659 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:01.780860 14071 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.834638  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:01.834326  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.843194  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:01.826887 13656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.016s	sys 0.019s
I20260709 12:28:01.844648  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:01.795825 14069 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:01.843691  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:01.849633  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48722.2us
I20260709 12:28:01.815169 14058 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:01.821421 13762 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:01.809871 14011 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:01.874012  7730 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:01.870738  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:01.853410  7658 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:01.876657  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.830052 14064 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:01.889434  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:01.851820  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:01.880259  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:01.883018 14067 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:01.842473 14047 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:01.896019 14064 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:01.883853  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 71937.1us
I20260709 12:28:01.897293 14047 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.898371 14064 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.899538 14067 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.900091 14067 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:01.894388  7728 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 10.
I20260709 12:28:01.873924 14069 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.884455 14048 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:28:01.874903 13656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.020s	sys 0.019s
I20260709 12:28:01.810751 13749 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.885360 14058 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.888003  7734 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:28:01.850960 13771 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:01.840903 14071 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:01.920269 13777 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:01.885155 13846 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:01.927196 13524 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:01.877736  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:01.931286  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 224241us
I20260709 12:28:01.950588  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:01.937435  8412 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 10.
I20260709 12:28:01.913255 14069 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:01.926867 14058 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:01.987528  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 736438us
I20260709 12:28:01.894418 13762 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:01.918107 14064 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:28:01.835850 13880 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:01.969282 14083 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:28:02.003693  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77394.7us
I20260709 12:28:02.021150  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:02.008795 13880 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:01.995036 14061 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:01.937009 13972 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:28:02.031719 13838 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:02.035003 13762 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:01.951026 13846 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.029165 13777 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:02.014443 13632 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 12:28:02.207582  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 190104us
W20260709 12:28:02.335573  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 498757us
W20260709 12:28:02.330242  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.296672  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1136) took 1772 ms. Trace:
I20260709 12:28:02.285091  7730 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:28:02.224313  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.128773  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 103343us
I20260709 12:28:02.339066  7654 rpcz_store.cc:276] 0709 12:28:00.524205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.246578 (+722373us) service_pool.cc:224] Handling call
0709 12:28:01.272211 (+ 25633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.301299 (+ 29088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.301358 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.499778 (+198420us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.806167 (+306389us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:01.806210 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.806281 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.296357 (+490076us) raft_consensus.cc:1685] finished
0709 12:28:02.296391 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.296584 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":636,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":11995,"thread_start_us":262018,"threads_started":1,"wal-append.queue_time_us":352343}
W20260709 12:28:02.328904  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 278888us
I20260709 12:28:02.350723  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 969) took 1086 ms. Trace:
I20260709 12:28:02.351246  8411 rpcz_store.cc:276] 0709 12:28:01.263928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.264106 (+   178us) service_pool.cc:224] Handling call
0709 12:28:01.298022 (+ 33916us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.384674 (+ 86652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.384883 (+   209us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.770207 (+385324us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b2c000e5b88. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:01.779065 (+  8858us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.045452 (+266387us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:02.045512 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.045619 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.350383 (+304764us) raft_consensus.cc:1685] finished
0709 12:28:02.350404 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.350651 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3280,"raft.queue_time_us":102718,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"spinlock_wait_cycles":10940800,"thread_start_us":212825,"threads_started":1,"wal-append.queue_time_us":542400}
I20260709 12:28:02.352561 14061 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:02.095453  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74391.7us
W20260709 12:28:02.306815 14085 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.385s	user 0.002s	sys 0.000s
W20260709 12:28:02.305830 13796 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.003s
W20260709 12:28:02.106865 14084 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:28:02.365216  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:02.106395 14086 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:02.059764 13524 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:02.366407  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:02.249563 13712 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.000s
W20260709 12:28:02.108106 14062 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.774s	user 0.004s	sys 0.000s
W20260709 12:28:02.389458 14060 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.005s
W20260709 12:28:02.320375 13356 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.003s
W20260709 12:28:02.396040 13905 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.114s	user 0.010s	sys 0.004s
I20260709 12:28:02.370585  7702 leader_election.cc:400] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16.
I20260709 12:28:02.396562  7702 leader_election.cc:403] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:28:02.088079 14066 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:02.096453 13529 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:02.256551 14090 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:02.345216 14011 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:02.414827  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.304771 13846 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:02.107406 14009 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.369625 13897 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:28:02.417511  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.385666  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:02.373965 14032 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.003s
I20260709 12:28:02.386097  7729 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:02.386842  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.381441 13524 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:02.417152  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.420754  8407 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:02.104398 14082 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:02.422839  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.098821 13777 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.391601  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1251) took 2003 ms. Trace:
I20260709 12:28:02.427600  7732 rpcz_store.cc:276] 0709 12:28:00.388348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.420070 (+ 31722us) service_pool.cc:224] Handling call
0709 12:28:00.554169 (+134099us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.655933 (+101764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.655968 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.754997 (+ 99029us) log.cc:844] Serialized 52 byte log entry
0709 12:28:01.137928 (+382931us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:01.249017 (+111089us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.761143 (+512126us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b18004d8000. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:01.904310 (+143167us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:01.923078 (+ 18768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.923197 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.391197 (+468000us) raft_consensus.cc:1685] finished
0709 12:28:02.391265 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.391497 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80561,"prepare.queue_time_us":80151,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3112,"raft.queue_time_us":550220,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":28678,"spinlock_wait_cycles":13939968,"thread_start_us":553123,"threads_started":3,"wal-append.queue_time_us":426154}
I20260709 12:28:02.393549 14101 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:02.395637 13356 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:02.397205  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43478.6us
I20260709 12:28:02.251785 13632 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.401146  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.414702  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.418792  8413 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:28:02.387845 13712 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.435875  8414 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:28:02.425791  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79473us
I20260709 12:28:02.402319 13656 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16.
I20260709 12:28:02.423468  8412 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:02.456526  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:02.402626 14066 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:02.469573  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.470582  7810 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.470688 14066 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.471288  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.471729  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.472137 14066 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.472337  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:02.474097  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.380455 13796 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.475628  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.476694  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.477111 13712 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.478058  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.479609  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.480127 13897 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:02.469601 14011 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:02.481144 13356 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.481230 14011 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:02.482139  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:02.396556 13905 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.439363 13632 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.484356 14074 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:02.439956  8407 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:28:02.446858 13762 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:02.447599  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.422554 13524 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.407471 13529 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.486799 13762 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:02.487849 13846 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:02.488685 13846 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.422919 14090 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.463101 14100 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:02.489784 14100 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.487160 13529 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.463402  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.464072  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.461624  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.464923  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.466042  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.468222 13985 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:02.457355 14108 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.487934  8409 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:28:02.489141 13846 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:02.464556 14096 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 12:28:02.495080  7805 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.492758  7809 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:02.493903  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:02.466089 14095 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:28:02.462579  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.505497  8415 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:02.502408  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46014.5us
I20260709 12:28:02.511415  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.514725  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:02.517174  7702 connection.cc:256] RPC call timeout handler was delayed by -3.2235613110001395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:28:02.517700  7702 connection.cc:256] RPC call timeout handler was delayed by -3.1695613110000522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 12:28:02.518224  7702 connection.cc:256] RPC call timeout handler was delayed by -3.1587613110000348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 12:28:02.518436  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.513870  7812 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:02.520558  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.520781 13656 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:02.521401 13656 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.521898  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.522764  7811 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.525570  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.527804  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:02.518784  7702 connection.cc:256] RPC call timeout handler was delayed by -3.14346131100001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 12:28:02.529192  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:02.494299 14084 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:28:02.529438  7702 connection.cc:256] RPC call timeout handler was delayed by -3.0381613110002945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:28:02.514354  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:02.512199  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:02.516979  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.531684  7658 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.531831  7653 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:02.531785  7702 connection.cc:256] RPC call timeout handler was delayed by -2.9400613110001359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:28:02.533162  7702 connection.cc:256] RPC call timeout handler was delayed by -2.8923613110000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
I20260709 12:28:02.529495  7662 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:02.538829  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.539536  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:02.540387  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.541332  7803 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.482551 14011 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.491181 14090 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.559033  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.560169 14011 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:02.543779  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6700613110001541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 12:28:02.560871  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6214613110000755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260709 12:28:02.560956  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1134) took 1429 ms. Trace:
W20260709 12:28:02.561241  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4798779100001411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260709 12:28:02.561643  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4267779100000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:28:02.561426  8409 rpcz_store.cc:276] 0709 12:28:01.131765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.131957 (+   192us) service_pool.cc:224] Handling call
0709 12:28:01.141530 (+  9573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.507125 (+1365595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.560646 (+ 53521us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400691ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:02.560886 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3089890560}
W20260709 12:28:02.562254  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3583779100003994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 12:28:02.562641  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3484779100003834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 12:28:02.562803  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.562989  7702 connection.cc:256] RPC call timeout handler was delayed by -2.345353570000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:28:02.563345  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2596613109999453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25920000000000004s for a response
W20260709 12:28:02.563659  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2110613109998667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2646s for a response
W20260709 12:28:02.563948  7702 connection.cc:256] RPC call timeout handler was delayed by -2.148253570000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:28:02.564221  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0933535700000903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:28:02.564484  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0118779100002939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
I20260709 12:28:02.565142  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:02.565260 13762 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.566401  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:02.546546 14047 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.566751 13762 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:02.546380  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:02.551358 13838 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:28:02.492270 14100 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:02.554586  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.555817  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.546314 14061 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:02.556510  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.556821  7653 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:02.560369  7805 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.567374 14047 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:02.568073  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.575737 14074 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:28:02.494799 13905 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:02.543128  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.550515  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.577301  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.578497  8415 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:02.477144 13796 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.540306  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:02.552023  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.583753  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.584026  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.586730  7654 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:02.507468 13972 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:28:02.620862  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.528601 13656 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:02.586700  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.596357  7804 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:02.599007  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.578006 13749 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.637782  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:02.641453  7728 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:28:02.600374  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 216104us
I20260709 12:28:02.612910  7803 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:28:02.614995  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:02.616976  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.200s (SENT)
I20260709 12:28:02.583727 13838 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:02.622628  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.658741  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:02.599562 13997 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:28:02.616043 14070 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.003s
I20260709 12:28:02.593612 14064 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:02.675866  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.677721  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:02.604733 14061 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:02.651885  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:02.600248 14074 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:02.690007  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 123705us
I20260709 12:28:02.695892  7658 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:02.677431 13838 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:02.678402 13997 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:02.696969  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 100552us
I20260709 12:28:02.699939  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.730315  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:02.702072 14067 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.004s
I20260709 12:28:02.702363  7733 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:02.621403 13972 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:02.737133  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.711723  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.712488 14070 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.745898 12650 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:28:02.748150 14070 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:02.748328 12650 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:02.713107 14047 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:02.749961  7807 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.713835  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.720067 13524 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.751437  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.752629  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.753715  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.715536  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:02.754817 13524 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:02.718490  7804 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:02.506801 14108 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.756385  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.757557 14108 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:02.637002 14095 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:02.723915  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.760833  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.761270  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.762264 13881 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:02.762895 13881 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.763381 13881 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.763893 14087 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:02.764496 14087 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.765056 14087 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.766240  7808 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:02.767380 14009 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:02.767864 14009 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.768249 14009 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:02.768363 13524 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:02.738932  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 121981us
I20260709 12:28:02.766726  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.718096 14064 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:02.731853  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:28:02.725018 14074 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:02.771463  7803 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12.
I20260709 12:28:02.745955  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.764341  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:02.775499  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:02.777875  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 311463us
W20260709 12:28:02.770874  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.226s (SENT) [suppressed 43 similar messages]
W20260709 12:28:02.793131  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 231570us
I20260709 12:28:02.732067 13997 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.800567  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:02.744320 13972 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:02.809340  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:02.810344  8409 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:02.812314  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.769518 13524 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:02.748347 13880 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:02.866942  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 102700us
I20260709 12:28:02.868127  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:02.838963  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.871008  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.806708 14047 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:02.847684  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:02.797865 14064 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:02.838797 14069 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.819s	user 0.014s	sys 0.000s
I20260709 12:28:02.814851 14060 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:02.734543 14112 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
W20260709 12:28:02.797678 14058 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.780s	user 0.006s	sys 0.008s
I20260709 12:28:02.895432 13978 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:02.895993 13978 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:02.897159 13978 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.898356 13978 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.897974 13838 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.899379  8412 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:02.908411  7810 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:02.889587 14069 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.838478 14113 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:02.743320 14067 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:02.855630 13524 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:02.886112 14037 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:02.925225  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57217.3us
I20260709 12:28:02.933527  7731 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:02.937178  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 161828us
I20260709 12:28:02.913193 13777 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:02.956312  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 116115us
W20260709 12:28:02.929100 14114 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:28:02.958756  7735 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:02.959066  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:02.914907 14113 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:02.962689  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:02.962980  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 239591us
I20260709 12:28:02.965353  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:02.924348  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 209120us
W20260709 12:28:02.978539  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.986063  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 114405us
I20260709 12:28:02.912302 14058 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:02.959928 13777 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.991313  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.973337  8412 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 12.
I20260709 12:28:02.994501  7804 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:02.979339  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:02.983726 14037 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:02.965629 14113 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.974270 13905 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:03.025516  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46351.6us
I20260709 12:28:02.916615 14061 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.000186 13777 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.982082 13846 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.083655  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:03.072410  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 113483us
W20260709 12:28:03.092890  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 154731us
I20260709 12:28:03.090332  7702 leader_election.cc:400] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16.
I20260709 12:28:03.122013  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:03.090370  8409 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:03.029587 13905 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:03.128043  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.559s (SENT)
I20260709 12:28:03.130116  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:03.132555  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 106014us
I20260709 12:28:03.009150 14120 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.116s
W20260709 12:28:03.079731 14116 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 12:28:03.140014  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 138768us
I20260709 12:28:03.050060 14123 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:28:03.071987 14118 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:28:03.071412 14117 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.091s	user 0.001s	sys 0.001s
W20260709 12:28:03.158295  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.081213 14132 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:03.169014  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 177828us
I20260709 12:28:03.169678  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:03.071039 13838 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.187153  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:03.168000 13529 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.007s
I20260709 12:28:03.185563  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:03.160161 14090 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:28:03.078927 14009 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:03.222347  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67403.1us
W20260709 12:28:03.213083  7728 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 12:28:03.144866 13762 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:28:03.175343 14011 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:03.251693  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91681.1us
I20260709 12:28:03.255574  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.257016 13897 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:03.257859  7810 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:03.265535  7728 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:28:03.267398  8380 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:03.220022 14037 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:03.269958  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:03.258879  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:03.237205 14069 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:03.291105  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 104051us
I20260709 12:28:03.292057  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.293090  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:03.296094  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 168086us
I20260709 12:28:03.296325  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.302822  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:03.214404 13529 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.277637 14037 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:03.309695 14037 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:03.268290 14011 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.313700 14110 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:28:03.317415  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.275434 14090 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:03.299171 14071 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.359s	user 0.014s	sys 0.002s
I20260709 12:28:03.325223  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.313736 14011 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:03.283638 14142 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:03.328601 13762 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:03.298146 13838 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:03.298127 14120 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:03.356761 14069 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:03.358465  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:03.324465 14123 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:03.391676  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 74539.8us
W20260709 12:28:03.402961  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100230us
I20260709 12:28:03.404150  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:03.337018 14058 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:03.348227 13529 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:03.405100  8379 leader_election.cc:400] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 11. Current term is 12.
I20260709 12:28:03.360558 14067 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:03.360190 13762 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:03.412767  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:03.413934  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55621.2us
W20260709 12:28:03.404145 13997 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.003s
I20260709 12:28:03.385362 14139 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:28:03.417162 13762 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:03.422326 14111 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:03.423452 14149 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:03.423799 14143 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:28:03.423997 14149 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:03.424453 14149 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:03.425449 14148 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:03.426105 14148 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.426829 14145 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:03.427850 14142 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:03.428435 14142 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:03.428772 14148 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:03.429584 14148 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:03.430349 14148 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:03.433209  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.434034  7661 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:28:03.435514  8409 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [15.15-17.16]
I20260709 12:28:03.438704  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.439845  7658 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:03.403825 13846 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.449621  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:03.450080  7656 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:03.405053 14130 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:03.409492 14047 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:03.450623  8405 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:03.432240 14074 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.618s	user 0.008s	sys 0.006s
I20260709 12:28:03.429967 14120 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:03.476681  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.477517  7655 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:03.418191 14058 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.348515 14071 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.431879 13838 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:03.481503  8411 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:28:03.432392 14139 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:03.427350 14122 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:03.557500  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1306) took 1131 ms. Trace:
I20260709 12:28:03.558148  8415 rpcz_store.cc:276] 0709 12:28:02.425746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.438464 (+ 12718us) service_pool.cc:224] Handling call
0709 12:28:03.557218 (+1118754us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:03.557404 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76952448}
W20260709 12:28:03.451658 14064 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.002s
I20260709 12:28:03.530058  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:03.435657 13905 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.563349  7805 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:03.552534  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:03.555352  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:03.567165  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.476908 14009 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:03.472780 14047 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:03.436576 13997 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.590817  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.591697  7653 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:03.565083 14133 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.364s	user 0.000s	sys 0.002s
I20260709 12:28:03.603982  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:03.575766 14134 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.361s	user 0.000s	sys 0.002s
I20260709 12:28:03.429487 14142 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.486958 14074 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.543740 14148 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.612236 13632 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.612603  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.667824  7656 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:03.613546  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.571682 14122 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.597273 13846 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:28:03.620978  7626 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.773s (SENT)
W20260709 12:28:03.626783 13978 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.728s	user 0.005s	sys 0.009s
I20260709 12:28:03.628327 14119 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:03.628583  7658 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:03.629918 13897 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:03.706602 13897 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:03.706848  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:03.634673  7661 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:03.707521  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:03.643411 13880 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:03.644548  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3155) took 1176 ms. Trace:
I20260709 12:28:03.646270 14009 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:03.647397 14069 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:03.650589 14155 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:28:03.653355  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:03.654340  7653 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:03.546149 14110 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:03.751602 14069 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.752315 14110 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.663784  7729 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:03.483841 14067 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.118s
W20260709 12:28:03.666781 14087 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.901s	user 0.006s	sys 0.009s
I20260709 12:28:03.527978 14058 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.589783 14047 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:03.614602 13762 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.676986  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:03.769686  7660 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:03.692719 14074 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:03.693599  7730 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 12: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:03.695554  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:03.695708  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 74750.8us
I20260709 12:28:03.775949  7662 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:28:03.698629  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3150) took 1231 ms. Trace:
I20260709 12:28:03.776604  7808 rpcz_store.cc:276] 0709 12:28:02.466952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.479341 (+ 12389us) service_pool.cc:224] Handling call
0709 12:28:03.698393 (+1219052us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54003ea8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:03.698562 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656516992}
I20260709 12:28:03.699126  7803 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:03.699846  7656 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:03.702641  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:03.704783  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3075) took 1051 ms. Trace:
I20260709 12:28:03.711359  8415 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:28:03.711493  7658 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
W20260709 12:28:03.705523  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 520085us
W20260709 12:28:03.669018 13881 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.905s	user 0.017s	sys 0.000s
I20260709 12:28:03.715090  7807 rpcz_store.cc:276] 0709 12:28:02.468561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.477762 (+  9201us) service_pool.cc:224] Handling call
0709 12:28:03.644324 (+1166562us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:03.644477 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623353728}
I20260709 12:28:03.576619 14064 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:03.701188 14070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.012s	sys 0.027s
I20260709 12:28:03.721797  7657 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:03.716447 13880 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:03.725714 13997 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:03.731788  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1157) took 1197 ms. Trace:
I20260709 12:28:03.806854  7659 rpcz_store.cc:276] 0709 12:28:02.533966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.602388 (+ 68422us) service_pool.cc:224] Handling call
0709 12:28:02.641248 (+ 38860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.913215 (+271967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.913282 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.931089 (+ 17807us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.000629 (+ 69540us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:03.000739 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.001108 (+   369us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.731422 (+730314us) raft_consensus.cc:1685] finished
0709 12:28:03.731457 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.731691 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":169380,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":20987,"thread_start_us":283391,"threads_started":2,"wal-append.queue_time_us":186371}
I20260709 12:28:03.735424 14139 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.737212 13712 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:28:03.763595  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 233626us
I20260709 12:28:03.779505  7735 rpcz_store.cc:276] 0709 12:28:02.653275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.653462 (+   187us) service_pool.cc:224] Handling call
0709 12:28:03.704484 (+1051022us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540037e954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:03.704679 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417720704}
I20260709 12:28:03.791894  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:03.836359  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.682s (SENT)
I20260709 12:28:03.759255 14067 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:03.801349  7656 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:28:03.786367 14108 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.820725 13762 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.821277 13880 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.830848  7557 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 11 to 12, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:03.819234  7662 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:03.768878 14087 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:03.836858  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:03.835590 13712 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:03.822940 13777 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.766s	user 0.017s	sys 0.007s
I20260709 12:28:03.850955  7656 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:03.717200 14119 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.871758  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2632) took 1318 ms. Trace:
W20260709 12:28:03.864394  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.691s (SENT)
I20260709 12:28:03.870512  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1276) took 1278 ms. Trace:
I20260709 12:28:03.874480  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:03.875401  7729 rpcz_store.cc:276] 0709 12:28:02.592459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.592727 (+   268us) service_pool.cc:224] Handling call
0709 12:28:02.613250 (+ 20523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.758040 (+1144790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.870170 (+112130us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540037c154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:03.870434 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415339904}
I20260709 12:28:03.872350  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1310) took 1291 ms. Trace:
I20260709 12:28:03.874270  7810 rpcz_store.cc:276] 0709 12:28:02.553214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.915412 (+362198us) service_pool.cc:224] Handling call
0709 12:28:02.915901 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.258397 (+342496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.871463 (+613066us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:03.871693 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785334528}
I20260709 12:28:03.854080  7557 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 7 to 11, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 11 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:03.800227 14070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.013s	sys 0.027s
W20260709 12:28:03.892589  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.843012 14067 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:03.794634 13881 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.877115  8412 rpcz_store.cc:276] 0709 12:28:02.581174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.583702 (+  2528us) service_pool.cc:224] Handling call
0709 12:28:03.872136 (+1288434us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:03.872280 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651378432}
W20260709 12:28:03.719051 14115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.005s	sys 0.035s
I20260709 12:28:03.900091  7557 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:03.884824  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 237805us
I20260709 12:28:03.866672 14123 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:03.900105 14139 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:03.931448  7808 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:28:03.854269 13997 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.066s
W20260709 12:28:03.880368 14037 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.570s	user 0.005s	sys 0.007s
W20260709 12:28:03.867173  7554 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 182) took 42255 ms (client timeout 15000 ms). Trace:
W20260709 12:28:03.937932  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.859392 14129 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:03.919438 13978 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:03.867871 13777 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:04.003949  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 139609us
W20260709 12:28:04.004144 13972 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.164s	user 0.015s	sys 0.003s
I20260709 12:28:04.004557  7557 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 10, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 10 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:03.853048 14108 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:03.897282 14069 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:03.865428 13712 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:04.010587 14087 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:04.011880  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:04.018597  7627 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.756s (SENT)
W20260709 12:28:03.925117 14115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.008s	sys 0.036s
I20260709 12:28:04.010334 13524 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.049162  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:04.039160  7655 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 12:28:04.056756  7655 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:04.046116  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.059346 13524 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:04.070472  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2628) took 1518 ms (client timeout 1862 ms). Trace:
W20260709 12:28:04.071015  7804 rpcz_store.cc:269] 0709 12:28:02.552146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.756067 (+203921us) service_pool.cc:224] Handling call
0709 12:28:04.070277 (+1314210us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001338d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:04.070403 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324217728}
I20260709 12:28:04.072966  7557 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 7 to 13, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 13 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:28:04.057219 14120 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 12:28:04.008904 14037 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:04.078804  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60243.4us
W20260709 12:28:04.086333  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.773s (SENT)
I20260709 12:28:04.087405  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:04.062984 13978 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:04.025857 13972 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.077245 14123 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.099443  7733 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:04.083523 14120 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.112735  7728 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
W20260709 12:28:04.184105  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.112691  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.542s (SENT)
I20260709 12:28:04.025707 14067 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.193640  8406 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:28:04.196753  7557 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 10, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 10 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.199345  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:04.198236  7657 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:04.201083  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.202018  8413 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:04.192780 14164 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:28:04.204124  7557 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 8, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 8 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:04.103430 14130 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:04.206012  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93354.4us
W20260709 12:28:04.115589 14090 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.780s	user 0.014s	sys 0.000s
I20260709 12:28:04.105225 14058 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:04.103824 14064 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:04.209955  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.643s (SENT)
I20260709 12:28:04.057265 13846 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:04.223779  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:04.232122  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.943s (SENT)
I20260709 12:28:04.235802  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:04.227326  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 353123us
I20260709 12:28:04.108932 14123 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.243676  7728 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:28:04.241703  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.202248 14069 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:28:04.209515 14151 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:04.221249 14090 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:04.248569  7557 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.219852 13777 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:04.261724 14166 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:28:04.262794  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2626) took 1711 ms. Trace:
I20260709 12:28:04.269187  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.238804 14120 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:04.301101  7803 rpcz_store.cc:276] 0709 12:28:02.551468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.903474 (+352006us) service_pool.cc:224] Handling call
0709 12:28:03.698583 (+795109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.778695 (+ 80112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.262555 (+483860us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54003ea8d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:04.262721 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827695488}
W20260709 12:28:04.302100 14163 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 12:28:04.272691  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.251570 14115 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:04.304104 13846 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:04.320451 14144 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.767s	user 0.016s	sys 0.012s
W20260709 12:28:04.321978  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52889.6us
I20260709 12:28:04.284399  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:04.290676  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.325135  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.296761  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.330219  8412 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:04.330905  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.323096  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:04.332583  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.333560  8415 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:04.345304  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.346619  7732 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:04.327584 14160 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:28:04.291973 14119 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:04.354228  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.361539  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.387183  8411 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:28:04.391072  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3165) took 1846 ms. Trace:
I20260709 12:28:04.391731  7805 rpcz_store.cc:276] 0709 12:28:02.544880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.838605 (+293725us) service_pool.cc:224] Handling call
0709 12:28:04.390782 (+1552177us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400132ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:04.390991 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594083200}
W20260709 12:28:04.342705 14161 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.287s	user 0.001s	sys 0.001s
I20260709 12:28:04.331923 14166 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:28:04.407088  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53018.8us
I20260709 12:28:04.411581  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.414376  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:04.342979 13997 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:04.353706 13777 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.407838  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.335562 14108 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:04.386924 14087 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:04.426720  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.429741  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.496273  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3078) took 1838 ms. Trace:
I20260709 12:28:04.499858  7730 rpcz_store.cc:276] 0709 12:28:02.658200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.658372 (+   172us) service_pool.cc:224] Handling call
0709 12:28:04.495891 (+1837519us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540037df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:04.496171 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340509440}
I20260709 12:28:04.431761  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.496718  7557 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 8 to 9, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.466816  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.503628  7728 raft_consensus.cc:2368] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:28:04.506220 13972 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:04.408565 14173 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.506651  7557 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 6 to 9, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.507656 14173 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:04.508888  7557 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 6 to 9, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 9 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.510344 14173 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:04.427552 14058 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:28:04.511847 14176 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:28:04.512202  7557 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 7 to 8, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 8 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:04.513667  7734 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15.
I20260709 12:28:04.515730  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 976) took 1071 ms. Trace:
I20260709 12:28:04.386706 14088 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.405293  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:04.436147  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2634) took 1882 ms. Trace:
I20260709 12:28:04.520862  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.421826 13529 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:04.530229  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.421211 14070 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.408946 14037 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:04.496209 14172 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:28:04.543319  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:04.544068 14170 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:28:04.546262  7734 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:28:04.525956 14088 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:04.527128  7808 rpcz_store.cc:276] 0709 12:28:02.553763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.777926 (+1224163us) service_pool.cc:224] Handling call
0709 12:28:03.778389 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.939675 (+161286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.435659 (+495984us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54003f5054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:04.436010 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343818496}
I20260709 12:28:04.552793  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:04.530778 14161 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:28:04.518644  7735 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:04.556257  7731 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:28:04.556951  8413 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 12:28:04.558924 14167 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:28:04.559310 14115 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:04.560004  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.561403  7731 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:04.567157  7731 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:28:04.571967  7728 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:28:04.577517 14123 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:04.541289  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.580652  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.424912 14111 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.538056  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.538491  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:04.540766  7557 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 12 to 13, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:04.493288 13997 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:04.525357 13777 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.548832  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.516759  8409 rpcz_store.cc:276] 0709 12:28:03.444405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.444828 (+   423us) service_pool.cc:224] Handling call
0709 12:28:03.472221 (+ 27393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.472497 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:03.853914 (+381417us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.514868 (+660954us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.514996 (+   128us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:04.515114 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.515290 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.515563 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137795,"prepare.queue_time_us":81387,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":84172,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3421,"spinlock_wait_cycles":48896,"thread_start_us":872996,"threads_started":3,"wal-append.queue_time_us":426798}
I20260709 12:28:04.583717  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2633) took 2030 ms. Trace:
W20260709 12:28:04.586928  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48973.2us
W20260709 12:28:04.588482  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.575043 14088 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:04.519279 14087 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:04.005913  7554 rpcz_store.cc:269] 0709 12:27:21.611990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.614932 (+  2942us) service_pool.cc:224] Handling call
0709 12:28:03.866816 (+42251884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:51.547660 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:51.548719 (+  1059us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:51.548798 (+    79us) write_op.cc:435] Acquired schema lock
0709 12:27:51.548830 (+    32us) tablet.cc:662] Decoding operations
0709 12:27:51.550444 (+  1614us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:51.550646 (+   202us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:51.550700 (+    54us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:27:51.727159 (+176459us) tablet.cc:701] Row locks acquired
0709 12:27:51.727235 (+    76us) write_op.cc:260] PREPARE: finished
0709 12:27:51.727417 (+   182us) write_op.cc:270] Start()
0709 12:27:51.727586 (+   169us) write_op.cc:276] Timestamp: P: 1783600071727341 usec, L: 0
0709 12:27:51.727639 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 12:27:51.863898 (+136259us) log.cc:844] Serialized 4016 byte log entry
0709 12:27:53.359373 (+1495475us) op_driver.cc:464] REPLICATION: finished
0709 12:27:54.713420 (+1354047us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc005958ef0
0709 12:27:54.899894 (+186474us) write_op.cc:301] APPLY: starting
0709 12:27:54.900015 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:55.328142 (+428127us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:55.334501 (+  6359us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:56.360835 (+1026334us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:56.361167 (+   332us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:27:56.361236 (+    69us) write_op.cc:312] APPLY: finished
0709 12:27:56.361612 (+   376us) log.cc:844] Serialized 97 byte log entry
0709 12:27:56.735170 (+373558us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:56.811162 (+ 75992us) write_op.cc:454] Released schema lock
0709 12:27:56.863775 (+ 52613us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:56.948532 (+ 84757us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:56.959175 (+ 10643us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21267,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1519190,"apply.run_cpu_time_us":69436,"apply.run_wall_time_us":2011272,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45927,"num_ops":10,"prepare.queue_time_us":201214,"prepare.run_cpu_time_us":18340,"prepare.run_wall_time_us":446877,"raft.queue_time_us":347602,"raft.run_cpu_time_us":34163,"raft.run_wall_time_us":1540391,"replication_time_us":1532878,"spinlock_wait_cycles":56106240,"thread_start_us":2208961,"threads_started":5,"wal-append.queue_time_us":735441,"wal-append.run_cpu_time_us":42266,"wal-append.run_wall_time_us":3109490}]]}
I20260709 12:28:04.525657 14108 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 12:28:04.621883  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.636548  8412 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:04.646261  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.646592  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.661072  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:04.673349  7657 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:28:04.643775  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:04.662186  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:04.675953  7655 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:04.592321  8415 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 12: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:04.644847  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.661414  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:04.678696  7661 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:04.597486  7812 rpcz_store.cc:276] 0709 12:28:02.553484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.333342 (+779858us) service_pool.cc:224] Handling call
0709 12:28:03.333769 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.334225 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.334268 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:03.406826 (+ 72558us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.712089 (+305263us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:03.712136 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.712208 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.583476 (+871268us) raft_consensus.cc:1685] finished
0709 12:28:04.583501 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.583649 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":23015,"thread_start_us":260988,"threads_started":1,"wal-append.queue_time_us":466831}
I20260709 12:28:04.637912 14104 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:04.738544 14104 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:04.740015 14145 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.270s	user 0.013s	sys 0.003s
I20260709 12:28:04.739719 14104 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.740638 14145 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:28:04.742316 14047 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.957s	user 0.015s	sys 0.000s
I20260709 12:28:04.742794 14047 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.745394 14047 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:04.749742  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.753978  7661 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:04.754801  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:04.762247  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.633220 13712 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:04.766670  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.767417  7732 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:04.594650 14173 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:04.765172  7662 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:28:04.758630 14113 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.720s	user 0.008s	sys 0.011s
I20260709 12:28:04.760632  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:04.771760  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:04.760545 14181 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:28:04.773976  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:04.733762 14090 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:04.776400  7662 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:04.779664  7657 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:04.782009  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.771844 14145 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:28:04.785187 14145 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:04.785722 14145 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:04.786250 14139 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.842s	user 0.008s	sys 0.006s
I20260709 12:28:04.791529 14090 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.794277 14090 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:04.591398 14111 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.604838 14166 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.195s
I20260709 12:28:04.768421 14180 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:04.774896 14173 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:04.859083 14173 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:04.861017  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.862365  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.863135  7735 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15.
I20260709 12:28:04.864375  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.865280  7735 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:04.865080 14087 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:04.867013 14122 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:04.867865  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.869374  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:04.871018 14111 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:04.786615 14145 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.877728  7732 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:04.878952  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.901209  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:04.892802  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:04.905761  7736 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:04.783085 14113 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:04.983279  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.004572  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:28:04.965063  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72409.8us
I20260709 12:28:05.008306  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:05.010265  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.024644  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:05.061173  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 459062us
I20260709 12:28:05.069859  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.072752  7729 raft_consensus.cc:2376] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14. Re-sending same reply.
W20260709 12:28:05.075978  7775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.016s	sys 0.045s
W20260709 12:28:05.076462  7775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.016s	sys 0.045s
I20260709 12:28:05.037595 13712 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:04.886662 13796 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:04.906525 14087 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:04.993405 13524 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.914s	user 0.011s	sys 0.004s
I20260709 12:28:05.085522  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:05.086509  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:05.045753  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.080699 14138 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.528s	user 0.007s	sys 0.005s
I20260709 12:28:04.814667 13997 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.053308  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.009636 14111 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:04.873010 14180 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:05.076809  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 658341us
I20260709 12:28:05.027931  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.076995  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.110013  7736 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:05.089113 13712 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:04.888425 14058 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.083889 13796 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.064513 14145 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:04.993571 13771 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:28:05.098464  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.099745  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1150) took 2284 ms. Trace:
I20260709 12:28:05.124229  8406 rpcz_store.cc:276] 0709 12:28:02.815531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.815758 (+   227us) service_pool.cc:224] Handling call
0709 12:28:02.847847 (+ 32089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.200394 (+1352547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.099457 (+899063us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:05.099679 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2919848704}
I20260709 12:28:05.081071 14071 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.089430 13777 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.111428  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.086951  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.108081  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.130138  7732 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:28:05.127426  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.131605  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.132522  7732 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:05.133612  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.100958 14153 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.122413 14088 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:05.134438 14153 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.134755 14088 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.136891 14153 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.139622  7735 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:05.118788  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90988.8us
W20260709 12:28:05.133695 14182 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:28:05.106760 13524 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.140509  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.151154  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.136274 14088 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.134949  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.142553  7812 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:05.153520  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.153995  7732 raft_consensus.cc:2368] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13.
I20260709 12:28:05.154207  7806 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:05.143208  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.153707  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.143616  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.145952  7808 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:05.146008  7805 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:05.146653  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.152352  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.158613  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.152385  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.160369  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.160789  7804 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:28:05.090895 14087 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.156571 13712 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.156863  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.166194  7735 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:05.167773  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.168788  7735 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:05.163110  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.157377 13777 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:05.169155  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.170475  7803 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:05.172534  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.174418  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.176474  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:05.177168  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.178097  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.111553 14111 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:05.158274 14149 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.733s	user 0.013s	sys 0.007s
I20260709 12:28:05.181006  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.181839  7804 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:05.182982  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.182049  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.181890  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.187678  7735 raft_consensus.cc:2376] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17. Re-sending same reply.
I20260709 12:28:05.149226 14186 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:05.192652 14058 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.204802 14111 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.205786 14149 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.209051 14149 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.211220 14087 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.165146 13771 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:05.212699 13771 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.214156 13771 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.216018  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2753) took 1607 ms. Trace:
I20260709 12:28:05.229043  7658 rpcz_store.cc:276] 0709 12:28:03.608899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.784341 (+175442us) service_pool.cc:224] Handling call
0709 12:28:03.790537 (+  6196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.918940 (+128403us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:03.919302 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.231310 (+312008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.231368 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:04.231414 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.215594 (+984180us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:05.215680 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.215917 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2629,"prepare.queue_time_us":1144,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":3418,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8356,"spinlock_wait_cycles":294622464,"thread_start_us":311624,"threads_started":1,"wal-append.queue_time_us":482366}
W20260709 12:28:05.224064  7626 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.755s (SENT)
I20260709 12:28:05.235020  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.235751  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.222357  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.222720  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.242602  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.245011  7661 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:05.148015 14185 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:28:05.170514 13777 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:05.246088  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.246205  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:05.251979  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.252126  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:05.252452  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.252713  8412 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:05.183816 13786 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:05.252563  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.155910 13524 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.253480  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.254501  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.222237 14009 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:05.257220  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.258237  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.262135  8409 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:05.259373  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.259383  7659 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:05.261488  7655 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:05.268592  8415 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:05.246206 14149 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:05.290861  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.294639  7658 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:05.291440  7662 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [11.7-15.9]
I20260709 12:28:05.232805 14186 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.254703 13777 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:05.306739  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.307582  7804 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:05.309206  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.312587  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.313550  7657 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:05.258853 13786 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.320806  7653 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [14.9-16.10]
W20260709 12:28:05.325639  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3522986449997916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 12:28:05.326236  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3144986449997305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 12:28:05.292443 14115 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:28:05.326560  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2118986449995646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 12:28:05.327257  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:05.328306  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:05.330956  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1713986449999538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:28:05.331346  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1641986449999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
W20260709 12:28:05.331652  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1398986449999029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 12:28:05.331984  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:05.332206  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1290986449998854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 12:28:05.332963  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0615986449997763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 12:28:05.333312  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.333802  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.334430  7656 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:05.338070  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.339114  7657 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:05.332614 14069 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.330492 14149 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:05.343542 14149 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.344264 13796 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.299182 14009 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.342497 14069 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.347374  7808 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:05.348196  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.358788  7804 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:05.257184 14122 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:28:05.361074  7660 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:05.369544  7660 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:05.371325  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.371609  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.373201  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.326886 13786 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.380293  7658 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:05.381712  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.325426 14126 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:05.381052  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47384us
I20260709 12:28:05.380985  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.385166  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.388353  7657 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:05.347012 14113 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.391621  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.394086  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:05.395608  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 293875us
W20260709 12:28:05.379174 14173 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.002s
I20260709 12:28:05.397526  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.399909  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:05.402083  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.404603  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.405036  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.405476  7735 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:05.383704 14191 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:05.408798  7627 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.094s (SENT)
I20260709 12:28:05.409627  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.407169  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.393683 14145 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:05.395339 13846 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:05.413381  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.414527  7735 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:05.412297  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.419957  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:05.431911  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.432747  7735 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:05.401852 14180 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.403394 14129 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:05.430663  7656 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:05.402181 14173 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.435860  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.388870 14126 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.350520 14186 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.406635 14111 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:28:05.446273  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.411454 14192 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:05.508618  7730 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:05.436406  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 275207us
I20260709 12:28:05.467729  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.514533  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:05.436508  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 313044us
I20260709 12:28:05.421444 14110 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:05.509104  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73413.9us
I20260709 12:28:05.509189 13846 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:05.515817  7804 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:05.518571  7728 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:05.519758  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99933us
W20260709 12:28:05.516247  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 261843us
I20260709 12:28:05.518654  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.432065 14122 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:28:05.522714 12650 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:05.526121 12650 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.526515 12650 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.521008  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:05.523602  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:05.413090 14188 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:05.479724 13524 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:05.464775  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56013.7us
I20260709 12:28:05.434849 14067 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:05.446482 14191 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:05.526561  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:05.536206  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.096s (SENT)
I20260709 12:28:05.487775 14149 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.547461  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.547938  7807 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:05.557093  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.560665  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.564170  7728 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:05.435379 14145 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:05.564589  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.117s (SENT)
I20260709 12:28:05.535880 13796 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:05.566656  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.567597  7734 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:05.537813 14180 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:05.568485  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.573288  7812 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:05.535569 13897 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:05.555208 12650 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.412561 14009 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.522884 14165 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:28:05.576009  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43707.7us
I20260709 12:28:05.579969  7736 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
W20260709 12:28:05.580370  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4260574760000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.267s for a response
I20260709 12:28:05.525339 14105 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:28:05.581424  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1280) took 2315 ms. Trace:
I20260709 12:28:05.527956 14110 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.556452 14087 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.573197 14180 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.525396 14129 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.528673 14166 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:28:05.588268  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.571877 13846 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:28:05.589113 12650 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.589862  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:05.591593  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.804s (SENT)
I20260709 12:28:05.591799  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:05.588085 14149 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:05.541920 14192 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:28:05.593338  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:05.577618 14069 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.599772 14180 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.588373  7733 rpcz_store.cc:276] 0709 12:28:03.266074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.266308 (+   234us) service_pool.cc:224] Handling call
0709 12:28:03.288781 (+ 22473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.105607 (+816826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.581137 (+1475530us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54003b56d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:05.581362 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863593856}
I20260709 12:28:05.597945  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.587450 13897 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:05.629199  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.586297 13881 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:05.581818 14201 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:28:05.614428 14166 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:05.603808  7736 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.603977  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.580840  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.585254 13712 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:05.586004 13796 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.611198 14104 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:05.632404 14201 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:05.599154 14087 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:05.640475  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48929.2us
W20260709 12:28:05.640057  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58568.4us
W20260709 12:28:05.668072  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.542s (SENT)
I20260709 12:28:05.668962  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.670013  7734 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:28:05.671494  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 78245.1us
W20260709 12:28:05.672582  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.673488  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.673425 13897 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.674212  7733 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:28:05.666613 13762 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:28:05.668288 14201 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:05.676395  7733 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:05.677233 13762 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:05.677772 13762 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:05.678916 13762 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.680371 13762 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:05.676433 14201 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.682066 13786 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.682809 14201 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:05.666113 14206 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:28:05.683513  7809 raft_consensus.cc:2368] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 17.
I20260709 12:28:05.639859 14211 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:28:05.688138  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1811822939998819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25910000000000005s for a response
I20260709 12:28:05.670508  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.645481  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:05.694150  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1024904280002374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3352s for a response
I20260709 12:28:05.697103  7656 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:05.692394  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.700155  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:05.705224  7806 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:05.645462 14105 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:05.708422 13524 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:05.693027 14211 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:05.626050 13777 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.708616 13786 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:05.712975  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:05.791947  7804 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:28:05.713506 14185 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:05.713565  7653 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:05.714962 14145 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.665920 13881 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.687876 13712 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.818718 12650 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.689872 13846 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:05.831696 13846 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.832253 13846 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:05.836423 13880 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:05.819327 12650 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.839051 12650 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:05.690151 14165 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:05.633625 14166 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:05.727521  7736 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:05.728945  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3037) took 1152 ms. Trace:
W20260709 12:28:05.730219  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84825.2us
W20260709 12:28:05.746713  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46701.4us
I20260709 12:28:05.750149  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.854593  8409 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:05.751592 13524 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.751914  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:05.755798  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.760707  7729 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:28:05.718302 14173 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.761431 14069 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.863272 14069 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.765046 14139 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:05.716610 14110 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:05.775982  7656 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:05.775578 13796 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.868165  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.868957  7656 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:05.778163  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.751597 14088 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.781112 14111 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.697012 14122 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:05.781917  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.784231  7657 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:05.784586  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.784844  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.785480  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.786387 14211 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:05.747000 14113 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:05.788399 14202 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:28:05.788859  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.789328 14207 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:28:05.790818  7812 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:05.696817 14210 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:05.820078  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.827239 13712 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.841589  7810 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:05.844146  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.790133 14126 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:05.850545  7731 rpcz_store.cc:276] 0709 12:28:04.578793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.579073 (+   280us) service_pool.cc:224] Handling call
0709 12:28:05.715109 (+1136036us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54002495d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:05.728882 (+ 13773us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266915712}
I20260709 12:28:05.870316  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.871974  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.935024  7656 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:05.874523  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:05.875841  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:05.874161  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.936561  7735 raft_consensus.cc:2368] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17.
W20260709 12:28:05.936414  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50365.1us
I20260709 12:28:05.817322 14105 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:05.876238  7658 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:05.879367  7654 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:05.878808  7734 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:28:05.872920  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:05.897805  7626 leader_election.cc:400] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 17.
I20260709 12:28:05.944864  7626 leader_election.cc:403] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:28:05.884986 14115 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:28:05.894121  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 168576us
I20260709 12:28:05.902207  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3042) took 1256 ms. Trace:
I20260709 12:28:05.914863  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3073) took 1052 ms. Trace:
I20260709 12:28:05.925295  7737 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:05.925535  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.919401 12650 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:28:05.923754  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.816638 14185 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.873647 14087 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:05.929695  7803 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
W20260709 12:28:05.881311  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 184029us
I20260709 12:28:05.938732  7659 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:05.944942  7657 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.901010  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:05.971974  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 71128.3us
I20260709 12:28:05.967074  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.946768 14069 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:05.947836  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:05.942142  7653 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:05.870904 14145 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:05.953789  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:05.951694 14115 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:05.944864 14149 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:05.966126  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.967298  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:05.840437 13881 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.967633 12650 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:05.968444  7736 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:05.968798  7732 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:28:05.967145  7729 rpcz_store.cc:276] 0709 12:28:04.862790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.074771 (+211981us) service_pool.cc:224] Handling call
0709 12:28:05.914601 (+839830us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:05.914779 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449875840}
I20260709 12:28:05.928128  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.967203  7730 rpcz_store.cc:276] 0709 12:28:04.645721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.646351 (+   630us) service_pool.cc:224] Handling call
0709 12:28:05.901874 (+1255523us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001450d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:05.902114 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923230464}
I20260709 12:28:05.877293 14192 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.914778 14165 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:05.986220  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58249.3us
I20260709 12:28:05.986761  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.987063  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:05.944190 14211 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:28:05.987448  7732 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:05.870898 14111 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.001210  8406 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [14.13-15.14]
I20260709 12:28:05.871191 14122 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:05.959102 14126 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:06.003805  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:05.995360 14145 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:06.000523 14090 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:06.009686 14111 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:06.024174  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47241.9us
I20260709 12:28:06.010942 13524 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.013361  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:05.942145 13762 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:28:06.013423  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:06.016489  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:06.022981  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:05.982332 13897 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:28:06.040525  8411 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:06.026525  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.014611  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:06.015168 14180 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.043792 14180 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:06.045341  8412 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:06.031944  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:06.032140  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 112985us
I20260709 12:28:06.046645  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:06.002336 14069 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:06.035426  7730 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:06.037490  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.049619  7732 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:06.055588  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.033126 14166 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.062265 14166 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.062139 14145 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.064774 14145 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:06.066386 14218 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:06.067961 13777 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:06.068920 14105 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:05.985960 14192 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.025015 14126 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.040388 14111 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:05.997165 14211 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:06.090842 13762 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:28:06.091925  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60092.8us
W20260709 12:28:06.256579  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 304490us
I20260709 12:28:06.089468 14226 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:06.092795  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46253.6us
I20260709 12:28:06.093361 14180 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:06.095554 14110 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:28:06.096170  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77948.2us
I20260709 12:28:06.270668  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.033420 13997 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:28:06.100692  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.101231  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:06.260886  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:06.102414 14203 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:28:06.104348 13881 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.105983 14185 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.040241 14149 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 17.
W20260709 12:28:06.116801  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 242367us
I20260709 12:28:06.118089  8415 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:06.119181  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:06.200367  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.264765  7733 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:28:06.001075 14165 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.126259 14126 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:06.277711  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 176537us
I20260709 12:28:06.311435  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:06.278875  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 381211us
I20260709 12:28:06.253726 13771 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.313028 13771 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.253923 13897 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:06.286535  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:06.315025 13897 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.257025 13762 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:06.287815  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 291972us
I20260709 12:28:06.320629 13762 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.321147  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:06.322070 13762 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:06.322413  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:06.323290 14173 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:28:06.289820 14196 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:06.328547  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:06.332103 14165 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:06.333482 14188 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:06.282655 14224 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:28:06.282444 14199 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:06.338308 13881 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.339605 14122 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.304136 14211 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.328388 14110 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:06.475572 14071 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.482262 14110 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.351670 14220 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:28:06.303248 14111 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:28:06.287967 13786 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:06.257930 13846 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:06.356271  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.433698  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104851us
I20260709 12:28:06.355455  7728 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:28:06.453377  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 142174us
W20260709 12:28:06.453657  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 195506us
I20260709 12:28:06.450093 14210 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:06.454586  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 193461us
W20260709 12:28:06.464179  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 345017us
I20260709 12:28:06.324077 13897 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.382854 14215 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.480038 14231 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:06.483971 14071 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.450428 14173 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:06.460932 14188 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:06.541464  7811 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:06.495836  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.498322  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.495735  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:06.544122  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48503.1us
I20260709 12:28:06.547205  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.547998  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.549183  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:06.548589  7812 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:06.553889  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.466401 14199 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:06.556016 14199 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.556742  7812 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:06.557070  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.559751  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.466665 14225 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:06.560420 14225 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:06.561039 14225 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.561177  7812 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:06.528834  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:06.563846  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:06.530866 13897 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 12:28:06.543058  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47315.6us
I20260709 12:28:06.559607  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:06.567564  7808 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:06.507841  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 207996us
I20260709 12:28:06.493791 14122 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.434149 14196 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:06.575158  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.560465  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:06.563289  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:06.527815  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 241385us
W20260709 12:28:06.578022  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.578943  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:06.529318  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:06.577466  8411 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
W20260709 12:28:06.580955  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51732us
I20260709 12:28:06.472450 14149 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:06.570531  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72341.4us
I20260709 12:28:06.598201  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.529394 14211 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:06.569532 14188 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:06.554598 14173 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.524899 14210 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.608492  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:06.603600  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82535.5us
I20260709 12:28:06.575454 14220 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:28:06.618800  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.575848 14231 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.601608 13897 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:06.623682  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:06.532388 14090 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:28:06.621191  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:06.530725 13838 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:28:06.616184 14111 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:06.641044  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:06.641227  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:06.639137  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.640554  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:06.654410  7654 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:06.619294 14173 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.596433 14122 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:06.657739  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89218.2us
I20260709 12:28:06.661296  7778 leader_election.cc:400] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17.
I20260709 12:28:06.629494 14069 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:06.532874 14215 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.664878  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.615527 14165 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:28:06.654439 14240 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.646910 14111 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.666020  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:06.679100 14230 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 12:28:06.680629 14232 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:28:06.634552 14231 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.682548 14058 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.678894 14239 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:06.758764 14239 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.758590 14058 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.759356 14239 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:06.691479 14227 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:06.696023  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:06.702112  7805 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.001s
I20260709 12:28:06.761327 14058 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.761492 14227 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:06.761811  7805 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:06.677330 14129 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:06.764858  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:06.756599  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 139526us
I20260709 12:28:06.763037 14227 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.628324 14234 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
W20260709 12:28:06.757392  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 159321us
W20260709 12:28:06.764931  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 210679us
I20260709 12:28:06.695986 14111 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:06.768810  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 150323us
I20260709 12:28:06.774066  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.662322 14236 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 12:28:06.671689 14209 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:28:06.801950  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2640) took 1563 ms. Trace:
I20260709 12:28:06.802533  7811 rpcz_store.cc:276] 0709 12:28:05.238468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.604854 (+366386us) service_pool.cc:224] Handling call
0709 12:28:05.605270 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.571246 (+965976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.801422 (+230176us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54000681d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:06.801857 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152393600}
W20260709 12:28:06.775274  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.756266 14069 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:06.788689  7804 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:28:06.814030  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.815043  7804 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:06.814742 14069 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.783994  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:06.792771  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96864.4us
W20260709 12:28:06.826278 14113 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.902s	user 0.013s	sys 0.001s
I20260709 12:28:06.826686 14250 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.826869 14113 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:06.827343 14250 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:06.827790  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.828199 14201 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:06.829918  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.786595 13846 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:06.772944  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.750095 13881 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:06.830190 14250 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:06.788882 13897 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:06.831867  7810 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:28:06.847458  7810 pending_rounds.cc:85] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 9
I20260709 12:28:06.794034  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:06.795001 14253 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:06.768477 14238 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:28:06.775482 14237 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:06.833788  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:06.699975 14240 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:06.837754  7655 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:28:06.841861 14236 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:06.864243  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.839984  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:06.841022  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:06.870512  7657 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:06.846032  7736 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:06.831094  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.843112  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:06.832020  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:06.847816  7810 pending_rounds.cc:107] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:28:06.807736 14248 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:06.825322 14227 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:06.856418  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.831945 14246 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.833971 14129 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:06.832888  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58886.6us
I20260709 12:28:06.860069  7808 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:06.914788  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.842465 14234 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:06.861068 13846 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:28:06.866134  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:06.866421 14235 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:28:06.863885  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 198788us
W20260709 12:28:06.988533  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 133859us
I20260709 12:28:06.806762 14149 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:06.873688  7730 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:06.873814  7810 raft_consensus.cc:2889] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:06.873118  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 79185.7us
I20260709 12:28:06.875849  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:06.853264 14253 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:06.991437  7808 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:28:06.992121  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 116419us
I20260709 12:28:06.882323  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3107) took 1131 ms. Trace:
I20260709 12:28:06.993093  7734 rpcz_store.cc:276] 0709 12:28:05.750520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.790321 (+ 39801us) service_pool.cc:224] Handling call
0709 12:28:05.819288 (+ 28967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.944548 (+125260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.871887 (+927339us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b540037cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:06.882235 (+ 10348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135876864}
I20260709 12:28:06.994045  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:06.915292  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100581us
I20260709 12:28:06.907608 14248 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:06.967969  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1327) took 1177 ms (client timeout 1543 ms). Trace:
W20260709 12:28:07.001694  8412 rpcz_store.cc:269] 0709 12:28:05.790482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.790616 (+   134us) service_pool.cc:224] Handling call
0709 12:28:06.967682 (+1177066us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:06.967886 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275619072}
I20260709 12:28:06.968145  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:06.868347 14236 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:06.863054 14191 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:06.838793  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:06.989177  7811 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:06.966818  7657 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:06.915887 14105 raft_consensus.cc:2764] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 12:28:07.013898  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1032) took 1851 ms (client timeout 1989 ms). Trace:
W20260709 12:28:06.996721  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 199119us
W20260709 12:28:07.014380  7805 rpcz_store.cc:269] 0709 12:28:05.162675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.162844 (+   169us) service_pool.cc:224] Handling call
0709 12:28:07.013588 (+1850744us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:07.013822 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2197846272}
I20260709 12:28:06.997249 14254 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:06.997877 14234 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:06.998042  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:06.988013  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73320us
I20260709 12:28:07.009424  7654 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:28:06.993599  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:07.028064  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.012804  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3090) took 1090 ms. Trace:
I20260709 12:28:06.958355 14129 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.016011  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2635) took 1784 ms. Trace:
I20260709 12:28:07.032598  7803 rpcz_store.cc:276] 0709 12:28:05.231881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.248868 (+ 16987us) service_pool.cc:224] Handling call
0709 12:28:05.249312 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.977861 (+728549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.015811 (+1037950us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54001324d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:07.015960 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564020992}
I20260709 12:28:06.960093 14259 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.036360  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.014967  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:07.029868  7733 rpcz_store.cc:276] 0709 12:28:05.922564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.942539 (+ 19975us) service_pool.cc:224] Handling call
0709 12:28:07.012536 (+1069997us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:07.012729 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654831872}
I20260709 12:28:07.011619 13777 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:07.000205 13712 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:07.042284  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:07.010823 14248 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:07.045190 12650 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:07.045589 14211 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:07.038981 14149 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:07.047052  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:07.047199 14258 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.047878 14264 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:07.010056 14260 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:07.050115  7733 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:07.017590 14191 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:07.054814  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.055783 14252 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:28:07.048839 13777 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.064833  7811 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:07.037065 14129 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:07.072274  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.076613 12650 consensus_queue.cc:1243] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:07.081423  7733 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:07.129215  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57051.7us
I20260709 12:28:07.044591 14254 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:07.014415 14231 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:06.853659 14257 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.240s
I20260709 12:28:07.130388  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:07.082705 14258 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.079900 14264 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.078374 14255 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:07.140453 14165 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:07.078351 14209 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.143929 14071 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:06.864846 14237 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.147884 13771 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:07.150337 13712 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.294905 13712 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:07.160979 14180 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:07.299660 14180 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:07.300928 14180 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.302284 14180 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:07.172102 14222 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:07.303444 14267 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.178331 14192 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.184880  7729 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:07.304529 14267 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.188508 14196 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.306327 14196 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:07.215806 13846 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:28:07.245517 14254 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:07.307974 13846 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:07.308627 14254 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:07.200523  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.201105  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 202428us
I20260709 12:28:07.313019 14258 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:07.314283 12650 consensus_queue.cc:1175] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 13 index: 12 } }
W20260709 12:28:07.211272 14245 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.545s	user 0.004s	sys 0.000s
I20260709 12:28:07.313725 14191 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.315583 14260 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.046533 14242 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:28:06.999851 14238 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:28:07.071980 14122 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.079209 14218 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:07.316250 14260 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:07.317544 13762 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.995s	user 0.011s	sys 0.004s
W20260709 12:28:07.233265  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 153639us
W20260709 12:28:07.240841  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.958114 14246 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.139065  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1326) took 1360 ms. Trace:
I20260709 12:28:07.319505  8413 rpcz_store.cc:276] 0709 12:28:05.779111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.785076 (+  5965us) service_pool.cc:224] Handling call
0709 12:28:05.785391 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:06.014355 (+228964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.014392 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.014984 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.312583 (+297599us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:06.312632 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.138528 (+825896us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400712ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:07.138608 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:07.138669 (+    61us) raft_consensus.cc:1685] finished
0709 12:28:07.138685 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.138994 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33937,"raft.queue_time_us":16549,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5783,"spinlock_wait_cycles":525642624,"thread_start_us":297126,"threads_started":1,"wal-append.queue_time_us":512868}
I20260709 12:28:07.320953 14246 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.229929  7733 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.321784  7733 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:07.322160 14260 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.322336 14240 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:07.322620 14225 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.000s
I20260709 12:28:07.251271 14264 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:07.258081 14165 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:07.324620 14165 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:07.088099 14166 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.025s	user 0.010s	sys 0.005s
W20260709 12:28:07.249635  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119333us
I20260709 12:28:07.327746 14166 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:28:07.327960  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.328584 14166 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:28:07.329262 14166 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:07.278824 14215 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:07.247064  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 193832us
I20260709 12:28:07.331902 14215 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.285168 14255 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:07.227314 13777 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:07.333498 14255 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.336002 14274 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.336580 14215 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.337160 14237 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.337116 14215 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:07.338228 14252 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:28:07.039562 13897 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.339654 14215 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.283726 14236 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.341078 14215 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:07.292222 14262 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 12:28:07.293932 14261 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:07.303718 14222 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:07.317022 14218 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.166808 14224 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.336938 14274 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.285876  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2649) took 1589 ms. Trace:
I20260709 12:28:07.412317 14257 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.413203 14257 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.414294 14274 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.326822 14165 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:07.292366 14266 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.415263 14266 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.417136 14257 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.417872 14266 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.313867  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.418491 12650 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:28:07.418972  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 105178us
I20260709 12:28:07.420595 14108 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:07.290656 14209 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.425583 14209 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.373653 13838 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:28:07.429999 14257 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.430914  7737 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:28:07.344516  7730 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:28:07.257694 14139 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:07.316238 14191 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:07.366900 14145 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.302s	user 0.015s	sys 0.000s
I20260709 12:28:07.228525 14211 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:07.356578 14218 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:07.359458 14215 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.461886 14215 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.362965 14224 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.466106 14224 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:07.364993 14199 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.000s
I20260709 12:28:07.314036 14249 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
I20260709 12:28:07.467147 14199 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.470100 14199 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.471184  7806 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:07.471736 13777 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.473992  7812 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:07.474390 13777 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:07.308620 13846 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:07.475906 14267 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.475848 13846 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.366072 14253 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.477049 13846 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:07.065570 14259 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.478080 14253 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:07.479353 14259 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.258638 14069 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:07.366990 14173 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.700s	user 0.008s	sys 0.002s
I20260709 12:28:07.480114 14069 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.480199 14173 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.343529 14225 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:07.343680 14126 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.997s	user 0.011s	sys 0.000s
W20260709 12:28:07.345889 14226 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.066s	user 0.010s	sys 0.005s
I20260709 12:28:07.483485 14226 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:07.254119 14149 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.484566 14149 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.485571 14225 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.482524 14173 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.488013  7728 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:28:07.487738 14248 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.489540 14126 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.490326 14248 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:07.353722 14237 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.378089 14236 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:07.491688  7731 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:28:07.353410 14111 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
I20260709 12:28:07.395553 14222 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.492825 14222 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:07.492578 14111 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.396791 13997 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:07.491711 14237 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.493891 13997 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:07.393391 14115 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.494985 13997 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.495863 14237 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.496028 14115 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:07.368952 14188 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.739s	user 0.007s	sys 0.004s
I20260709 12:28:07.499540 14188 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.502676 14188 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.412721  7809 rpcz_store.cc:276] 0709 12:28:05.696050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.788877 (+ 92827us) service_pool.cc:224] Handling call
0709 12:28:05.789324 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.947801 (+158477us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:06.258467 (+310666us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.412908 (+154441us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.449157 (+ 36249us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.449347 (+   190us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:06.449388 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.803838 (+354450us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:06.803921 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:07.285418 (+481497us) raft_consensus.cc:1685] finished
0709 12:28:07.285549 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.285782 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42459,"prepare.queue_time_us":159108,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":66,"raft.queue_time_us":63825,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":31138,"spinlock_wait_cycles":335937920,"thread_start_us":383885,"threads_started":2,"wal-append.queue_time_us":350858}
I20260709 12:28:07.503113 13712 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.504285  7736 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:07.291357 14272 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:07.505281 14272 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:07.432444  7730 pending_rounds.cc:85] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:07.507426 14126 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.506650 14272 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.507526  7730 pending_rounds.cc:107] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:28:07.508888  7730 raft_consensus.cc:2889] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:07.509074 14272 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:07.511003 14262 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.511611 14149 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.495079 14111 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:07.430889 14257 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.446496  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:07.513000  7729 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:07.445320 14139 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.513980 14139 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:07.514519 14257 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.446453  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.216233 14234 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:07.515940  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69626.8us
I20260709 12:28:07.518123 14111 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:07.517864 14129 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.518611 14111 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.519057 14111 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:07.520309 14234 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:07.477180 14239 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.000s
I20260709 12:28:07.526393 14239 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.330750 14166 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:28:07.496666  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.648s (SENT)
I20260709 12:28:07.531864  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.532994 13712 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:07.533345  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.810s (SENT)
I20260709 12:28:07.534789 14149 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:07.536387 14239 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.482471 14069 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.496326 13997 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:07.535410 14257 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.538201 14257 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.511668 14262 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.541335 14257 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.517926  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.544680  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:07.545418 12650 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:28:07.439971  7733 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:28:07.546485  7733 pending_rounds.cc:85] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 13
I20260709 12:28:07.546924  7733 pending_rounds.cc:107] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 12:28:07.547387  7733 raft_consensus.cc:2889] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:07.547652  7737 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:28:07.549242  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:07.550269  7730 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:07.543159 14069 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:07.551501 14069 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:07.552855 14262 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.553711 14069 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.554340 14129 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:07.559985 14239 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:28:07.569653 14210 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.932s	user 0.010s	sys 0.003s
I20260709 12:28:07.570410 14210 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.573604 14249 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:07.575543 13712 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:07.576161 13712 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:07.577235  7729 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:07.577263 13712 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.582896 13712 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:07.583680 13777 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:28:07.592221  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.584539 14249 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:28:07.591811  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73960.3us
I20260709 12:28:07.594053 14249 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:07.588907 14129 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.594631 14249 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:07.595042 14129 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.578732  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:07.588860 14239 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:07.596369 14249 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:07.598770 14227 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.684s	user 0.009s	sys 0.001s
I20260709 12:28:07.593715 14139 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.599125 14249 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:07.599632  7734 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:07.602147 14139 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.597986  7737 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:07.590035  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:07.597388 14218 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.595140  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:07.631790  7730 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:07.648770  7729 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:28:07.649597  7729 pending_rounds.cc:85] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 7
I20260709 12:28:07.649937  7729 pending_rounds.cc:107] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 13.8
W20260709 12:28:07.650031  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54979.5us
I20260709 12:28:07.650326  7729 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:07.653403  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.654848  7662 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:28:07.672829  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:07.673817  7657 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:07.676558  8379 connection.cc:256] RPC call timeout handler was delayed by -1.7484327489996758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
I20260709 12:28:07.481922 14264 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:07.680960  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.683544  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.685786  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.686837  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:07.688678  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.697216  7737 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:07.599125 14129 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.698462  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:07.711576  8413 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 11: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:07.712543  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:07.708967  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.707087  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:07.714491  8409 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:07.710285  8407 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 12:28:07.638024 14239 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:07.711781  8412 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:07.709184  7735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.018s	sys 0.001s
W20260709 12:28:07.720470  7735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.019s	sys 0.002s
I20260709 12:28:07.717777  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:07.723392  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:07.726066  8415 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:07.723829  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.729446  8412 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:07.678756 14196 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:07.728346  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:07.741436  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.706230 14115 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.753183 14115 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.649235 14248 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:28:07.678001 14199 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
W20260709 12:28:07.764812  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.725368 14239 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.733522 14058 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:28:07.762843  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:07.677762 14126 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:28:07.759656  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.005s	sys 0.002s
W20260709 12:28:07.782511  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.002s	sys 0.006s
W20260709 12:28:07.783358  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.003s	sys 0.006s
W20260709 12:28:07.785771  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 787744us
I20260709 12:28:07.788182  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:07.758819 14210 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:07.770327 14239 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:07.772317  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.006s	sys 0.003s
I20260709 12:28:07.774418 14273 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:07.801493  7775 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.018s	sys 0.045s
W20260709 12:28:07.855937  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67842.3us
W20260709 12:28:07.856602  7775 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.018s	sys 0.045s
W20260709 12:28:07.856959  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 593489us
W20260709 12:28:07.857395  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.858608  7657 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:07.780005 14108 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:28:07.814548  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.786679 14110 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:07.789661 14129 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.869714  7661 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:07.772585 14088 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:28:07.838052  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 73912.6us
I20260709 12:28:07.843020  7804 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:28:07.836664 14139 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:07.892614  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.893448  7805 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:28:07.893663  8378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.019s
W20260709 12:28:07.894096  8378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.000s	sys 0.020s
I20260709 12:28:07.837011 14115 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:07.848443  8379 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.803s (SENT) [suppressed 19 similar messages]
I20260709 12:28:07.848912  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:07.851581  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.004s
I20260709 12:28:07.862134  7657 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:07.859854  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.899191  7736 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:07.869994  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.830508  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:07.873534  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:07.900763  7737 raft_consensus.cc:2410] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 17 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
W20260709 12:28:07.899187  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.008s	sys 0.004s
W20260709 12:28:07.901767  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 581241us
I20260709 12:28:07.889904  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.902588  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:07.897285  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.891517  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.900544  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.905992  7657 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:07.907282  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.907996  7657 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:07.903007  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:07.913398  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:07.914846  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.915740  7805 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:07.915786  7653 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:28:07.916752  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.917569  7805 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:07.917729  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.919634  7660 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:28:07.859859 14210 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:07.923580  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:07.924181  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50728.9us
I20260709 12:28:07.863391 14273 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:07.926329  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.930897  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.935406  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.937418  7653 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:07.859046 14218 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.938761  8415 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
W20260709 12:28:07.956033  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.931953  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.957176  7805 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:07.925793  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.938645 14258 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:07.942265  7700 leader_election.cc:400] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:07.958324  7700 leader_election.cc:403] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:28:07.943212 14279 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:28:07.942915 14166 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:07.973052  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.974671  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.975458  7737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
W20260709 12:28:07.943878 14277 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:28:07.943883 14108 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:07.993487 14108 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.952726  7656 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:28:07.918913 14139 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:07.996001  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.996743  7656 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:07.997681  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:07.963096  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.963409  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:07.999776  7811 raft_consensus.cc:2368] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:07.965282  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.966311  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.970152  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.975051  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.899341 14264 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:07.980557  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.986397  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.009019  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.992274  7658 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: [11.7-15.9]
I20260709 12:28:08.009843  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.946722 14254 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.013064  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.003629  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.003631  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:07.998834  7653 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:08.002499  8411 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:28:08.006008  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67022.8us
I20260709 12:28:08.014657  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.007174  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:07.933315 14266 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:08.021610  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.895476 14110 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.016392  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.030224  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.031242  8411 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:08.015873  7657 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.017714  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.018536  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:07.997596 14139 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:08.020507 13846 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:08.020330 14258 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:08.007984  7736 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:08.023808  7728 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:08.020069  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:08.054095  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:08.015901  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.056839  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:08.023048  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:07.898147 14129 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.017022  7810 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:08.059888  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.031857  7653 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 13: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:08.061312  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.035533  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2655) took 1255 ms. Trace:
I20260709 12:28:08.062649  7804 rpcz_store.cc:276] 0709 12:28:06.780482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.815827 (+ 35345us) service_pool.cc:224] Handling call
0709 12:28:06.816166 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.873093 (+1056927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.035161 (+162068us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003f5054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:08.035462 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360880512}
I20260709 12:28:08.064455  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.037066  7731 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 16. (term mismatch)
I20260709 12:28:08.065466  7731 pending_rounds.cc:85] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 15
I20260709 12:28:08.065817  7731 pending_rounds.cc:107] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 17.16
I20260709 12:28:08.066143  7731 raft_consensus.cc:2889] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.054953  7661 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:08.036983  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.068327  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.069168  7661 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:08.032513  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1299) took 1414 ms. Trace:
I20260709 12:28:08.065217  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:08.072365  7735 rpcz_store.cc:276] 0709 12:28:06.618064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.620204 (+  2140us) service_pool.cc:224] Handling call
0709 12:28:06.620682 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:06.889054 (+268372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.889098 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:07.007292 (+118194us) log.cc:844] Serialized 52 byte log entry
0709 12:28:07.720945 (+713653us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:07.963187 (+242242us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.029379 (+ 66192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.029487 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.029537 (+    50us) raft_consensus.cc:1685] finished
0709 12:28:08.029551 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.032434 (+  2883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65715,"raft.queue_time_us":124772,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":148,"spinlock_wait_cycles":3993984,"thread_start_us":1088225,"threads_started":3,"wal-append.queue_time_us":818574}
I20260709 12:28:08.065287  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:08.080803  7656 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:08.081012 14087 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:08.085218  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:08.062153  8411 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:08.094863  7657 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:08.096596  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.098363  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:08.086818  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.099823  7653 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:08.112421  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46503.8us
I20260709 12:28:08.112881  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:08.113603  7660 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.115163  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:08.118062  7732 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:28:08.119084  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:08.118520  7732 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:28:08.120985  7732 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.110848 14227 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.122855 14166 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:08.123540 13997 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:08.123430 14258 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.124127 14166 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:08.124234 13997 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:08.124656 14166 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:08.113391  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:08.126523  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:08.031657 14283 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.134421  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69313.5us
I20260709 12:28:08.139894  7656 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:08.141507  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:08.143139  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.144006  8413 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:08.125653 14281 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:28:08.146309 14281 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:08.146791 14281 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:08.147912 14281 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.123459 14266 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.149973 14266 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.152526 14266 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:08.153864  7660 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:08.155753  7660 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:28:08.159148  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:08.161702  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.162775  7660 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:08.163343  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.164261  8405 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:28:08.165724  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.167068  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:08.125695 14166 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.168903 14166 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.170584  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:08.143816  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.171526 14264 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:08.171924 14264 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.172295 14264 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.173542  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:08.174247  7807 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:08.174576  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.060498 14253 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:08.175457 14139 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:08.175913 14139 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.083679 14255 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:28:08.176898  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:08.177134  7662 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:28:08.177734 14115 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:08.178141 14115 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:08.178544  7662 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:28:08.037848 13846 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:08.020104 14254 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.172178  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.187342  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
W20260709 12:28:08.181847  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.174271  8411 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.176326 14139 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.190943  8406 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:08.188917  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:08.185163  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46380.3us
I20260709 12:28:08.192862  7737 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:08.165601  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:08.193442  7737 raft_consensus.cc:614] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:08.183714  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:08.142299  8409 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:28:08.184868  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.133424 14288 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:28:08.207471  8414 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:08.210578  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:08.211853  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.210510  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:08.216895  8408 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:28:08.218130  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.220409  7658 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:08.217665  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:08.181115 14234 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:08.178546 14115 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:08.237880 14129 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:08.238732 14129 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:28:08.187757 14272 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.006s
I20260709 12:28:08.140865 14249 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:08.239974 14249 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.240453 14249 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:08.241297  8413 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:08.216760 14199 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:28:08.244020  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.244752 14255 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:08.245376 14255 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.245990  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.239156 14129 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.251995 14129 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:08.227865  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.254263  7806 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:08.244755  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:08.255326 14088 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:08.125353 13997 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.227591  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:08.260154  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.260386  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:08.227700  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:08.235826 14234 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:08.229960 13777 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:08.184259 14188 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:08.229835  7803 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:28:08.180260 14283 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.263468  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.270370  7653 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:28:08.272334  8408 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:28:08.274268  7804 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:08.280608  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:08.281514  8413 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:08.283500  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.290336  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:08.291169  8414 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.294049 14283 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:08.295181 14234 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.297389  8415 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:08.297706 14115 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.300177  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:08.301226  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.226502 14218 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:08.302707  8409 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24790f3be4e8465581029a2e0d526117"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:28:08.304622  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:08.305495  7654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.000s	sys 0.026s
W20260709 12:28:08.327834  7654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.026s
W20260709 12:28:08.269731 14111 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.000s
I20260709 12:28:08.269409 14199 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.337545  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.340029  7803 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:08.126343 14273 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.287350 14088 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:08.323993  8414 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:08.324031  7811 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:08.304744  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.328722  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.330054 13846 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.352293 14110 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:08.352569 13846 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.330988  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.357271  8406 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:08.358446  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53829.6us
I20260709 12:28:08.359445 14278 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:08.360553 14254 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.351622  7737 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:08.365743 14265 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:28:08.366330  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.367014  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:08.367965  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:08.368620 14196 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.000s
I20260709 12:28:08.369122 14196 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:08.369788  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:08.370195  8407 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:08.373121 14264 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.374287  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:08.375947  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:08.376133 13712 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.738s	user 0.014s	sys 0.000s
I20260709 12:28:08.377297  7810 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:28:08.377923 14255 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:08.346537 14184 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:28:08.298753 14188 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.336375 14253 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:28:08.381126 14188 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.381747 14188 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:08.385375 14248 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:08.344345 14111 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:08.387866  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51022.4us
I20260709 12:28:08.388892 14115 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.389593 14115 raft_consensus.cc:2764] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:28:08.346532 14088 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.389817  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:08.390362 14088 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:08.390461 14115 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.390861 14087 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.400574  8413 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:08.402683  8411 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:08.306440  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.403960  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:08.327296 14218 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.423207  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:08.423208  8405 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:08.413578  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:08.415658  7737 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:08.420212  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.449198  7655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.016s	sys 0.025s
I20260709 12:28:08.389626 14111 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:08.476468  7655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.016s	sys 0.026s
I20260709 12:28:08.435716  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.462575  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:08.486740  7737 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:08.428747 14239 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:28:08.490809 14239 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:08.492615  7812 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:08.460424 14284 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 12:28:08.437574  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45411.7us
I20260709 12:28:08.505370  7810 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:08.429134 14218 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:08.509855  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.406584 14251 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:08.502810  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:08.552385  7656 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:28:08.429266 14188 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:08.510939  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:08.526328  7737 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:08.531360  7658 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [12.9-14.10]
W20260709 12:28:08.556031  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45180.4us
I20260709 12:28:08.557484  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:08.537817  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.422829 14139 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:08.546455  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.513423  7812 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:28:08.391099 14115 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.557349  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:08.531508  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.563171  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:08.564205  7811 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:08.564507  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.567860  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.559128  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.561558  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:08.569510  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.568143 14227 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.569947  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.388892 14248 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 12:28:08.462827 14087 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:28:08.569420  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:08.576051  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:08.579824  7653 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:08.581619  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:08.559546 14184 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:08.565658 14111 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.558671 14188 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:08.575001 14255 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:08.588935  7737 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:08.569561 14166 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.571126 14267 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:28:08.593135  8379 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.014s	sys 0.010s
W20260709 12:28:08.594005  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.595693  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:08.648435  8415 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:08.553011 14239 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:08.581359 13897 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.551658 14251 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:08.647809  8379 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.014s	sys 0.011s
I20260709 12:28:08.661073  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:08.664348  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 815983us
I20260709 12:28:08.626226  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:08.596361 14282 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:08.665020  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:08.607741 14188 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.599828 14227 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:08.671053  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44954.9us
I20260709 12:28:08.605921 14184 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:08.609198 14254 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 12:28:08.675421 14210 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.670804 14255 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:08.606801 14249 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.688133 14281 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.606416 14111 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.681116 14282 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:08.711910 14264 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.709582 14210 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.757053 14264 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.761840 14115 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.750355 14129 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:08.784143 14139 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.792645 14264 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:08.778412 14165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.016s	sys 0.025s
W20260709 12:28:09.088136 14165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.017s	sys 0.026s
W20260709 12:28:08.990219  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.009s	sys 0.004s
W20260709 12:28:09.090492  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.010s	sys 0.004s
W20260709 12:28:09.093297  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 881458us
W20260709 12:28:08.991019  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.001s	sys 0.009s
W20260709 12:28:09.094889  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.002s	sys 0.009s
W20260709 12:28:09.095650  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 885088us
W20260709 12:28:08.992341  7659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.001s	sys 0.009s
W20260709 12:28:09.099637  7659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.001s	sys 0.009s
W20260709 12:28:08.993571  7728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.009s	sys 0.004s
W20260709 12:28:08.597469  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.996851  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:09.107056  7737 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:09.001706  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1181) took 1339 ms. Trace:
I20260709 12:28:08.598618  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:09.116281  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 517618us
W20260709 12:28:08.827306 14108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.047s
I20260709 12:28:09.117726 14258 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:09.118518 14258 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:09.119095 14258 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:09.120239  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:09.121343  7653 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:28:09.004053  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.005579  7733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.004s	sys 0.028s
W20260709 12:28:09.123395 14234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.009s	sys 0.025s
W20260709 12:28:09.006387  7734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.396s	user 0.008s	sys 0.027s
W20260709 12:28:09.123961 14234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.009s	sys 0.025s
W20260709 12:28:09.007041  7735 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.007s	sys 0.016s
W20260709 12:28:09.124529  7735 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.008s	sys 0.016s
W20260709 12:28:09.008781  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.125837 14249 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:09.012275  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 347229us
W20260709 12:28:09.013450  7809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.014s	sys 0.018s
W20260709 12:28:09.101490  7728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.010s	sys 0.004s
W20260709 12:28:09.131399  7809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.015s	sys 0.018s
I20260709 12:28:09.133489  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2661) took 1180 ms. Trace:
I20260709 12:28:09.014546  7808 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:28:09.134536  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3098) took 1098 ms. Trace:
I20260709 12:28:09.016906  7806 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 16. Ops: []
W20260709 12:28:09.017135  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.018289 12650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.018s	sys 0.025s
W20260709 12:28:09.021037 13997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.011s	sys 0.007s
W20260709 12:28:09.027611 14058 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.016s	sys 0.002s
I20260709 12:28:08.825739 14251 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:08.777405 14255 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:08.893483 14271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.015s	sys 0.043s
I20260709 12:28:09.171798  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:09.172681 14271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.016s	sys 0.043s
I20260709 12:28:09.174533 14264 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:08.805932 14282 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:08.845932 14291 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.128s	user 0.000s	sys 0.002s
I20260709 12:28:08.813346 14139 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.181177  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.183935  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3148) took 1460 ms. Trace:
I20260709 12:28:09.184600  7729 rpcz_store.cc:276] 0709 12:28:07.723163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.723330 (+   167us) service_pool.cc:224] Handling call
0709 12:28:07.753452 (+ 30122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.753958 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.753997 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.754672 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.228835 (+474163us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:08.228903 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.229010 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.183363 (+954353us) raft_consensus.cc:1685] finished
0709 12:28:09.183409 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.183698 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37426,"raft.queue_time_us":4587,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4943,"thread_start_us":473652,"threads_started":1,"wal-append.queue_time_us":924856}
I20260709 12:28:08.771720 14184 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:09.189690  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.932845 14293 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:28:09.192309  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.192773  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:09.193333  8414 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.799533 14115 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.204142 14115 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.207914  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1183) took 1533 ms. Trace:
I20260709 12:28:09.209067 14087 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:09.209453 14166 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:28:09.209923 14087 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:09.149827 13997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.012s	sys 0.007s
W20260709 12:28:09.052699  8408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.005s	sys 0.007s
W20260709 12:28:09.069595  8412 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.008s	sys 0.013s
W20260709 12:28:09.079378  7731 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.008s	sys 0.010s
I20260709 12:28:09.134135  7809 rpcz_store.cc:276] 0709 12:28:07.953026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.002137 (+ 49111us) service_pool.cc:224] Handling call
0709 12:28:08.024944 (+ 22807us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.058281 (+ 33337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.058318 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.071767 (+ 13449us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.132558 (+1060791us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:09.132680 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.132888 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.132992 (+   104us) raft_consensus.cc:1685] finished
0709 12:28:09.133012 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.133287 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37186,"raft.queue_time_us":38079,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":31507,"thread_start_us":986828,"threads_started":1,"wal-append.queue_time_us":785543}
W20260709 12:28:09.211905  7731 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.008s	sys 0.011s
W20260709 12:28:09.079864  7732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.006s	sys 0.004s
I20260709 12:28:09.082819 14273 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.113049 14111 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.108712  7654 rpcz_store.cc:276] 0709 12:28:07.662009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.662254 (+   245us) service_pool.cc:224] Handling call
0709 12:28:07.662911 (+   657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.663491 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.663621 (+   130us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.697414 (+ 33793us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.328284 (+630870us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:08.328336 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.328421 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.000756 (+672335us) raft_consensus.cc:1685] finished
0709 12:28:09.000848 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.001334 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":254,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":28189,"thread_start_us":572660,"threads_started":1,"wal-append.queue_time_us":991914}
W20260709 12:28:09.123590  7733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.357s	user 0.005s	sys 0.029s
W20260709 12:28:09.124063  7734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.514s	user 0.008s	sys 0.027s
I20260709 12:28:09.010855  7736 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
W20260709 12:28:09.136273 12650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.018s	sys 0.026s
I20260709 12:28:09.021539 14224 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:28:09.137267  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.002s	sys 0.006s
W20260709 12:28:09.118059 14108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.001s	sys 0.047s
I20260709 12:28:09.147249 14278 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:09.152390 13897 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.151552  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.007s	sys 0.000s
I20260709 12:28:09.157600  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.157776 14199 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:09.158936 14129 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:09.159991  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.160605 14267 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:09.165609  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.149593  7626 leader_election.cc:400] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:09.168898 14210 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.135274  7728 rpcz_store.cc:276] 0709 12:28:08.059198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.072039 (+ 12841us) service_pool.cc:224] Handling call
0709 12:28:08.083697 (+ 11658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.084034 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.084076 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.126180 (+ 42104us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.132226 (+1006046us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:09.133097 (+   871us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.133587 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.133778 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.133873 (+    95us) raft_consensus.cc:1685] finished
0709 12:28:09.133896 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.134391 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3999,"raft.queue_time_us":3256,"thread_start_us":912257,"threads_started":1,"wal-append.queue_time_us":769264}
W20260709 12:28:09.147140 14058 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.017s	sys 0.003s
I20260709 12:28:09.202332  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:09.211489  8408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.005s	sys 0.007s
I20260709 12:28:09.182675 14139 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:09.211594  8412 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.009s	sys 0.013s
W20260709 12:28:09.261020  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.006s	sys 0.006s
I20260709 12:28:09.253577  7626 leader_election.cc:403] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:28:09.215519  7732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.007s	sys 0.005s
W20260709 12:28:09.191272  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.003s	sys 0.020s
I20260709 12:28:09.242906  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:09.234949  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.140985 14249 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.233124 14088 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.842s	user 0.015s	sys 0.005s
W20260709 12:28:09.274562  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.006s	sys 0.020s
W20260709 12:28:09.270558  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.012s	sys 0.001s
I20260709 12:28:09.208660  7655 rpcz_store.cc:276] 0709 12:28:07.674647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.674831 (+   184us) service_pool.cc:224] Handling call
0709 12:28:07.715711 (+ 40880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.715966 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.716019 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.716554 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.477008 (+760454us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:08.477051 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.477295 (+   244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.207288 (+729993us) raft_consensus.cc:1685] finished
0709 12:28:09.207351 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.207668 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":710597,"threads_started":1,"wal-append.queue_time_us":1060523}
I20260709 12:28:09.273756 14264 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.242197  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:09.285353 14258 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.276140 14264 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.288441 14115 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.289733  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:09.290000 14139 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.290297  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.287061 14267 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.220163 14165 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:28:09.299921 14088 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:09.301237  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 108673us
I20260709 12:28:09.220710 14251 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.315538  8407 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:09.326370  7658 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:09.259614 14224 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.234805 14319 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.321622  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3188) took 1450 ms. Trace:
I20260709 12:28:09.312762  8406 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:09.320580  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.317627  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:09.328349  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3145) took 1719 ms. Trace:
I20260709 12:28:09.329236  7808 rpcz_store.cc:276] 0709 12:28:07.879674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.879978 (+   304us) service_pool.cc:224] Handling call
0709 12:28:07.880575 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.135066 (+1254491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.321288 (+186222us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003f4654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:09.321562 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604894464}
I20260709 12:28:09.366459  7737 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
W20260709 12:28:09.320067 14294 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.599s	user 0.002s	sys 0.004s
I20260709 12:28:09.325851 14264 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.322975 14088 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.374838  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:09.346316 14258 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.328613 14224 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:09.380532  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 56680.3us
W20260709 12:28:09.381603  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 282757us
I20260709 12:28:09.321940 14267 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.349398  7734 rpcz_store.cc:276] 0709 12:28:07.608562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.608722 (+   160us) service_pool.cc:224] Handling call
0709 12:28:07.609103 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.609544 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.609572 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.610081 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.325669 (+1715588us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:09.325794 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.325909 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.325985 (+    76us) raft_consensus.cc:1685] finished
0709 12:28:09.325995 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.328266 (+  2271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1946,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":3722,"thread_start_us":1642421,"threads_started":1,"wal-append.queue_time_us":1227213}
I20260709 12:28:09.396351  8411 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:09.390300  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3107) took 1114 ms. Trace:
I20260709 12:28:09.397603 14129 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:09.397996  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:09.398908  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:09.404801 14251 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.405395 14251 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:09.405891 14251 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:09.407382 13846 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.996s	user 0.008s	sys 0.022s
I20260709 12:28:09.409104 14087 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.410578  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2800) took 1465 ms. Trace:
I20260709 12:28:09.411160  7660 rpcz_store.cc:276] 0709 12:28:07.944779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.165820 (+221041us) service_pool.cc:224] Handling call
0709 12:28:08.166145 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.179383 (+ 13238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.179413 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.185740 (+  6327us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.301671 (+1115931us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:09.301797 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.301972 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.410205 (+108233us) raft_consensus.cc:1685] finished
0709 12:28:09.410228 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.410471 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3514,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"thread_start_us":987179,"threads_started":1,"wal-append.queue_time_us":1178005}
I20260709 12:28:09.412505  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3152) took 1296 ms. Trace:
I20260709 12:28:09.413079  7732 rpcz_store.cc:276] 0709 12:28:08.115839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.117490 (+  1651us) service_pool.cc:224] Handling call
0709 12:28:08.117854 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.121774 (+  3920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.121827 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.239790 (+117963us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:08.240232 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.373003 (+1132771us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:09.393872 (+ 20869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.412036 (+ 18164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.412122 (+    86us) raft_consensus.cc:1685] finished
0709 12:28:09.412140 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.412431 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23913856,"thread_start_us":1030257,"threads_started":1,"wal-append.queue_time_us":716378}
I20260709 12:28:09.423813  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3150) took 1700 ms. Trace:
I20260709 12:28:09.423902 14253 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:09.424631  7733 rpcz_store.cc:276] 0709 12:28:07.723844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.725769 (+  1925us) service_pool.cc:224] Handling call
0709 12:28:07.726182 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.728853 (+  2671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.728886 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.729376 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.378233 (+1648857us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:09.389318 (+ 11085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.423397 (+ 34079us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.423456 (+    59us) raft_consensus.cc:1685] finished
0709 12:28:09.423471 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.423733 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2123,"raft.queue_time_us":3004,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":25360,"thread_start_us":1487182,"threads_started":1,"wal-append.queue_time_us":1110364}
W20260709 12:28:09.427635  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 116478us
I20260709 12:28:09.427824  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:09.430143  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:09.431509 14129 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:09.432220 14129 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.432736 14129 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:09.348278 14273 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.437845 14319 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.438890 14166 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:09.440177 14166 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:09.436266  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.440982 14166 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:09.441752 14108 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:09.442503 14108 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:09.442775 14281 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.725s	user 0.014s	sys 0.007s
I20260709 12:28:09.410823 14087 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:09.397289  7729 rpcz_store.cc:276] 0709 12:28:08.271488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.317200 (+1045712us) service_pool.cc:224] Handling call
0709 12:28:09.385588 (+ 68388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.346424 14249 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.377544 14069 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:09.449056  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3105) took 1173 ms. Trace:
I20260709 12:28:09.454246  7728 rpcz_store.cc:276] 0709 12:28:08.270825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.274963 (+1004138us) service_pool.cc:224] Handling call
0709 12:28:09.444250 (+169287us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:09.456667  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 276049us
W20260709 12:28:09.459688  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.438192 14273 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.442631 14166 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.374413 13897 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.467005 14284 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:09.467837  7809 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:09.474423 14249 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.448809 14264 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.499271  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1343) took 1189 ms. Trace:
I20260709 12:28:09.499416  8406 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:09.499718 14110 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.500547 14110 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.506430  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:09.507390 14283 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:09.511618  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2860) took 1583 ms. Trace:
I20260709 12:28:09.511888 14166 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:09.512322  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.513386  8405 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:28:09.513294  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.514760 14305 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.512172  7659 rpcz_store.cc:276] 0709 12:28:07.927712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.928104 (+   392us) service_pool.cc:224] Handling call
0709 12:28:07.964617 (+ 36513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.976684 (+ 12067us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:09.100492 (+1123808us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.311342 (+210850us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.419736 (+108394us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.419922 (+   186us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:09.419956 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.420063 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.511328 (+ 91265us) raft_consensus.cc:1685] finished
0709 12:28:09.511351 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.511537 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11544,"prepare.queue_time_us":1296655,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":57787,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":12594,"thread_start_us":1246804,"threads_started":2,"wal-append.queue_time_us":291527}
I20260709 12:28:09.511915  7808 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:09.528793  8407 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:28:09.532510  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 138197us
I20260709 12:28:09.524825 14139 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:09.515923 14305 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.535826  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:09.537290  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.539856  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.545083 14129 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.545859  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3153) took 1429 ms. Trace:
I20260709 12:28:09.546370  7735 rpcz_store.cc:276] 0709 12:28:08.116407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.117217 (+   810us) service_pool.cc:224] Handling call
0709 12:28:08.129179 (+ 11962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.129740 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:09.125267 (+995527us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.327574 (+202307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.426110 (+ 98536us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.426308 (+   198us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:09.426344 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.426462 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.545554 (+119092us) raft_consensus.cc:1685] finished
0709 12:28:09.545596 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.545786 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58534,"prepare.queue_time_us":6030,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":82676,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":63290,"thread_start_us":1130718,"threads_started":2,"wal-append.queue_time_us":363994}
I20260709 12:28:09.550372  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3158) took 1313 ms. Trace:
I20260709 12:28:09.545511 14278 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:28:09.555856  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.557168  8415 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:28:09.558248  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1161) took 1646 ms. Trace:
I20260709 12:28:09.517517  8408 rpcz_store.cc:276] 0709 12:28:08.303633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.344868 (+ 41235us) service_pool.cc:224] Handling call
0709 12:28:08.345229 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:08.345474 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:09.282057 (+936583us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.393208 (+111151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.445113 (+ 51905us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:09.459044 (+ 13931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.484236 (+ 25192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.492557 (+  8321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26066,"prepare.queue_time_us":572858,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":123,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":56669,"thread_start_us":1002000,"threads_started":2}
I20260709 12:28:09.561782 14199 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.516222 14058 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.564865 14058 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.566499 14319 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.568022  8405 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:09.567741 14264 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.571545  8409 raft_consensus.cc:606] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:09.572212  8409 raft_consensus.cc:614] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:09.575564  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:09.577163  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44882.1us
I20260709 12:28:09.578260  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.579536 14234 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:09.580507 14234 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.529155 14284 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.582324  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3106) took 1311 ms. Trace:
W20260709 12:28:09.584645 14188 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.900s	user 0.017s	sys 0.010s
I20260709 12:28:09.585218 14188 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:09.586215 14227 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.904s	user 0.022s	sys 0.007s
I20260709 12:28:09.586715 14227 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.522385 14110 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:09.587682 14286 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.587844 14234 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.589607 14234 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.586745 14284 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.550879  7735 rpcz_store.cc:276] 0709 12:28:08.236693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.549003 (+1312310us) service_pool.cc:224] Handling call
0709 12:28:09.549571 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.549931 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.549963 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.550024 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:09.550063 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.550135 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.550277 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.591127  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3149) took 1867 ms. Trace:
I20260709 12:28:09.538743 14305 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:09.595274  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67235.3us
I20260709 12:28:09.596647  8411 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:09.582926  7736 rpcz_store.cc:276] 0709 12:28:08.271131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.435316 (+164185us) service_pool.cc:224] Handling call
0709 12:28:09.581925 (+1146609us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:09.582201 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205360512}
I20260709 12:28:09.591527 14267 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.466434 14108 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.604996  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:09.601217  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59578 (request call id 15) took 1272 ms. Trace:
W20260709 12:28:09.611250  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.606670  8409 rpcz_store.cc:276] 0709 12:28:08.302432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.302564 (+   132us) service_pool.cc:224] Handling call
0709 12:28:09.574731 (+1272167us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:09.574907 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859854592}
W20260709 12:28:09.591540 14218 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.037s	user 0.013s	sys 0.012s
I20260709 12:28:09.596411  7730 rpcz_store.cc:276] 0709 12:28:07.723524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.723641 (+   117us) service_pool.cc:224] Handling call
0709 12:28:07.733696 (+ 10055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.734269 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.734325 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.735024 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.192668 (+457644us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:08.192727 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.193875 (+  1148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.318964 (+1125089us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:09.516095 (+197131us) raft_consensus.cc:1685] finished
0709 12:28:09.516143 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.590942 (+ 74799us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39438,"raft.queue_time_us":1296,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":5623,"spinlock_wait_cycles":5067648,"thread_start_us":416458,"threads_started":1,"wal-append.queue_time_us":906129}
I20260709 12:28:09.592504 13897 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:09.630074  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52000.2us
W20260709 12:28:09.639954  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.646682  7658 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:09.558723  8412 rpcz_store.cc:276] 0709 12:28:07.912085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.912302 (+   217us) service_pool.cc:224] Handling call
0709 12:28:07.917019 (+  4717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:08.089267 (+172248us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:09.299432 (+1210165us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.507372 (+207940us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.532650 (+ 25278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.555358 (+ 22708us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:09.555415 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.557919 (+  2504us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:09.557980 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.558191 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178915,"prepare.queue_time_us":848476,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":95,"raft.queue_time_us":108992,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":47710,"spinlock_wait_cycles":395515136,"thread_start_us":1203708,"threads_started":2,"wal-append.queue_time_us":263770}
I20260709 12:28:09.599797 14267 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.625064 14264 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.627462 14249 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.666620  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.671877  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.633935 14319 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:09.648094 14258 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:09.654498 14129 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.657354 14273 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.691649  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3160) took 1306 ms. Trace:
I20260709 12:28:09.698837  7736 rpcz_store.cc:276] 0709 12:28:08.374891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.598513 (+1223622us) service_pool.cc:224] Handling call
0709 12:28:09.662666 (+ 64153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.680475 (+ 17809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.680524 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.680605 (+    81us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:09.680661 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.680756 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.680943 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.694161  7808 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:09.668078 14234 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:09.634342 14218 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.630285 14139 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:28:09.697157  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:09.704962  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 179858us
I20260709 12:28:09.702474  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:09.681080 14188 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.710580 14227 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.678297 14249 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.722538  7653 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:09.727659  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:09.676043 14251 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.729137  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.721854  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:09.706832 14319 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:09.706344 14058 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:09.740434  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.759971  7805 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:09.719853 14286 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:09.735808 14305 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:09.767463  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:09.727035 14258 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:09.768910  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88694.3us
I20260709 12:28:09.728411 14165 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.771273  7655 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:09.757526 14111 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.757135 13846 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.775637 14249 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:09.775985  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 158795us
I20260709 12:28:09.776583 14264 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:09.777072 14264 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:09.777501 14264 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:09.778362 14341 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:09.756085 14273 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.779919  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3161) took 1383 ms. Trace:
I20260709 12:28:09.780395  7736 rpcz_store.cc:276] 0709 12:28:08.396865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.705214 (+1308349us) service_pool.cc:224] Handling call
0709 12:28:09.750530 (+ 45316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.779413 (+ 28883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.779452 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.779537 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:09.779594 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.779672 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.779846 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.781769 14273 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:09.782653  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3162) took 1377 ms. Trace:
W20260709 12:28:09.784099 14337 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:28:09.777417  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.783152  7736 rpcz_store.cc:276] 0709 12:28:08.405079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.781533 (+1376454us) service_pool.cc:224] Handling call
0709 12:28:09.781979 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.782269 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.782301 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.782354 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:09.782392 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.782456 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.782586 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.773681  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:09.764148 14286 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.736869 14224 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:09.796353  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.794041  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3155) took 1554 ms. Trace:
I20260709 12:28:09.799805  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:09.786829 14267 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.764599 14227 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.797963  7734 rpcz_store.cc:276] 0709 12:28:08.235823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.453351 (+1217528us) service_pool.cc:224] Handling call
0709 12:28:09.467912 (+ 14561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.468753 (+   841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.468807 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.469587 (+   780us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.601452 (+131865us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:09.601498 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.601575 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.789653 (+188078us) raft_consensus.cc:1685] finished
0709 12:28:09.789688 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.789894 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10994,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4399,"thread_start_us":131307,"threads_started":1,"wal-append.queue_time_us":205439}
W20260709 12:28:09.817169  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139392us
I20260709 12:28:09.793062 14302 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:09.830299  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:09.832801  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55519us
W20260709 12:28:09.833334 14346 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:28:09.833847  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:09.834928 14115 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:09.814853 14165 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.815366 14305 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:09.826773 14218 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.835176  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:09.812765 14258 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.846809  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3159) took 1590 ms. Trace:
I20260709 12:28:09.835491  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:09.811285 13846 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:28:09.835469 14115 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.856258 14115 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:09.839365 14139 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:09.859983 14139 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.860431 14139 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:09.863782  7736 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:09.852241 14166 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:28:09.823191 14319 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.844689 14251 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.853039  7735 rpcz_store.cc:276] 0709 12:28:08.236969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.597408 (+1360439us) service_pool.cc:224] Handling call
0709 12:28:09.632351 (+ 34943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.655845 (+ 23494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.655904 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.695052 (+ 39148us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.796556 (+101504us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:09.796603 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.796692 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.827460 (+ 30768us) raft_consensus.cc:1685] finished
0709 12:28:09.827482 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.827640 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7905,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":3187,"thread_start_us":69443,"threads_started":1,"wal-append.queue_time_us":107195}
I20260709 12:28:09.879772  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3163) took 1474 ms. Trace:
W20260709 12:28:09.871433  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55857.1us
I20260709 12:28:09.901980  7736 rpcz_store.cc:276] 0709 12:28:08.405482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.789630 (+1384148us) service_pool.cc:224] Handling call
0709 12:28:09.846623 (+ 56993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.866548 (+ 19925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.879671 (+ 13123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:09.904534  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.868198  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3164) took 1460 ms. Trace:
I20260709 12:28:09.905383  7734 rpcz_store.cc:276] 0709 12:28:08.407374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.814825 (+1407451us) service_pool.cc:224] Handling call
0709 12:28:09.863501 (+ 48676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.867811 (+  4310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.867843 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.867888 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:09.867918 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.867980 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.868107 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.828662 14341 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.887326  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:09.843156 14165 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.873409 14218 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:09.874611  7733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "d72176ccaed049958ac68f706d4f052d"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:28:09.928385  7733 raft_consensus.cc:606] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Received request to transfer leadership to TS d72176ccaed049958ac68f706d4f052d
I20260709 12:28:09.928803  7733 raft_consensus.cc:614] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:09.912428 14069 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:09.873097 14129 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:09.912992 14258 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:09.914300 14115 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:09.934738 14069 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.935504 14258 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:09.936051 14258 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:09.937367 14115 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:09.938202 14115 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:09.938730 14115 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:09.915625  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48919.9us
I20260709 12:28:09.823455 14302 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.868945 14253 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:09.919692  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84679.9us
I20260709 12:28:09.923632  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3156) took 1687 ms. Trace:
I20260709 12:28:09.912555  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:09.951614  7728 rpcz_store.cc:276] 0709 12:28:08.236173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.493158 (+1256985us) service_pool.cc:224] Handling call
0709 12:28:09.599539 (+106381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.609941 (+ 10402us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:28:09.693215 (+ 83274us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.846441 (+153226us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.890843 (+ 44402us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.892545 (+  1702us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:09.892616 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.892722 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.923368 (+ 30646us) raft_consensus.cc:1685] finished
0709 12:28:09.923389 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.923559 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2895,"prepare.queue_time_us":98247,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":14768,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":17010,"thread_start_us":186676,"threads_started":3,"wal-append.queue_time_us":56538}
I20260709 12:28:09.953938 14283 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:09.954276  7728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "539d703b24874183847fa9a8889111e9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:28:09.954995  7728 raft_consensus.cc:606] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:09.934048  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3167) took 1400 ms. Trace:
I20260709 12:28:09.955808  7733 rpcz_store.cc:276] 0709 12:28:08.533685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.933015 (+1399330us) service_pool.cc:224] Handling call
0709 12:28:09.933426 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.933690 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.933717 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.933766 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:09.933800 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.933856 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.933982 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.958226  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3168) took 1419 ms. Trace:
I20260709 12:28:09.958690  7733 rpcz_store.cc:276] 0709 12:28:08.538545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.957049 (+1418504us) service_pool.cc:224] Handling call
0709 12:28:09.957481 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.957777 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.957816 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.957894 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:09.957937 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.958009 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.958158 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.960436  7733 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:09.961194  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1316) took 1420 ms. Trace:
I20260709 12:28:09.961524  7733 rpcz_store.cc:276] 0709 12:28:08.540643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.959713 (+1419070us) service_pool.cc:224] Handling call
0709 12:28:09.960229 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.960898 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.961135 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.927974 14218 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:09.963531 14218 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:09.937522 14069 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.954869 14283 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:09.869921 14224 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:09.968119 14111 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:09.968816 14111 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:09.970701  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.970527 14111 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.973143  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:09.968963  8409 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:09.974673 14111 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:09.976729  7728 raft_consensus.cc:614] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:09.875211 13846 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:28:09.975958  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:09.986548  7733 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:28:09.987802  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44451.7us
I20260709 12:28:09.975299 14218 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.931488 14129 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:09.943596 14305 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:09.916412 14264 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:09.976401 14253 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:09.994046  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3169) took 1435 ms. Trace:
I20260709 12:28:10.000501  7733 rpcz_store.cc:276] 0709 12:28:08.554785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.962374 (+1407589us) service_pool.cc:224] Handling call
0709 12:28:09.962965 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.987488 (+ 24523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.990408 (+  2920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.939961 14115 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.000025  8414 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:28:10.005102  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.002279  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.012085 14264 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.012671  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.012950 14264 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.013896 14264 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.014470  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:10.015511  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.019613  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.019984 14305 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.020881  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.021620 14264 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.021771  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:10.022984  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.024914  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.025096 14302 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.031428  7733 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:28:10.032018  7733 pending_rounds.cc:85] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:10.032266  7733 pending_rounds.cc:107] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:28:10.032617  7733 raft_consensus.cc:2889] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:10.033123  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.034020  7806 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:10.034616  7733 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:10.036404 14286 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.049865  7728 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:28:10.057302  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1314) took 1678 ms. Trace:
I20260709 12:28:10.059437  7730 rpcz_store.cc:276] 0709 12:28:08.379020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.624545 (+1245525us) service_pool.cc:224] Handling call
0709 12:28:09.625067 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.625318 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:09.755526 (+130208us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.056953 (+301427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:10.057014 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:10.057060 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.057129 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.057236 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20963,"prepare.queue_time_us":54549,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":54037,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":122,"thread_start_us":287150,"threads_started":2,"wal-append.queue_time_us":254711}
W20260709 12:28:10.065476  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.067060  7728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:28:10.068125  7728 raft_consensus.cc:606] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:10.068895  7728 raft_consensus.cc:614] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:10.064376  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3166) took 1553 ms. Trace:
I20260709 12:28:10.071813  7735 rpcz_store.cc:276] 0709 12:28:08.510728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.929291 (+1418563us) service_pool.cc:224] Handling call
0709 12:28:09.984929 (+ 55638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.002548 (+ 17619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.002589 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:10.002648 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:10.063924 (+ 61276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.064045 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.064279 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:10.066182 14341 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:10.091958 14341 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:10.092285 14302 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.093024 14302 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.093416 14341 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.020056 14227 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.095727 14302 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:10.097040 14338 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.346s	user 0.003s	sys 0.001s
I20260709 12:28:10.098776  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1310) took 1909 ms. Trace:
I20260709 12:28:10.098889 14286 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:10.099310  7737 rpcz_store.cc:276] 0709 12:28:08.189258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.394424 (+1205166us) service_pool.cc:224] Handling call
0709 12:28:09.503782 (+109358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:09.504372 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.504407 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:09.505353 (+   946us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.639428 (+134075us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:09.718909 (+ 79481us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.734489 (+ 15580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.734577 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.073680 (+339103us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b240010c6b8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee7409d 00007fc001aa610e 00007fc001abd88c 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221
0709 12:28:10.098391 (+ 24711us) raft_consensus.cc:1685] finished
0709 12:28:10.098512 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.098671 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15196,"prepare.queue_time_us":68941,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":27260,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":6587,"spinlock_wait_cycles":14289664,"thread_start_us":176953,"threads_started":2,"wal-append.queue_time_us":188564}
W20260709 12:28:10.103281  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 288994us
I20260709 12:28:10.104372  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.114015 14258 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:10.108515  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 167491us
I20260709 12:28:10.129420  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.130311  8414 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:10.130940  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.126600  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.133091  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:10.134563  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120206us
I20260709 12:28:10.140810  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.143312  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.109113 14302 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:28:10.150942  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.150965  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.115051 14129 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.145876  7808 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:10.160884  7655 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:10.165320  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.146447 14286 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.145531 14165 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:10.168941  7660 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:10.159920 13846 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:10.178897  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:10.179466  7653 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:10.163727 14267 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:10.180308 14165 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.183481  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.183997 14139 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.185423  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.186112  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.187516  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.188036 14354 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:10.182849 14352 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:10.187414 14253 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.201635  7733 raft_consensus.cc:2393] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:10.203673  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.188889 14115 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:10.163326 14264 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:28:10.190004 13846 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.208655  8412 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:28:10.197286 14165 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:10.247855  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.207877 14286 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.249507  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.250653 14139 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.251348 14278 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.253878  7808 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:10.192555 14267 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:10.259382 14356 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:10.259526  7659 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:10.247928 14350 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:10.237502  7730 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:10.284524  7660 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:10.273999  7654 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:10.285512  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 161868us
W20260709 12:28:10.281679  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 101319us
I20260709 12:28:10.268118 13846 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:10.267817 14115 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:10.268298 14264 consensus_queue.cc:1243] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:10.298208  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.299454 14218 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.298792 14115 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.297371  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.284209 14356 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.299481  7656 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:10.281260 14278 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.310372  7654 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:10.304463 14356 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:10.318774  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.321226 14283 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:10.321722 14283 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.321825  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.322721  7656 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:10.322145 14283 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.304713 14227 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:10.328102  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3157) took 2091 ms. Trace:
I20260709 12:28:10.329669  7729 rpcz_store.cc:276] 0709 12:28:08.236444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.288703 (+2052259us) service_pool.cc:224] Handling call
0709 12:28:10.315140 (+ 26437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.327419 (+ 12279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.327463 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:10.327554 (+    91us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:10.327683 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.327802 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.328007 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:10.339542  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:10.343307  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:10.308692 14319 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:10.344952  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.345970  7654 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:10.347975  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.352943 14278 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.341760 14227 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:10.360522  7656 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:10.360309 14227 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.361047 14286 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.363435 14069 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:10.364128 14069 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.364553 14069 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:10.370875  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 229000us
I20260709 12:28:10.372061  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:10.376149 14278 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:10.376754 14278 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:10.377183 14278 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.377939 14267 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.379462 14267 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.382071 14129 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:10.384105 14253 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.344974 14302 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:10.392592 14302 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.393074 14302 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.394203  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.395905  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.396634  7659 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
W20260709 12:28:10.398445  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.402536 14319 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.404011 14199 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:28:10.408879  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.419850  7653 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:10.411190  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.432862  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:10.397271 14264 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.438058  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.438704  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.442848  7654 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:10.449826  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.452198  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:10.432780 14351 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.235s	user 0.003s	sys 0.000s
I20260709 12:28:10.411813 14319 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.465749  7653 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:28:10.372617 14224 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:28:10.438014 14166 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:10.435256 14253 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:10.467043  8409 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.439317 14251 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:10.435710 14341 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.486234  7659 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:28:10.471271  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44662.2us
I20260709 12:28:10.493448 14282 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.494270 14282 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.495150  8411 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:10.496635 14282 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:10.476697 14312 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:10.477768 14166 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.468621 14115 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.480324 14253 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:10.526677  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92340.3us
I20260709 12:28:10.532402  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.485854 14251 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:10.485461 14319 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:10.541386  7657 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:10.498543 14302 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.471873 14264 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.549086  8413 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:10.534621 14166 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:10.555938  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.546026 14253 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:10.547616 14251 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:10.520035 14165 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.571064  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.574869  7808 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:10.542280 14115 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.575241  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:10.582247  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.583451  7659 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:10.598207  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.560246 14312 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.557832 14341 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.588289 14286 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:28:10.545702 14111 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:28:10.612622  7808 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.614171  8409 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:10.616729  7657 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:10.620653  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48986.4us
I20260709 12:28:10.620059  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.657320  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:10.662554  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.664819  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:10.665882  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53578.5us
W20260709 12:28:10.648131 14366 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:28:10.672989  7659 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:10.673373 14302 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.673606  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.682214  7809 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:10.679402 14369 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:28:10.674963 14278 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:10.683435  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.643738 14286 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.655476 13846 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.690225  7657 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:10.700678  7808 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:28:10.690968  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1186) took 1449 ms (client timeout 1888 ms). Trace:
W20260709 12:28:10.708736  7658 rpcz_store.cc:269] 0709 12:28:09.241236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.241534 (+   298us) service_pool.cc:224] Handling call
0709 12:28:10.690652 (+1449118us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5400116554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:10.690836 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107380992}
W20260709 12:28:10.691784  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3200) took 2553 ms (client timeout 1636 ms). Trace:
I20260709 12:28:10.708420  7812 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:10.713562  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.737720  8409 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:10.689411 14184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.278s	user 0.010s	sys 0.009s
I20260709 12:28:10.739164  7659 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:28:10.691105 14331 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.980s	user 0.004s	sys 0.000s
I20260709 12:28:10.690470 14319 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:10.735618  7805 rpcz_store.cc:269] 0709 12:28:08.115198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.158833 (+ 43635us) service_pool.cc:224] Handling call
0709 12:28:10.648977 (+2490144us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540042a754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:10.664990 (+ 16013us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2983923968}
I20260709 12:28:10.705714 14286 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:10.750241  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.758826  7655 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.752039  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.752677  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.767802  7808 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:10.769513 14286 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.745931  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.752065  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:10.743072 14184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.465s	user 0.012s	sys 0.009s
I20260709 12:28:10.771492  8413 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:10.795885  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.732242 14302 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:28:10.809295  7655 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:10.797927  7812 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:10.732582 14165 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.741185 14111 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.695703 14356 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.815946  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.827607  7658 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:10.815527  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.824651  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.825387  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.828995  8409 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:10.828894  7659 raft_consensus.cc:2376] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16. Re-sending same reply.
I20260709 12:28:10.688758 14253 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.820725  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:10.824775  7657 raft_consensus.cc:2376] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15. Re-sending same reply.
I20260709 12:28:10.836210  7655 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:10.836578  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.837586  8413 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:10.810823 13997 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:10.820837 14227 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:10.852695  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.853739  7654 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:28:10.822597 14115 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.854913  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.855909  7658 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:10.856882  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.849319  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.860076  7655 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:10.858950  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.861663  8412 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:28:10.874131  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.873898  7659 raft_consensus.cc:2376] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19. Re-sending same reply.
W20260709 12:28:10.874703  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 264214us
W20260709 12:28:10.877197  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 211695us
I20260709 12:28:10.823859 14111 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.884356  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3154) took 2762 ms. Trace:
I20260709 12:28:10.884682  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:10.846135 13997 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:10.890501  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:10.853250 14227 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:10.900715  7659 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:10.906376 14278 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:10.907510  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.908533 14227 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:10.909606  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:10.911878  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.912956  7805 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:10.913864 14251 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:10.915385 14069 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.003s
I20260709 12:28:10.916060  7660 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:10.915863 13997 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:10.917513 13997 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:10.919819 14341 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.920333 14286 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:10.920723 14341 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:10.915895 14069 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:10.922366  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.910869  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:10.923990 14069 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.924705  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.925920  7732 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:10.927970 13846 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.917567  7660 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:28:10.929171  7660 pending_rounds.cc:85] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 11
I20260709 12:28:10.929476  7660 pending_rounds.cc:107] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:28:10.909152 14111 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.929689  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:10.930729 14267 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.002s
I20260709 12:28:10.931468  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.932143  7733 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:10.932821 14253 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:10.934012  7729 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:10.936192  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.930576  7728 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:10.929718  7660 raft_consensus.cc:2889] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:10.893674  7657 raft_consensus.cc:2376] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15. Re-sending same reply.
I20260709 12:28:10.979326  8413 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:28:10.979076  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:10.924688 14069 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:10.985711  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:10.986685  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.987211  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:10.992436  8409 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:10.887210  7731 rpcz_store.cc:276] 0709 12:28:08.116804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.136742 (+ 19938us) service_pool.cc:224] Handling call
0709 12:28:08.137380 (+   638us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.212649 (+1075269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.212701 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.213625 (+   924us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.447752 (+234127us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:09.447853 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.457980 (+ 10127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.690650 (+1232670us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b240010c6b8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee7409d 00007fc001aa610e 00007fc001abd88c 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221
0709 12:28:10.859756 (+169106us) raft_consensus.cc:1685] finished
0709 12:28:10.859799 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.879272 (+ 19473us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27393,"raft.queue_time_us":53294,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":30016,"spinlock_wait_cycles":19021568,"thread_start_us":1253744,"threads_started":2,"wal-append.queue_time_us":425064}
W20260709 12:28:10.995062  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84307.1us
I20260709 12:28:11.004801  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.006544  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:11.007303  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.007980  7653 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:28:11.008755 14370 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 12:28:11.009879 14283 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.000s
I20260709 12:28:11.010365 14283 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.011730  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.012470  7660 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:28:11.015499  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:11.011986 14378 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:28:11.022766  7657 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:11.023626  8413 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:28:11.026062  7660 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:28:11.028270  8412 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:11.044838  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:10.995905 14115 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:11.048619 14283 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:11.048856  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.049700  7806 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:11.050319  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.004957 14312 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.056531  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:11.051550 14375 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
I20260709 12:28:11.065654  8414 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:11.067119  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:11.069160  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.087307  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:11.004254 14069 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.088845  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:11.091557  8409 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:11.093330  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.103456  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:11.070603 14365 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.070267 14319 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:11.111867  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:11.100456  7806 raft_consensus.cc:2376] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17. Re-sending same reply.
I20260709 12:28:11.117184  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.123466  7805 raft_consensus.cc:2376] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21. Re-sending same reply.
I20260709 12:28:11.068222 14166 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.071480 14286 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 12:28:11.133560  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 433748us
I20260709 12:28:11.114872 14341 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:11.140347  7662 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:28:11.142208 14255 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.143086 14255 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.145931 14255 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:11.149482  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46232.4us
I20260709 12:28:11.105942 14356 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.107313 14267 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.115949 14319 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:11.161679  7657 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:11.131526 14129 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:11.165944  7805 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:11.172322  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.124220 14115 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:28:11.137421 14251 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.115047 14363 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:28:11.177764 14129 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:11.188807  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.163683 14319 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:11.201097  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 148137us
I20260709 12:28:11.201862  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.202733  7808 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:28:11.209254 14393 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:28:11.176190 14267 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.212932  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.137362 14365 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.209546  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:11.183496 14312 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:28:11.222141 14251 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.228216 14394 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:28:11.247819  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 78421.2us
I20260709 12:28:11.185777 14391 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:28:11.179136 14395 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 12:28:11.256409 14383 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:28:11.257329  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.260546 14166 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.260964 14267 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.261941 14391 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:11.262490 14391 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:11.263595 14391 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.265411  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:11.267123 14111 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:28:11.268281 14111 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:11.268193 14251 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.269273 14264 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.269516 14111 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.270838 14251 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.241582 14385 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.272646 14365 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.273130  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:11.274152  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.010s	sys 0.018s
I20260709 12:28:11.242612  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:11.182551 14129 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:11.279296  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.010s	sys 0.018s
W20260709 12:28:11.281767  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 847644us
W20260709 12:28:11.280227  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90236us
I20260709 12:28:11.278394  7661 raft_consensus.cc:2376] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14. Re-sending same reply.
I20260709 12:28:11.288679  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:11.290091 14392 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260709 12:28:11.290904 14385 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.293944 14385 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.277571 14394 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:11.295610 14394 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:11.304283  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63218.5us
I20260709 12:28:11.307323  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:11.305573  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:11.311466  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.226804 14401 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:28:11.311055  7700 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:11.285908 14391 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:11.320911  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 99166.8us
I20260709 12:28:11.320341 14210 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:11.322762  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:11.325398 14371 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.645s	user 0.003s	sys 0.000s
I20260709 12:28:11.317245 14166 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:11.327417  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:11.332957  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:11.334309  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.335439  7660 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.336980  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.337975  7812 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.339526  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.344718  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:11.344910  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.346028  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:11.347703  8409 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:11.340627  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.350888  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.317595 14267 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:11.310156 14069 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.328591 14385 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:11.362349  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.359951 14312 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:11.367499 14403 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:11.367710  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.378618  7662 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:28:11.361207  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.341068 14115 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:11.381114 14403 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.395603 14403 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:11.384931  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62263.7us
I20260709 12:28:11.400082  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:11.401212  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:11.402312 14282 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:11.402829 14282 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.403280 14282 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.405285 14395 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:11.405833 14395 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.406243 14395 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:11.408239  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:11.377713 14264 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:28:11.359578 13997 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.409739 13997 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.353883 14363 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:11.412835  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.414716 14224 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.346179 14166 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:11.374212 14312 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:11.387730  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.427767 14312 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.431716  7658 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.437845  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:11.387955  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.440752  7806 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:11.370136 14278 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:11.443357 14286 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.443639 14278 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.444136 14286 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.446485 14278 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.377327 14267 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:11.448047 14267 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.426767 14166 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.385443 14210 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:11.439841  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:11.454146  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:11.454447  7736 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 11.
I20260709 12:28:11.456238 14210 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:11.457548 14210 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.458961  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:11.460594  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:11.462399  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.463719  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:11.487564  7660 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:11.432385 14400 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:28:11.514518  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:11.516278  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:11.515410  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:11.524287  7658 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:11.532667  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 118234us
I20260709 12:28:11.551977  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:11.537297  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:11.556931  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:11.447046 14286 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.562964  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:11.565272  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.570302  7656 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:11.567848  7728 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.568939  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.512140 14278 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:11.578408  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:11.587883  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:11.591995  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 387170us
I20260709 12:28:11.588850  7661 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:11.537783 14365 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.598312  7731 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:28:11.605093  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 146224us
I20260709 12:28:11.562664 14385 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:11.568416 14363 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.613197  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:11.615512  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.620136  7662 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:11.624951  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:11.625787  7661 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:11.535138 14069 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:28:11.630859  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.631898  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:11.632797  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.635257  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:11.637365  8415 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:11.637521  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:11.647614 14374 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:11.648360 14374 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.648739 14374 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:11.655956  7731 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:11.656241  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:11.657119  7736 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:28:11.619424 14385 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.657933 14385 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:11.664367  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:11.669785 13997 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:11.669802 14129 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:28:11.674376 14390 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:11.683640  7728 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:11.637863 14401 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:11.678272 14403 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.607314 14365 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.697254  7806 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:11.689352  7736 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 12.
I20260709 12:28:11.708461 14224 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:11.700213 13997 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.710561 14267 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.696230 14278 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.745211  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.634370 14363 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:11.711583 14069 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.703963 14286 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:28:11.774506 13997 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:11.781211  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.742765 14255 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
W20260709 12:28:11.850636  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 227087us
I20260709 12:28:11.774314 14267 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.811832 14224 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:28:11.810384 14278 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:11.873922  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 133981us
W20260709 12:28:11.858958 14391 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.001s
I20260709 12:28:11.856729 14253 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.822659 14069 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:11.866128  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 254361us
W20260709 12:28:11.917629  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61136.7us
I20260709 12:28:11.898550 14218 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:11.934960  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 298152us
I20260709 12:28:11.910254 14253 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.972493  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:11.974238  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:11.976822  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:11.976979  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:11.981236  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:11.978281  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62749.3us
I20260709 12:28:11.984112  8414 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
W20260709 12:28:11.985608  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49097.7us
I20260709 12:28:11.967549 13997 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.944293 14218 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:11.976675 14165 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.978940 14255 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:11.996173 14403 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.011169 14417 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:28:12.013942 14425 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:28:12.014119 14409 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.015805 14267 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.994779 13997 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.017082 13997 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.020377 13997 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.016510 14267 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:11.979498 14391 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:28:11.982052 14286 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:11.993008 14406 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
W20260709 12:28:11.991582 14405 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:28:11.988313 14356 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:28:12.072209  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98109us
I20260709 12:28:11.993036 14432 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:11.992269 14365 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:11.992122 14129 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:28:12.034837 14423 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 12:28:12.077481  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.992848 14426 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.023300 14255 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:12.014879 14409 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.076751 13997 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.071955 14286 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.097404  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.107932  7736 raft_consensus.cc:2410] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:28:12.072134 14356 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:12.112315  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 131177us
I20260709 12:28:12.078282 14432 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:12.037446 14165 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.078617 14267 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.078802 14417 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:28:12.092747 14403 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:28:12.224068  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.171775  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:12.227864  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 154592us
I20260709 12:28:12.097623 14419 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 12:28:12.135811 14278 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:12.233647  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 225630us
W20260709 12:28:12.226383 14411 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.312s	user 0.001s	sys 0.001s
I20260709 12:28:12.111613 13997 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:12.231784  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 210659us
I20260709 12:28:12.251206  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:12.170210 14302 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.165446 14069 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.243860 14365 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:12.120579 14426 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:12.281800  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 110109us
I20260709 12:28:12.283375  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:12.258608 14410 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:12.287288  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58563.6us
I20260709 12:28:12.290290  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:12.291339  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.265105 14403 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:12.278286 14319 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.059s	user 0.015s	sys 0.002s
I20260709 12:28:12.276299 14069 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:12.286381 14374 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.637s	user 0.014s	sys 0.000s
I20260709 12:28:12.271397 14302 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.321195  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.323313 14184 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:12.248541 14356 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:12.338652 14431 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:28:12.339211 14267 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.339651 14439 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:12.279665 14365 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:12.338702 14395 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.932s	user 0.012s	sys 0.004s
I20260709 12:28:12.340775 14437 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.341094 14395 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.341539 14437 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.342427 14365 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.256151 14255 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:12.344332 14443 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:12.348555  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.326787  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.313659 14448 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:28:12.331269  7805 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:12.333830  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.278545 14433 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:12.367522  8414 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:10.854691  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:12.319489 14319 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.337144 14442 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.337877 14436 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:28:12.339349 14356 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:12.316435 14087 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:28:12.325459 14388 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:12.385094  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.385915  7806 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:12.314421 14427 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:12.331377  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57732.9us
W20260709 12:28:12.358858  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 74721.7us
W20260709 12:28:12.372290  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 321695us
I20260709 12:28:12.280944 14253 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.377148  8408 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:12.378082  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:12.299744 14410 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.363732 14255 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:12.349462 14069 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:12.416642 14459 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:12.417138  7734 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:12.417336 14459 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.417901 14459 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:12.419438 14430 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:12.420122 14430 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:12.420893 14450 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.140s	user 0.004s	sys 0.000s
I20260709 12:28:12.272325 14432 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.421176 14430 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.422215 14432 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:12.423418 14460 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.423903 14286 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.424075 14460 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.424575 14426 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:12.425052 14458 consensus_queue.cc:1175] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 19 status { last_received { term: 18 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:12.426146 14087 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:12.426353 14227 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:12.318995 14374 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.427670 14409 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.348927 14403 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.429059 14390 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:12.429630 14390 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:12.364117 14439 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:12.432526 14439 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.411512 14302 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.434974 14319 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.430845 14390 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:12.415814  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.380458 14442 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.441171 14374 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.444094 14395 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:12.426770 14087 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:12.446935 14087 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.448791 14087 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:12.449723 14407 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.554s	user 0.000s	sys 0.001s
I20260709 12:28:12.452754 14199 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:12.460048  7662 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:12.460583 14437 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.417371 14069 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.434981 14210 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:12.462083 14210 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:12.463829  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.418020  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1319) took 1521 ms. Trace:
I20260709 12:28:12.465102  7734 rpcz_store.cc:276] 0709 12:28:10.896347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.416398 (+1520051us) service_pool.cc:224] Handling call
0709 12:28:12.416883 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.417687 (+   804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.417939 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:12.415650 14258 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.408404  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:12.478163  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69871.9us
I20260709 12:28:12.479629 14390 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:12.482141  7654 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:12.482872 14410 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.483880  7658 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:12.485255  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:12.494272  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.498245  7736 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:12.498553  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:12.499675  7658 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:12.476030 14302 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.507136  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.472992  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.475545  7661 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [14.9-17.10]
I20260709 12:28:12.491736 14108 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:12.463017 14210 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.482134 14460 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:12.501284 14282 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.097s	user 0.009s	sys 0.013s
I20260709 12:28:12.500079 14437 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:12.495328 14115 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.521852  7731 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:28:12.517021  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:12.488737 14165 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:12.530722  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57845.2us
I20260709 12:28:12.497979 14218 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.531875  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.501559 14319 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.538882  7734 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:28:12.539430  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:12.545742  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.495177 14199 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:28:12.498142 14286 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:28:12.549285  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.557449  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.514986 14302 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.526122 14409 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.534827 14460 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.504887 14401 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:28:12.562716  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:12.562716  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.550676 14455 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.580060  7730 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:12.542053 14258 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:12.582082  7732 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:12.554931 14312 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:12.553064 14165 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:12.588171  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.556098 14437 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:12.566115 14267 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:12.590867  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.526610 14108 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:12.586870  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 296677us
I20260709 12:28:12.535035 14282 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.550683 14444 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:12.594053  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.571781 14253 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:12.598469  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:12.647794  7735 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.604825  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:12.649152  7734 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:12.607102  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.593905 14403 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.609176  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.553766 14319 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.612638  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.613107 14457 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 12:28:12.613832  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.617705  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.618598  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.620198  7807 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:28:12.620450  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:12.621315  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:12.607014  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.687554  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.625892  7808 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:12.689690  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.691154  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.692143  7808 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:12.693094  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:12.693702  7808 raft_consensus.cc:2376] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
I20260709 12:28:12.694676  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:12.695530 14385 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.037s	user 0.015s	sys 0.002s
I20260709 12:28:12.696162 14385 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.698673 14385 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:12.626955  7812 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:12.550836 14453 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 12:28:12.551669 14115 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.608518  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:12.712733  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 104313us
I20260709 12:28:12.713658  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:12.602246 14278 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:12.588796 14442 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.626827  8413 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:12.699786  7728 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:12.722111  7806 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:12.579735 14409 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.704685  7803 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:12.706488  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 97430.4us
I20260709 12:28:12.708101  7809 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:12.684482  7804 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:28:12.715126  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.723512  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:12.646601 14428 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:12.583799 14460 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.588964 14391 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:28:12.622408 14388 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:28:12.734571  7805 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:28:12.736639  7807 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:12.737802  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:12.737819  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:12.712309 14210 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:12.765870  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:12.812186  7806 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:12.721578 14258 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:12.814574  7810 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:12.734026 14278 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:12.708338 14286 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:28:12.758438 14385 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:12.650995 14444 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.812223 14253 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:12.811116 14442 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.649816 14455 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.823975 14464 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:12.832664 14461 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:12.893605 14464 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:12.943435 14388 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:28:12.954061  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.923420 14461 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:12.870152 14253 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.923626 14442 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:12.964112 14282 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:12.966074 14444 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:12.823544 14437 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:12.885075 14428 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.993420 14286 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:28:12.993973 14466 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:28:12.994315 14434 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:12.995018  7737 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
W20260709 12:28:13.000438 14418 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.859s	user 0.001s	sys 0.001s
I20260709 12:28:13.129060  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:13.055490 14388 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:13.078336 14253 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:12.732357 14463 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.159s
I20260709 12:28:13.087438  7658 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:13.050496 14430 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:13.128713 14286 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:13.152292 14461 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.152745 14286 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.154318 14461 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.093362 14429 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.155649 14429 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:13.158541 14468 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:28:13.133096 14428 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.102851  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.166445  7662 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:13.145474 14469 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:28:13.217013  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.226055  7737 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:13.228106  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:13.219213 14464 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.177412  7733 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:13.245414  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:13.231168  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.246466  7737 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:13.178043  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.195312  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.249941  7658 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:13.202155  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.168098  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.167121  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1020) took 1454 ms. Trace:
I20260709 12:28:13.193666  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.185880  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:13.204780  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.261808  7731 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:13.262359  7732 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 12: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:13.168507  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.640s (SENT) [suppressed 15 similar messages]
I20260709 12:28:13.255684  7662 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:13.283932  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.282848  7778 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:13.257443  8412 rpcz_store.cc:276] 0709 12:28:11.713168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.713357 (+   189us) service_pool.cc:224] Handling call
0709 12:28:11.750690 (+ 37333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.918781 (+168091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.918833 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.919518 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 12:28:12.063465 (+143947us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:12.082259 (+ 18794us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.166314 (+1084055us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b5400013720. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e9d733 00007fc001f22dbf 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c
0709 12:28:13.166413 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.166561 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.166640 (+    79us) raft_consensus.cc:1685] finished
0709 12:28:13.166795 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.167019 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17847,"prepare.queue_time_us":17244,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":245678,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":64224,"spinlock_wait_cycles":13057920,"thread_start_us":546458,"threads_started":3,"wal-append.queue_time_us":181126}
I20260709 12:28:13.294919  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.297622  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:13.299708  7653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.054s	sys 0.032s
W20260709 12:28:13.300870  7653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.055s	sys 0.032s
I20260709 12:28:13.201032 14395 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:13.234392 14363 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:13.231992 14129 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.194536 14458 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:13.195501 14111 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:28:13.324002  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.001s	sys 0.016s
I20260709 12:28:13.236557 14464 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:13.281453 14462 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.515s	user 0.005s	sys 0.000s
I20260709 12:28:13.281244 14286 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:13.336730  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:13.308600  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.008s	sys 0.005s
W20260709 12:28:13.357441  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.009s	sys 0.005s
W20260709 12:28:13.336874  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.001s	sys 0.016s
W20260709 12:28:13.358512  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 827639us
W20260709 12:28:13.360531  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.356788  7732 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:28:13.367760  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.013s	sys 0.007s
W20260709 12:28:13.368027  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 654429us
W20260709 12:28:13.355870  7655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.046s	sys 0.051s
W20260709 12:28:13.370687  7655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.046s	sys 0.051s
I20260709 12:28:13.311983 14395 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:13.381533 14442 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.329460 14165 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.386691  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.393147  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.393332  7736 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:13.335580 14108 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:13.394029  8413 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:13.352924 14415 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.393308  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:13.407109  8414 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:13.397054  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:13.395741  7659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.009s	sys 0.011s
W20260709 12:28:13.379231 14227 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.952s	user 0.016s	sys 0.011s
W20260709 12:28:13.368541  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.014s	sys 0.007s
I20260709 12:28:13.408097  7732 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:13.394025 14459 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.976s	user 0.017s	sys 0.063s
I20260709 12:28:13.353586 14363 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:28:13.383320 14452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.040s	sys 0.083s
I20260709 12:28:13.417032  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.362455 14111 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.336184 14129 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.383808 14302 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:13.408267  7659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.010s	sys 0.011s
I20260709 12:28:13.458248  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.391273 14403 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.459621 14403 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:13.460928 14432 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.038s	user 0.021s	sys 0.023s
I20260709 12:28:13.461563 14111 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.461866 14403 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.462749  7661 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:13.422406  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:13.463877 14356 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.078s	user 0.014s	sys 0.013s
I20260709 12:28:13.464293  8405 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:13.423869  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.465451  8408 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:13.419953  7729 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:13.466619 14409 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.407362 14165 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.469595 14165 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:13.470155 14439 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.470685 14439 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:13.472072  7735 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.824s	user 0.039s	sys 0.019s
I20260709 12:28:13.413630 14108 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:28:13.428393 14452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.040s	sys 0.084s
I20260709 12:28:13.472853 14108 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:13.473028 14395 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.474100 14108 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.474860  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:13.475833  7735 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:13.423059 14415 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.482453 14415 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:13.441939 14429 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.412220 14453 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:28:13.484150  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.484512 14480 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.484943  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:13.485109 14480 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.466902  8412 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:13.238024 14388 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.427728 14459 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.414140 14278 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.475937  7736 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:13.454945 14129 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.479964  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:13.480384 14087 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.031s	user 0.017s	sys 0.005s
W20260709 12:28:13.452371 14427 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.040s	user 0.020s	sys 0.004s
I20260709 12:28:13.485960  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.440018 14460 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.492978  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.507838  7654 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:13.501905  7659 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:13.492295 14453 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.522105  8414 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:13.499388 14365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.010s	sys 0.014s
W20260709 12:28:13.506286 14258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.013s	sys 0.011s
I20260709 12:28:13.511298 14427 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:13.512053 14424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.017s
I20260709 12:28:13.502539  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
W20260709 12:28:13.576150  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 329035us
I20260709 12:28:13.578641  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:13.583885  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:13.584821  7736 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:13.532716 14365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.010s	sys 0.014s
I20260709 12:28:13.580945  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.585505  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.585728  7654 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:13.587936  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.586184  7659 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:13.595453  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:13.573954 14129 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:13.582998 14424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.011s	sys 0.018s
I20260709 12:28:13.602897  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.604049  7654 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:13.605799  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:13.606381  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 437857us
I20260709 12:28:13.606895  7736 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:13.612883  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.586999 14087 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.615736  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.617525  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:13.621791  7654 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:13.626573  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.627278  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
W20260709 12:28:13.566671 14258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.014s	sys 0.011s
I20260709 12:28:13.627111  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:13.536244 14453 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.675149  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:13.685922 14255 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.242s	user 0.023s	sys 0.013s
I20260709 12:28:13.771344  7659 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:13.771072  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2862) took 1278 ms. Trace:
W20260709 12:28:13.663441  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 329122us
I20260709 12:28:13.676388  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.657835  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:13.727042  7732 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:13.607846 14356 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:13.674949  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.783427  7732 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:28:13.763690  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:13.748931 14424 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:13.797009  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.791071  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:13.618922 14403 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.152s
I20260709 12:28:13.785813  8414 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:28:13.616956 14429 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.167s
W20260709 12:28:13.794910  7700 leader_election.cc:336] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.639s (SENT)
W20260709 12:28:13.801648  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 518864us
I20260709 12:28:13.773384  7653 rpcz_store.cc:276] 0709 12:28:12.492968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.493095 (+   127us) service_pool.cc:224] Handling call
0709 12:28:12.503969 (+ 10874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.504199 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:12.725608 (+221409us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.770651 (+1045043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:13.770716 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:13.770765 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.770857 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.771004 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5217,"prepare.queue_time_us":385331,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":155568,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":1238130,"threads_started":3,"wal-append.queue_time_us":850157}
I20260709 12:28:13.628387 14129 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.784238  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3123) took 1021 ms. Trace:
I20260709 12:28:13.843587  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:13.795488 14227 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:13.745707 14258 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:13.818985  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.822827  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:13.777228 14302 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:13.631639 14087 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:13.811159 14390 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.329s	user 0.030s	sys 0.032s
I20260709 12:28:13.811733  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:13.845010  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 289334us
I20260709 12:28:13.840643  7732 rpcz_store.cc:276] 0709 12:28:12.763029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.782759 (+1019730us) service_pool.cc:224] Handling call
0709 12:28:13.783128 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.783909 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.784157 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:13.804029  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:13.839933  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:13.842893  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:13.797253 14424 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:13.819679 14356 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.812431 14498 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.881853  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.835652 14409 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:13.882239  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:13.898437  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 94492.8us
I20260709 12:28:13.896396  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.899626  7653 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
W20260709 12:28:13.894098  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82458.8us
W20260709 12:28:13.900246  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 242593us
I20260709 12:28:13.908388  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:13.841593 14480 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:13.904343  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:13.882320 14430 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.736s	user 0.023s	sys 0.002s
I20260709 12:28:13.918151  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:13.920944  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:13.921805  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81986us
I20260709 12:28:13.922487  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:13.913826  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 118929us
W20260709 12:28:13.904582  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.840466 14452 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:28:13.929330  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:13.861681 14227 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:13.915588  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:13.919559  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 292528us
I20260709 12:28:13.928303  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.922948  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:13.901759 14302 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:13.930826  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:13.932430  7806 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:13.902041 14409 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.944193 14409 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.924979  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:13.944391 14302 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:13.947036 14480 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:13.947593 14480 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.948025 14480 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:13.947798 14302 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.894085 14341 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:13.936053  7659 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:13.905858 14456 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:13.961741  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46254.6us
I20260709 12:28:13.962858  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:13.954402  7732 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:13.967787  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:13.968909  7807 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:13.916617 14499 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:13.994127  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:13.934206 14365 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:13.932983 14165 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:13.930002 14460 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:13.934490 14430 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:13.953352 14227 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:14.012977  8413 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:13.952791 14496 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:13.907114 14356 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.011099  8414 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:28:13.968295 14302 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.039464  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:14.002164 14485 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.379s	user 0.000s	sys 0.001s
I20260709 12:28:13.913280 14498 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.895938 14424 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:13.926551 14069 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:28:13.973039 14517 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:14.086849  7659 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:28:14.003013 14365 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:14.060732  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97975.2us
I20260709 12:28:14.002100 14419 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:14.072691  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 149818us
I20260709 12:28:14.004130 14504 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:13.967780 14341 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.079988  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.129601  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.037108 14499 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:28:14.054157 14484 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
I20260709 12:28:14.053012  7732 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:14.075435 14165 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:14.126479  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:14.094412  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 163691us
W20260709 12:28:14.081698 14355 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 3.975s	user 0.011s	sys 0.018s
I20260709 12:28:14.123955 14365 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:14.152858  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 216772us
I20260709 12:28:14.042837 14496 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.085615 14415 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:14.173869  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:14.079528 14253 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.911s	user 0.015s	sys 0.005s
W20260709 12:28:14.133098 14490 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.401s	user 0.000s	sys 0.001s
I20260709 12:28:14.050730 14460 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.090102 14517 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.133508 14504 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.005350 14258 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.064837 14516 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:14.085072 14403 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:28:14.128201 14424 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:14.191066  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:14.193270  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.133391 14430 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:28:14.182168 14464 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.819s	user 0.013s	sys 0.002s
I20260709 12:28:14.170321 14282 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.241765  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.171401 14165 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.226861  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:14.208765 14385 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.323s	user 0.010s	sys 0.008s
I20260709 12:28:14.190181 14496 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:14.234398  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.275590  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.170459 14510 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.282183  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.214439 14460 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:14.281934  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.207702 14108 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.276878  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:14.298902  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.275369  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.223929 14087 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.220317 14517 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:14.283639  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.312188  7810 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:14.225276 14504 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:14.133165 14419 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.294064  7803 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
W20260709 12:28:14.302443  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111468us
I20260709 12:28:14.305229  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3190) took 1736 ms. Trace:
I20260709 12:28:14.317338  7736 rpcz_store.cc:276] 0709 12:28:12.568959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.609954 (+1040995us) service_pool.cc:224] Handling call
0709 12:28:13.655710 (+ 45756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.758777 (+103067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.758829 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.794003 (+ 35174us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.916285 (+122282us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.916329 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.916405 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.304955 (+388550us) raft_consensus.cc:1685] finished
0709 12:28:14.304977 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.305136 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66943,"raft.queue_time_us":105299,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":22393,"thread_start_us":71163,"threads_started":1,"wal-append.queue_time_us":411094}
I20260709 12:28:14.209477 14253 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.302356  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:14.304773 14497 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.333s	user 0.000s	sys 0.002s
I20260709 12:28:14.326267  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.327178  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:14.296136 14199 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:14.307647  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2860) took 1815 ms. Trace:
I20260709 12:28:14.232636 14415 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:14.278329 14165 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:14.333226  7660 rpcz_store.cc:276] 0709 12:28:12.492008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.492195 (+   187us) service_pool.cc:224] Handling call
0709 12:28:12.503343 (+ 11148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:12.503627 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:12.503656 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:12.513747 (+ 10091us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.410520 (+896773us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.410586 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.410699 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.307331 (+896632us) raft_consensus.cc:1685] finished
0709 12:28:14.307368 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.307556 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920,"thread_start_us":858954,"threads_started":1,"wal-append.queue_time_us":1062541}
I20260709 12:28:14.306541  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:14.294934 14385 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:14.306807  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.319296 14522 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 12:28:14.330515  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.284176 14501 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.333616  7736 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:28:14.305341 14403 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:14.317094  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:14.270551 14430 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.338709  7807 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:14.340657  7812 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:14.342730  7736 pending_rounds.cc:85] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:14.352070  7736 pending_rounds.cc:107] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:28:14.286123 14282 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.356570  7734 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:14.336479  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.352489  7736 raft_consensus.cc:2889] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:14.340209  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 278667us
I20260709 12:28:14.358095  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3197) took 1085 ms. Trace:
I20260709 12:28:14.358698  7736 rpcz_store.cc:276] 0709 12:28:13.273207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.319481 (+1046274us) service_pool.cc:224] Handling call
0709 12:28:14.320458 (+   977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.357782 (+ 37324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.358027 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:14.359040  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:14.338097 14087 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.360646  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.334920 14505 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.349053 14495 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
I20260709 12:28:14.296891 14510 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.397413 14430 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
W20260709 12:28:14.399061 14210 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.571s	user 0.010s	sys 0.007s
I20260709 12:28:14.400120 14505 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.350162 14501 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.405758 14501 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.406044 14429 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:14.406418 14501 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:14.410530  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3181) took 1866 ms. Trace:
I20260709 12:28:14.419265 13997 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.370680 14521 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:14.374601 14403 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:14.423336 14403 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:14.425209  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1328) took 2769 ms (client timeout 1948 ms). Trace:
W20260709 12:28:14.425817  7729 rpcz_store.cc:269] 0709 12:28:11.656083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.663994 (+  7911us) service_pool.cc:224] Handling call
0709 12:28:14.424856 (+2760862us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54003b56d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:14.425081 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4026950912}
I20260709 12:28:14.427073 14415 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.345134 14129 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.372673  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:14.309970 14108 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.451303  7806 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:14.356940 14511 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.274s	user 0.000s	sys 0.002s
I20260709 12:28:14.383406  7807 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:14.357771 14199 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:14.364048 14302 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.467046 14529 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.451668 14363 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.467722 14529 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.467958 14199 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.468678 14439 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.470227 14529 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.470995 14510 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.393049 14253 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.444350  7808 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:28:14.411054  7733 rpcz_store.cc:276] 0709 12:28:12.544102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.544739 (+   637us) service_pool.cc:224] Handling call
0709 12:28:13.177117 (+632378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.245864 (+ 68747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.410281 (+1164417us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54003b47d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:14.410453 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410233728}
I20260709 12:28:14.445957  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3199) took 1154 ms. Trace:
W20260709 12:28:14.384064 14461 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.229s	user 0.017s	sys 0.006s
I20260709 12:28:14.463301 14523 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:14.464423 14471 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:14.466125 14419 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.504680  7805 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:28:14.468401 14363 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.472404  7729 rpcz_store.cc:276] 0709 12:28:13.292115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.427010 (+1134895us) service_pool.cc:224] Handling call
0709 12:28:14.445130 (+ 18120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.445458 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.445499 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.445567 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:14.445617 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.445697 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.445863 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:14.390898 14506 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:28:14.470929  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:14.474629  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 202009us
I20260709 12:28:14.450716 14463 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.508635 14463 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.481479  7806 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:14.470863 14529 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.512385 14463 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.511988 14529 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.455421  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.525470 14529 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.503110 14471 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:14.528817 14525 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.087s	user 0.011s	sys 0.011s
W20260709 12:28:14.528347 14507 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.201s	user 0.000s	sys 0.003s
I20260709 12:28:14.533105  7729 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:28:14.538465  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1335) took 1173 ms. Trace:
I20260709 12:28:14.539110  7729 rpcz_store.cc:276] 0709 12:28:13.365425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.506849 (+1141424us) service_pool.cc:224] Handling call
0709 12:28:14.507438 (+   589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.538027 (+ 30589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.538380 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:14.472688 14505 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:14.546862  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.543028  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.565475  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:14.564260  7803 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:28:14.568099  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 187193us
I20260709 12:28:14.569276  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.378819 14087 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.577687  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:14.553383 14530 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:28:14.535385 14471 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.529618 14435 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.587074  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:14.529691 14129 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:14.563825 14508 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.563s	user 0.008s	sys 0.005s
I20260709 12:28:14.536463 14515 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.457194 14409 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.595419  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:14.616309  8407 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:14.598559  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:14.564941 14356 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.601630  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.581487 14529 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:14.555366 14363 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.553128 14498 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.552278 14524 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.603976  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.605093  8411 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:14.606935  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:14.584213 14253 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:14.558348 14526 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:14.555083 14108 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:14.556631 14282 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.621222  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:14.628083  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:14.620806 14471 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:14.620096  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 242133us
W20260709 12:28:14.621308 14286 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.254s	user 0.010s	sys 0.005s
I20260709 12:28:14.611321 14430 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:14.619609  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 451846us
I20260709 12:28:14.631495  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:14.621443 14429 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:14.622788 14458 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:14.643816  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.644055  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:14.623392 14529 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.650408  7729 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:14.647166  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:14.650714  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.651844  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:14.649293  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 522880us
I20260709 12:28:14.658653  8407 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:14.632315 14464 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:14.629056 14526 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.633395 14524 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.660238 14069 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:14.660449 14480 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.712s	user 0.015s	sys 0.000s
I20260709 12:28:14.648878 14463 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.660951 14529 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:14.644585 14430 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.643944 14108 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.661549 14227 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
I20260709 12:28:14.648679 14528 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.613999 14129 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.611691 14435 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:14.671289  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.694924  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.629477 14496 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.651400 14282 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:14.677888  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.679714  7660 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:28:14.663661 14512 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 12:28:14.669672 14286 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:14.680841  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.662079 14356 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:14.637503 14253 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:14.700385 14227 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.754822  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:14.756238  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:14.701527 14455 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.675729 14069 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.686307 14480 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.707947 14388 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:14.767937  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.769381  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.769652 14480 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.770668 14480 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.692447 14463 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.772487  8413 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:14.773270 14319 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.774148 14069 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:14.712404  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:14.713104 14531 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.779687 14531 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.780067 14253 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.713601 14451 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.783466  7810 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:14.783017 14451 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.714774 14373 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:14.713625 14459 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.722468 14435 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.724644  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:14.726886  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:14.729761  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1021) took 1339 ms. Trace:
I20260709 12:28:14.740806 14282 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:14.733456 14129 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.735096 14108 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.736377 14517 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.738498 14409 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:14.664933 13997 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.622983 14515 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.672187 14526 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:14.682700 14302 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:14.654851 14363 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:28:14.650066 14498 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
W20260709 12:28:14.749606  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 74319us
I20260709 12:28:14.809725  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:14.745812 14391 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:28:14.687825 14430 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:14.811087 14454 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:14.811646 14454 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.812175 14454 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:14.746099 14286 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:14.813995 14286 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:14.748715 14312 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:28:14.785784 14227 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.816002 14409 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.816496 14227 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.816679 14409 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.755846 14356 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:14.817114 14227 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.817893 14356 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:14.787932  8412 rpcz_store.cc:276] 0709 12:28:13.391611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.392778 (+  1167us) service_pool.cc:224] Handling call
0709 12:28:13.418175 (+ 25397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.486518 (+ 68343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.729340 (+1242822us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:14.729663 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111038848}
I20260709 12:28:14.732609 14115 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.674039 14365 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:14.745985 14534 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.757051 14533 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:14.747381 14523 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.794445  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:14.694007 14464 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:28:14.798424  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1202) took 1426 ms. Trace:
I20260709 12:28:14.801738 14302 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:14.815666  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:14.746646 14528 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.821177 14108 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:15.007985 14456 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.035s	user 0.010s	sys 0.005s
W20260709 12:28:14.855875  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 112848us
W20260709 12:28:15.242739  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.990113  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3191) took 2420 ms. Trace:
I20260709 12:28:14.948381  8413 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 12:28:15.243748  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.821694  7803 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:14.947746  7804 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.001211  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3200) took 1696 ms. Trace:
W20260709 12:28:15.244634  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 429030us
I20260709 12:28:14.821890  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2923) took 1016 ms. Trace:
I20260709 12:28:14.892195  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.015241  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:15.241688  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 613633us
I20260709 12:28:14.821763  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:14.866602  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 218687us
I20260709 12:28:14.927011  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.257756  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:15.257889  8409 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:15.029987 14356 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.255892  7733 rpcz_store.cc:276] 0709 12:28:13.304466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.472482 (+1168016us) service_pool.cc:224] Handling call
0709 12:28:14.528670 (+ 56188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.578668 (+ 49998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.578708 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.579403 (+   695us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.748281 (+168878us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:14.748346 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.748441 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.000936 (+252495us) raft_consensus.cc:1685] finished
0709 12:28:15.000961 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.001128 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60829,"raft.queue_time_us":61296,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":15252,"thread_start_us":168452,"threads_started":1,"wal-append.queue_time_us":403557}
W20260709 12:28:15.259927  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.260248  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.260910  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:15.261044 14356 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.254607  7731 rpcz_store.cc:276] 0709 12:28:12.569262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.718437 (+1149175us) service_pool.cc:224] Handling call
0709 12:28:13.725514 (+  7077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.725735 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:13.777515 (+ 51780us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.777565 (+1000050us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b180043d0e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:28:14.989479 (+211914us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.989562 (+    83us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:14.989631 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.989734 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.989917 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49346,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":18778,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":41478528,"thread_start_us":211529,"threads_started":1}
I20260709 12:28:15.268049  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:15.237576 14424 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.996s	user 0.012s	sys 0.000s
I20260709 12:28:14.796548 14282 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:15.022018 14504 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.003s
I20260709 12:28:15.255260  7657 rpcz_store.cc:276] 0709 12:28:13.371878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.372055 (+   177us) service_pool.cc:224] Handling call
0709 12:28:13.402739 (+ 30684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.403049 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.403087 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.428039 (+ 24952us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.824496 (+396457us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:13.824537 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.824612 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.798158 (+973546us) raft_consensus.cc:1685] finished
0709 12:28:14.798182 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.798355 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":355732,"threads_started":1,"wal-append.queue_time_us":745912}
I20260709 12:28:15.261121  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:15.014873 14516 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.256181  7654 rpcz_store.cc:276] 0709 12:28:13.781383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.784866 (+  3483us) service_pool.cc:224] Handling call
0709 12:28:14.724384 (+939518us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:28:14.797947 (+ 73563us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28457216}
I20260709 12:28:15.313771  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:15.296464  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.318614  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.324447  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1073) took 1086 ms. Trace:
I20260709 12:28:15.008724 14505 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:28:15.191087 14471 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.002s
I20260709 12:28:15.320874  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.350983  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.247664 14515 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:15.241765 14460 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.931s	user 0.014s	sys 0.018s
W20260709 12:28:15.029680 14165 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.669s	user 0.002s	sys 0.007s
W20260709 12:28:15.321956  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.358301  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97111.5us
I20260709 12:28:15.380208  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:15.380426  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:15.381276  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:15.254720 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.013s	sys 0.013s
I20260709 12:28:15.247557 14302 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:14.995935 14463 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:14.995425 14227 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:14.947819 14087 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.365628  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.313858  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1367) took 1383 ms. Trace:
I20260709 12:28:15.244287 14533 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:15.368393  7661 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2909) took 2314 ms (client timeout 2052 ms). Trace:
I20260709 12:28:14.943429 14115 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:15.364639  7806 rpcz_store.cc:276] 0709 12:28:14.238206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.296571 (+ 58365us) service_pool.cc:224] Handling call
0709 12:28:15.324139 (+1027568us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.324342 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262123392}
I20260709 12:28:15.396528  8414 rpcz_store.cc:276] 0709 12:28:13.930432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.930683 (+   251us) service_pool.cc:224] Handling call
0709 12:28:13.968799 (+ 38116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.050582 (+ 81783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.313579 (+1262997us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b540013fbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:15.313786 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96015104}
I20260709 12:28:14.991983 14493 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:15.262272 14498 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.254659 14456 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:15.373960  7653 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:15.366106 14504 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.376071  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.412462  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.413661 14453 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.413836  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.414920  7653 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:15.416286  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.262526  7624 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:15.377315  8413 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.320642 14496 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.378396  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.262488 14286 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:15.360786 14278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.006s	sys 0.010s
I20260709 12:28:15.382082  7733 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:14.798609 13997 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.406176 14115 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.425930 14498 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:15.435166 14428 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.437336 14115 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.438396 14453 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:15.439035 14453 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.439971  7735 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:28:15.440233  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2718) took 1193 ms. Trace:
I20260709 12:28:15.440639  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3131) took 2053 ms. Trace:
I20260709 12:28:15.441030  7735 rpcz_store.cc:276] 0709 12:28:13.386950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.430901 (+2043951us) service_pool.cc:224] Handling call
0709 12:28:15.439603 (+  8702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.440395 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.440584 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.441747 14453 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.443666 14409 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.445959 14385 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:15.446491 14385 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:15.446969 14286 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.440667  7805 rpcz_store.cc:276] 0709 12:28:14.246565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.353553 (+106988us) service_pool.cc:224] Handling call
0709 12:28:14.354090 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.505337 (+151247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.439975 (+934638us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400134cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:15.440171 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344997248}
I20260709 12:28:15.142319 14430 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.018970 14454 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:14.999190 14069 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.145105 14529 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.495720  7733 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:28:15.496558  7733 pending_rounds.cc:85] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 12
I20260709 12:28:15.496869  7733 pending_rounds.cc:107] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:28:15.497138  7733 raft_consensus.cc:2889] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:15.370332 14435 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:15.391827  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.393007  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:15.500229  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107551us
I20260709 12:28:15.435195 14498 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:15.264242 14523 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.312435 14258 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 12:28:15.396574  7731 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
W20260709 12:28:15.385816 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.015s	sys 0.015s
I20260709 12:28:15.398561  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.377053 14464 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:15.370675 14505 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:15.400274 14533 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:15.397333  7661 rpcz_store.cc:269] 0709 12:28:13.054043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.201735 (+147692us) service_pool.cc:224] Handling call
0709 12:28:15.368100 (+2166365us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003999d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.368292 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484275584}
I20260709 12:28:15.119987 14458 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:15.275863 14534 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.319259 14457 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.349221 14526 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.412462  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:15.419001  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 156585us
I20260709 12:28:15.421979  8407 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:15.370826 14516 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.506389 14498 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:14.796933 14480 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.384805 14165 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.426997  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:15.518471 14286 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:15.427850  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.382723 14460 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:15.430378 14278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.007s	sys 0.011s
I20260709 12:28:15.501971  8412 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:28:15.502875  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3134) took 2053 ms. Trace:
I20260709 12:28:15.547832  7733 rpcz_store.cc:276] 0709 12:28:13.449658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.454853 (+2005195us) service_pool.cc:224] Handling call
0709 12:28:15.494983 (+ 40130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.497600 (+  2617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.502798 (+  5198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.502601  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1216) took 1058 ms. Trace:
I20260709 12:28:15.509044  7657 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:28:15.522076  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2928) took 1659 ms (client timeout 2056 ms). Trace:
I20260709 12:28:15.549618  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:15.549486  7659 rpcz_store.cc:269] 0709 12:28:13.862680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.951548 (+ 88868us) service_pool.cc:224] Handling call
0709 12:28:15.521843 (+1570295us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b54001a3bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.522009 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210120576}
I20260709 12:28:15.550423  7657 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:15.551548  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.551479  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.552702  7659 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:15.535519 14458 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.548738  7662 rpcz_store.cc:276] 0709 12:28:14.444192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.379113 (+934921us) service_pool.cc:224] Handling call
0709 12:28:15.502527 (+123414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.553965  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.554808  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.555670 14458 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.522440  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1068) took 1285 ms. Trace:
I20260709 12:28:15.555691  7662 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:28:15.553965  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.542883 14115 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:28:15.557878 14515 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:15.558297 14515 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:15.558532  7733 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.558938 13997 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.559275  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1337) took 2070 ms. Trace:
I20260709 12:28:15.531035  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:15.559988  8408 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.495754 14528 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.560168 14165 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.560971  7736 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:15.556430  7804 rpcz_store.cc:276] 0709 12:28:14.236758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.317308 (+ 80550us) service_pool.cc:224] Handling call
0709 12:28:14.336390 (+ 19082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.255847 (+919457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.522194 (+266347us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:15.522380 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402279168}
I20260709 12:28:15.565124  7733 rpcz_store.cc:276] 0709 12:28:13.489460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.558016 (+2068556us) service_pool.cc:224] Handling call
0709 12:28:15.558358 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.558996 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.559193 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.539649  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.540841  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.402798 14463 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:15.567651  8407 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:15.568509  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.568966  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.569222  8413 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:15.569744  8407 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:15.567883 14463 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:15.570943 14463 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:15.556685  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1205) took 2175 ms. Trace:
I20260709 12:28:15.571641  7662 rpcz_store.cc:276] 0709 12:28:13.381771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.554524 (+2172753us) service_pool.cc:224] Handling call
0709 12:28:15.556624 (+  2100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.572963 14529 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.567307  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1213) took 1116 ms. Trace:
I20260709 12:28:15.575816  7659 rpcz_store.cc:276] 0709 12:28:14.436873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.551049 (+1114176us) service_pool.cc:224] Handling call
0709 12:28:15.553419 (+  2370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.576272  8409 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:28:15.577656  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1211) took 2076 ms. Trace:
I20260709 12:28:15.578114 14526 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.578306  8408 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:15.575368 14534 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:15.541565  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89177.9us
I20260709 12:28:15.546607  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.559376 14515 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.531910  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.578104  7659 rpcz_store.cc:276] 0709 12:28:13.501212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.576753 (+2075541us) service_pool.cc:224] Handling call
0709 12:28:15.577113 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.577302 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.577330 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.577377 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:15.577413 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.577471 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.577589 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.586194  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.586441  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:15.519927  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:15.588320  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68488.3us
I20260709 12:28:15.552456  7657 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:15.592758 14515 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.594135  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2913) took 2083 ms. Trace:
I20260709 12:28:15.595373  7731 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:15.596778  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.598258 14458 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:15.598805 14458 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.599195 14458 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:15.599540 14435 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:15.600216 14435 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:15.594676  7661 rpcz_store.cc:276] 0709 12:28:13.510481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.582553 (+2072072us) service_pool.cc:224] Handling call
0709 12:28:15.593329 (+ 10776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.593707 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.593747 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.593802 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:15.593843 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.593915 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.594067 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1767,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114}
I20260709 12:28:15.603605 14517 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.610674  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:15.607758  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:15.630245  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 383988us
I20260709 12:28:15.630661  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:15.632544  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:15.544106 14365 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.544729 14286 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.495718 14385 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.651171 14385 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:15.558364 14516 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.636677  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.636919  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:15.637297  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:15.614975 14251 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:15.556773 14480 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.649297 14504 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.662775  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:15.678968  8406 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:15.539530 14457 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.673804  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.681283  7659 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:15.655763 14286 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.635525 14453 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.638258 14526 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:15.682932  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3195) took 2471 ms. Trace:
I20260709 12:28:15.655345 14165 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:15.642746 14543 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:15.629875 14464 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:28:15.693411  7732 rpcz_store.cc:276] 0709 12:28:13.211424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.964481 (+753057us) service_pool.cc:224] Handling call
0709 12:28:14.002767 (+ 38286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.159636 (+156869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.682526 (+1522890us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b540037e954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:15.682810 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114912512}
W20260709 12:28:15.698487  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.667929 14517 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:15.702679  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.705613  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1336) took 2290 ms. Trace:
I20260709 12:28:15.684782 14258 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:15.706087  7735 rpcz_store.cc:276] 0709 12:28:13.414820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.448686 (+2033866us) service_pool.cc:224] Handling call
0709 12:28:15.449045 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.449269 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:15.528762 (+ 79493us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.705220 (+176458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.705277 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:15.705325 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.705411 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.705553 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2789,"prepare.queue_time_us":200551,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"thread_start_us":249115,"threads_started":2}
I20260709 12:28:15.681493 14251 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:15.712124  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:15.712939  7735 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:15.714125 14464 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:28:15.714591 14548 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:28:15.692641 14526 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.715296 14454 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.715325 14251 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.716316  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.716886  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1338) took 2227 ms. Trace:
I20260709 12:28:15.717095  7658 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:15.714646 14464 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:15.718127  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:15.718962 14464 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.719352 14517 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.720306 14464 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:15.721788 14517 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:15.723744 14545 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:28:15.724282 14258 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:15.725370 14258 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.714100  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:15.726495  7735 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:15.727730  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:15.733667 14453 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:15.733829 14430 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.734289 14453 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.717345  7733 rpcz_store.cc:276] 0709 12:28:13.489853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.566052 (+2076199us) service_pool.cc:224] Handling call
0709 12:28:15.566511 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.566945 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.566982 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.567563 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.682253 (+114690us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:15.682303 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.682381 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.716610 (+ 34229us) raft_consensus.cc:1685] finished
0709 12:28:15.716634 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.716819 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1261,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":117,"thread_start_us":114254,"threads_started":1,"wal-append.queue_time_us":144358}
I20260709 12:28:15.738227  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.695690 14165 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.744944  7658 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:15.748931  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2920) took 2134 ms. Trace:
I20260709 12:28:15.749384  7658 rpcz_store.cc:276] 0709 12:28:13.614496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.719064 (+2104568us) service_pool.cc:224] Handling call
0709 12:28:15.743823 (+ 24759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.748464 (+  4641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.748847 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.750664  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.746001  8406 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:15.698333 14543 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.755810  7658 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:15.758067  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2921) took 2127 ms. Trace:
I20260709 12:28:15.758536  7658 rpcz_store.cc:276] 0709 12:28:13.630976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.754939 (+2123963us) service_pool.cc:224] Handling call
0709 12:28:15.755314 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.757682 (+  2368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.757991 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.760664  7658 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:28:15.761345  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1215) took 1317 ms. Trace:
I20260709 12:28:15.761714  7658 rpcz_store.cc:276] 0709 12:28:14.443763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.760016 (+1316253us) service_pool.cc:224] Handling call
0709 12:28:15.760326 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.761107 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.761292 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.734797 14453 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:15.742945  7735 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:15.666293 14505 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.715545 14526 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:15.747013 14550 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.271s	user 0.000s	sys 0.002s
I20260709 12:28:15.766623  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2863) took 1103 ms. Trace:
I20260709 12:28:15.775447  7658 rpcz_store.cc:276] 0709 12:28:14.663459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.765510 (+1102051us) service_pool.cc:224] Handling call
0709 12:28:15.765877 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.766075 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.766112 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.766166 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:15.766211 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.766271 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.766554 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.753759 14165 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:15.752737 14069 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.778008  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2864) took 1114 ms. Trace:
I20260709 12:28:15.763423 14480 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:28:15.780524  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1339) took 1878 ms. Trace:
I20260709 12:28:15.780407  7658 rpcz_store.cc:276] 0709 12:28:14.663807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.777022 (+1113215us) service_pool.cc:224] Handling call
0709 12:28:15.777452 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.777675 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.777699 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.777739 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:15.777773 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.777829 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.777947 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.781256  7733 rpcz_store.cc:276] 0709 12:28:13.901670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.770382 (+1868712us) service_pool.cc:224] Handling call
0709 12:28:15.780033 (+  9651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.780177 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.780221 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.780265 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:15.780301 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.780356 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.780465 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.780777  8406 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:15.765417 14543 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:15.769022 14504 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.796202 14517 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.796900 14517 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.802438 14463 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.804723 14459 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:15.805433 14459 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:15.804778 14458 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.808779  7811 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:15.809533  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2709) took 1927 ms. Trace:
W20260709 12:28:15.809777  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.810231 14458 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.812095  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3202) took 1140 ms. Trace:
I20260709 12:28:15.812578  7733 rpcz_store.cc:276] 0709 12:28:14.671860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.782370 (+1110510us) service_pool.cc:224] Handling call
0709 12:28:15.811443 (+ 29073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.811699 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.811734 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.811791 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:15.811825 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.811893 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.812016 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.819725 14395 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.820068 14457 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.824973  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3187) took 3257 ms. Trace:
I20260709 12:28:15.825589 14463 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:28:15.795037 14302 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.827800  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2926) took 2029 ms. Trace:
I20260709 12:28:15.827666 14505 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:15.822506 14543 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.787719 14504 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.825559 14533 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:15.837348 14543 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.838438 14533 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.843865 14523 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.809857  7811 rpcz_store.cc:276] 0709 12:28:13.882346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.895152 (+ 12806us) service_pool.cc:224] Handling call
0709 12:28:15.808102 (+1912950us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:28:15.808376 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.809206 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.809454 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6825216}
I20260709 12:28:15.832957  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3237) took 1139 ms. Trace:
W20260709 12:28:15.812155 14514 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.616s	user 0.007s	sys 0.000s
I20260709 12:28:15.828694  7660 rpcz_store.cc:276] 0709 12:28:13.798953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.457129 (+658176us) service_pool.cc:224] Handling call
0709 12:28:15.827506 (+1370377us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540028c654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.827707 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393132928}
I20260709 12:28:15.839411  7729 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:15.861929  7733 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:15.861008  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1204) took 2479 ms. Trace:
I20260709 12:28:15.806504 14459 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:15.838613 14504 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.847760  7810 rpcz_store.cc:276] 0709 12:28:14.690665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.712091 (+ 21426us) service_pool.cc:224] Handling call
0709 12:28:15.829784 (+1117693us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b54003c55d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.829937 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19387776}
I20260709 12:28:15.852632 14341 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.854307 14531 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:15.828989 14302 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:15.856323  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:15.857393 14517 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:15.835163  7659 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:15.825551  7734 rpcz_store.cc:276] 0709 12:28:12.568004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.812326 (+244322us) service_pool.cc:224] Handling call
0709 12:28:12.812759 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.357128 (+1544369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.824634 (+1467506us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400072bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:15.824901 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3549897600}
I20260709 12:28:15.862797  7662 rpcz_store.cc:276] 0709 12:28:13.381476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.572390 (+2190914us) service_pool.cc:224] Handling call
0709 12:28:15.572753 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.573847 (+  1094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.573884 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.574386 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.749558 (+175172us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:15.749590 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.749647 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.860750 (+111103us) raft_consensus.cc:1685] finished
0709 12:28:15.860784 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.860941 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":514,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":174778,"threads_started":1,"wal-append.queue_time_us":238997}
I20260709 12:28:15.864040  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3203) took 1191 ms. Trace:
I20260709 12:28:15.834359  7657 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:15.869027  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.866602 14523 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.912853  7733 rpcz_store.cc:276] 0709 12:28:14.672254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.813570 (+1141316us) service_pool.cc:224] Handling call
0709 12:28:15.861723 (+ 48153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.863764 (+  2041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.863976 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:15.913529  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.914664  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:15.876482  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.924045  8405 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:15.921958  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:15.930104  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3238) took 1238 ms. Trace:
I20260709 12:28:15.930925  7803 rpcz_store.cc:276] 0709 12:28:14.691886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.769084 (+ 77198us) service_pool.cc:224] Handling call
0709 12:28:15.929799 (+1160715us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b54003c4bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:15.930005 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118786304}
I20260709 12:28:15.967862  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.968708  7657 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:15.970852  7734 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:15.971498  7733 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:15.971966  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3206) took 1298 ms. Trace:
I20260709 12:28:15.972491  7734 rpcz_store.cc:276] 0709 12:28:14.673641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.916480 (+1242839us) service_pool.cc:224] Handling call
0709 12:28:15.970644 (+ 54164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.971652 (+  1008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.971890 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.974395  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.978808  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:15.988508  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3205) took 1315 ms. Trace:
I20260709 12:28:15.989204  7733 rpcz_store.cc:276] 0709 12:28:14.672925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.916198 (+1243273us) service_pool.cc:224] Handling call
0709 12:28:15.971223 (+ 55025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.987900 (+ 16677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.988391 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.924947 14459 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:15.931089 14543 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:28:15.970391 14514 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:28:15.930836 14517 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.934404 14463 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:15.871389 14282 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.018674  7731 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:16.020390  7733 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:16.031615  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3204) took 1359 ms. Trace:
I20260709 12:28:16.032285  7729 rpcz_store.cc:276] 0709 12:28:14.672599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.350677 (+678078us) service_pool.cc:224] Handling call
0709 12:28:16.031392 (+680715us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.031554 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069820288}
I20260709 12:28:16.033846  7729 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:16.034541  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3213) took 1295 ms. Trace:
I20260709 12:28:16.031595 14463 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:16.040540 14463 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.023310 14543 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:16.043987 14543 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.048650  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2854) took 3578 ms. Trace:
I20260709 12:28:16.049319 14282 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.049242  7656 rpcz_store.cc:276] 0709 12:28:12.470446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.470633 (+   187us) service_pool.cc:224] Handling call
0709 12:28:12.483780 (+ 13147us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:28:12.489454 (+  5674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:12.496801 (+  7347us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:13.302467 (+805666us) spinlock_profiling.cc:243] Waited 9.52 ms on lock 0x7b1800334480. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:28:13.629698 (+327231us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.882964 (+253266us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.892643 (+  9679us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.892828 (+   185us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.892865 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.892950 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.811469 (+1918519us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:16.048187 (+236718us) raft_consensus.cc:1685] finished
0709 12:28:16.048225 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.048447 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21154,"prepare.queue_time_us":371797,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":156937,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":33303,"spinlock_wait_cycles":44327040,"thread_start_us":418684,"threads_started":2,"wal-append.queue_time_us":413410}
I20260709 12:28:16.054239  7656 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:16.055179  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2869) took 1386 ms. Trace:
I20260709 12:28:15.933751 14523 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.055821  7656 rpcz_store.cc:276] 0709 12:28:14.668744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.053216 (+1384472us) service_pool.cc:224] Handling call
0709 12:28:16.053720 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.054797 (+  1077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.055102 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.058416  7656 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:16.059687  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2870) took 1390 ms. Trace:
I20260709 12:28:16.054925 14498 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.060222  7656 rpcz_store.cc:276] 0709 12:28:14.669047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.057677 (+1388630us) service_pool.cc:224] Handling call
0709 12:28:16.058159 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.059310 (+  1151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.059624 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.061003  7653 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:16.050381  7807 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:16.023114 14517 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.025981  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3212) took 1287 ms. Trace:
I20260709 12:28:16.021670 14460 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.068318  7729 rpcz_store.cc:276] 0709 12:28:14.738662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.033329 (+1294667us) service_pool.cc:224] Handling call
0709 12:28:16.033671 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.034253 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.034487 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.070068  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3214) took 1331 ms. Trace:
I20260709 12:28:16.070467  7729 rpcz_store.cc:276] 0709 12:28:14.738949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.069182 (+1330233us) service_pool.cc:224] Handling call
0709 12:28:16.069505 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.069726 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.069762 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.069803 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:16.069831 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.069885 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.070008 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.065028  7733 rpcz_store.cc:276] 0709 12:28:14.738343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.990274 (+1251931us) service_pool.cc:224] Handling call
0709 12:28:16.020174 (+ 29900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.024582 (+  4408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.025772 (+  1190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:16.073235  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.073478  7657 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:16.079576  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2871) took 1410 ms. Trace:
I20260709 12:28:16.080840  7656 rpcz_store.cc:276] 0709 12:28:14.669354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.064998 (+1395644us) service_pool.cc:224] Handling call
0709 12:28:16.066226 (+  1228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.066427 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.066454 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.066520 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:16.066574 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.066640 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.079496 (+ 12856us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.083259  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.062530 14286 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.087697  7656 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:16.088302  7662 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:16.092545  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.105937  8414 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:16.105293  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.106906  7656 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:16.089704 14516 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.108637  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.109350  7656 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:16.113235 14516 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.111922  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2873) took 1442 ms. Trace:
I20260709 12:28:16.121433  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:16.129894  8411 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:16.094440 14453 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.129515  7656 rpcz_store.cc:276] 0709 12:28:14.669953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.110970 (+1441017us) service_pool.cc:224] Handling call
0709 12:28:16.111356 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.111570 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.111597 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.111644 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:16.111675 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.111733 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.111843 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.132506  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:16.124053  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2861) took 3631 ms. Trace:
I20260709 12:28:16.119403 14516 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.134994  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3185) took 3567 ms. Trace:
I20260709 12:28:16.105937 14341 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:28:16.141434  7730 rpcz_store.cc:276] 0709 12:28:12.567331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.592826 (+ 25495us) service_pool.cc:224] Handling call
0709 12:28:12.618172 (+ 25346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.869734 (+251562us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:13.329426 (+459692us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.761580 (+432154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:13.761656 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:13.761706 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.134672 (+2372966us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400144bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:16.134729 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.134905 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9435,"prepare.queue_time_us":368350,"prepare.run_cpu_time_us":211,"prepare.run_wall_time_us":207,"raft.queue_time_us":12259,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":17876,"spinlock_wait_cycles":577578752,"thread_start_us":817013,"threads_started":2,"wal-append.queue_time_us":844548,"wal-append.run_cpu_time_us":4439,"wal-append.run_wall_time_us":1358751}
I20260709 12:28:16.137624  7655 rpcz_store.cc:276] 0709 12:28:12.492422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.492663 (+   241us) service_pool.cc:224] Handling call
0709 12:28:12.503372 (+ 10709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.721476 (+218104us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:13.371146 (+649670us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.517672 (+2146526us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b180039e8c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:28:15.517796 (+   124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.517849 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:15.517902 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.123753 (+605851us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54000d07d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:16.123825 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.123983 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145938,"prepare.queue_time_us":168876,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":179861,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":33099,"spinlock_wait_cycles":615193600,"thread_start_us":984857,"threads_started":3,"wal-append.queue_time_us":499911,"wal-append.run_cpu_time_us":3394,"wal-append.run_wall_time_us":1669129}
W20260709 12:28:16.164743 14515 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.004s
W20260709 12:28:16.177322  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.140244 14115 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:28:16.170424 14549 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.722s	user 0.004s	sys 0.000s
I20260709 12:28:16.187045  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.189198  8411 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:16.192312  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:16.192386  7730 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:28:16.190641  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:16.193679  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:16.197392 14453 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:16.197614 14562 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:16.199226  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2866) took 1534 ms. Trace:
I20260709 12:28:16.199379 14526 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.199687  7658 rpcz_store.cc:276] 0709 12:28:14.664396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.782209 (+1117813us) service_pool.cc:224] Handling call
0709 12:28:15.782540 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.782915 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.782950 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.786686 (+  3736us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.023995 (+237309us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:16.024074 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.024158 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.198944 (+174786us) raft_consensus.cc:1685] finished
0709 12:28:16.198978 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.199152 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":462,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4122,"thread_start_us":175322,"threads_started":1,"wal-append.queue_time_us":259353}
I20260709 12:28:16.201289 14515 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:16.201782 14526 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.202061 14515 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.202610 14515 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.203860  8405 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:16.204027 14498 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.204813 14498 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:16.189905 14435 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.001s
I20260709 12:28:16.185415 14115 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.216202  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3216) took 1395 ms. Trace:
I20260709 12:28:16.234457  7655 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:28:16.234742  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:16.211145  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2875) took 1540 ms. Trace:
W20260709 12:28:16.207404  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.234587  7733 rpcz_store.cc:276] 0709 12:28:14.820365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.072239 (+1251874us) service_pool.cc:224] Handling call
0709 12:28:16.101482 (+ 29243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.101855 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:16.109281 (+  7426us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.215803 (+106522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.215858 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:16.215901 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.215971 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.216109 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7715,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":496,"thread_start_us":105642,"threads_started":1}
I20260709 12:28:16.232861  7730 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:16.239456  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:16.177908 14516 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:16.244796  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2876) took 1567 ms. Trace:
W20260709 12:28:16.238770  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.246369  7655 rpcz_store.cc:276] 0709 12:28:14.677173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.206303 (+1529130us) service_pool.cc:224] Handling call
0709 12:28:16.234200 (+ 27897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.244446 (+ 10246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.244725 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.242547  8405 raft_consensus.cc:2368] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:28:16.195595 14460 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:16.205206 14498 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.236356  7654 rpcz_store.cc:276] 0709 12:28:14.670506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.200066 (+1529560us) service_pool.cc:224] Handling call
0709 12:28:16.200637 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.210560 (+  9923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.210610 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.210691 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:16.210740 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.210840 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.211047 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.247738  7733 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:28:16.235033 14563 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:28:16.238916  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:16.231966 14543 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.181753 14165 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.238649 14115 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:16.259172  7730 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:16.261509  7624 leader_election.cc:400] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:28:16.263386  7624 leader_election.cc:403] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:28:16.263943  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.246176 14516 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:16.270975  7733 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:16.248360 14341 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.272508  7735 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 16. Ops: []
I20260709 12:28:16.254752 14498 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.279145  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1207) took 1418 ms. Trace:
I20260709 12:28:16.266206 14286 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.262782 14517 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.237012 14435 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.281463  8415 rpcz_store.cc:276] 0709 12:28:14.860495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.861351 (+   856us) service_pool.cc:224] Handling call
0709 12:28:14.865343 (+  3992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.895057 (+ 29714us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:14.926300 (+ 31243us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.948162 (+ 21862us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.365430 (+417268us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:15.365478 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.365556 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.133495 (+767939us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:16.277363 (+143868us) raft_consensus.cc:1685] finished
0709 12:28:16.277404 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.279017 (+  1613us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131725,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":5648128,"thread_start_us":354461,"threads_started":1,"wal-append.queue_time_us":436893}
I20260709 12:28:16.297026 14504 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.297801 14504 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.299401 14531 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.300223 14531 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.306458  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.311813  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3215) took 1491 ms. Trace:
I20260709 12:28:16.311892 14463 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:16.313330  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2879) took 1628 ms. Trace:
I20260709 12:28:16.312832 14463 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.313815  7654 rpcz_store.cc:276] 0709 12:28:14.684806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.312492 (+1627686us) service_pool.cc:224] Handling call
0709 12:28:16.312834 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.313017 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.313040 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.313085 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:16.313112 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.313160 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.313272 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.315128  7655 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:16.315452  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2872) took 1645 ms. Trace:
I20260709 12:28:16.315824  7657 rpcz_store.cc:276] 0709 12:28:14.669627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.074586 (+1404959us) service_pool.cc:224] Handling call
0709 12:28:16.074904 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.090380 (+ 15476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.090413 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.091013 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.214909 (+123896us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:16.214952 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.215021 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.315165 (+100144us) raft_consensus.cc:1685] finished
0709 12:28:16.315189 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.315392 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":342,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2691,"spinlock_wait_cycles":5504,"thread_start_us":95340,"threads_started":1,"wal-append.queue_time_us":165517}
I20260709 12:28:16.312547  7729 rpcz_store.cc:276] 0709 12:28:14.820045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.071402 (+1251357us) service_pool.cc:224] Handling call
0709 12:28:16.072010 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.072410 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:16.072443 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.072942 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.248731 (+175789us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:16.248788 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.248855 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.311547 (+ 62692us) raft_consensus.cc:1685] finished
0709 12:28:16.311570 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.311741 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":4774,"thread_start_us":140998,"threads_started":1,"wal-append.queue_time_us":215726}
I20260709 12:28:16.332252  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.338436 14435 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.340023 14463 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.303031 14531 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:16.264163 14543 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.280493 14341 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:16.360543  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3121) took 3598 ms (client timeout 3117 ms). Trace:
I20260709 12:28:16.361021  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:16.380999  7737 rpcz_store.cc:269] 0709 12:28:12.762491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.248141 (+485650us) service_pool.cc:224] Handling call
0709 12:28:16.360324 (+3112183us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54002f1554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.360469 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3439963392}
I20260709 12:28:16.361234  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3198) took 3074 ms. Trace:
I20260709 12:28:16.382702  7736 rpcz_store.cc:276] 0709 12:28:13.286666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.361299 (+1074633us) service_pool.cc:224] Handling call
0709 12:28:14.452538 (+ 91239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.561404 (+1108866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.360972 (+799568us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002486d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:16.361156 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548709632}
I20260709 12:28:16.316087 14517 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:16.384260  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:16.361632  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 105453us
I20260709 12:28:16.363265  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2867) took 1698 ms. Trace:
I20260709 12:28:16.369879  7728 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:16.329584 14415 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.399649  8408 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:16.399866 14415 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.402088 14543 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.386802  7662 rpcz_store.cc:276] 0709 12:28:14.664716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.975560 (+1310844us) service_pool.cc:224] Handling call
0709 12:28:16.074383 (+ 98823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.089089 (+ 14706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.362928 (+273839us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54000f3f50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.363141 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226397440}
I20260709 12:28:16.405110  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.405864  7662 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:16.342756 14448 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.407433 14286 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:16.407562 14448 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.409809 14504 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:16.410030 14448 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.332634 14516 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.375523 14531 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:16.389843 14568 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:28:16.390945 14566 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:28:16.391788 14505 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.004s
W20260709 12:28:16.393787  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0079233249998651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 12:28:16.394629  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:16.412089 14416 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:16.412577 14416 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.395401  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:16.412966 14416 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.413235  7730 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.395542 14451 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:16.416108 14451 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:16.416947 14341 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.417771 14451 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:16.396569 14526 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.418757 14527 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:16.419795 14527 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:16.420276 14527 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:16.421173  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.421911  7804 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:16.421578 14527 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.419515 14526 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.406884  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.427562  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.429039  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.429730  7803 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:16.430058  7730 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:16.430346  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.431917  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:16.433455  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:16.433744  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.434146  7811 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.436100  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.437448  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.435750  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:16.438642  7730 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:16.439615  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.440080  7655 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:16.440508  7806 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.441579  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.439049  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:16.442451  7655 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:16.442752  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:16.432412  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.443666  7805 raft_consensus.cc:2376] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18. Re-sending same reply.
I20260709 12:28:16.445212  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.445979  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.446631  7810 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:16.447525  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.448849  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:16.451550  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.452248  7804 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:16.450920  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.445417  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:16.454653  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.443948  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 12.
I20260709 12:28:16.455021  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:16.455809  7736 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:16.456234  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.426278 14415 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:16.457701  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.458484 14286 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:16.459367 14286 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.459801  7810 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:16.460664 14543 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.465159  7811 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:16.465008  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:16.604929  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.620021  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.419059 14451 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:16.502225  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.529798  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.466480  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.499688  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.558827  7806 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:16.627947  7803 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:16.628119  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:16.628839  7732 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:16.618578  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.629123  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.491551  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.630031  7805 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:16.630327  7812 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.626187 14451 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:16.642566  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.621373  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2874) took 1951 ms. Trace:
I20260709 12:28:16.473726  7804 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:16.642640  7805 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:28:16.644693  7804 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:28:16.426545 14527 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:16.627381  7810 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.585016  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.649266  7655 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:16.465885  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.635798  7660 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:16.630697  7736 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:16.641187  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.594658 14517 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14.
I20260709 12:28:16.651806  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.653671  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.494630 14526 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.603924 14286 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.523890 14463 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:16.524130 14464 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:16.641146 14449 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:16.631026 14460 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:16.664660  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.676548  7661 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:28:16.666515  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.674417  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
W20260709 12:28:16.677173  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.676483  7812 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:16.678102  7805 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:28:16.600807 14115 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.572168 14515 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.678838  7806 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 12:28:16.492909 14543 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:16.679687  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 214751us
I20260709 12:28:16.685617  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.645944 14416 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.702433  7658 raft_consensus.cc:2368] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18.
I20260709 12:28:16.573275 14574 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 12:28:16.690428  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.697880 14543 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.700110 14515 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.700588  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:16.706542  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.515s (SENT)
I20260709 12:28:16.676016 14449 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:16.716629  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.717351 14449 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:16.718598 14449 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:16.680727 14464 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:16.720176 14464 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.720542  7732 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:16.725071  7662 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:16.727058  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.534s (SENT)
I20260709 12:28:16.727829  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.732959  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.733722  7806 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:16.643452  7656 rpcz_store.cc:276] 0709 12:28:14.670225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.167662 (+1497437us) service_pool.cc:224] Handling call
0709 12:28:16.168114 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.168422 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:16.334520 (+166098us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.620988 (+286468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.621047 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:16.621095 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.621165 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.621309 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":331440,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":99,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9475,"thread_start_us":442019,"threads_started":2}
I20260709 12:28:16.735854  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.740945  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.743770 14460 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:16.744977 14460 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:16.751894  7627 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.640s (SENT)
I20260709 12:28:16.688984 14524 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:16.770845  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.773025  7656 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:16.773672 14524 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:16.774111  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.774922  7656 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:16.776041  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:16.777436  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.778088  7656 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
W20260709 12:28:16.780032 14575 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:28:16.785037  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.785984  7812 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:16.785830  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.787310  7812 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:16.787766  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.789748  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.791007  7809 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:28:16.763726 14540 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.338s	user 0.003s	sys 0.030s
I20260709 12:28:16.734977 14501 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:16.779073  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.798563 14341 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:16.811031  8409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1031) took 1431 ms (client timeout 1869 ms). Trace:
I20260709 12:28:16.737316 14517 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:16.813130  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:16.813287  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.750100 14530 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.804891  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:16.811651  8409 rpcz_store.cc:269] 0709 12:28:15.369241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.412143 (+ 42902us) service_pool.cc:224] Handling call
0709 12:28:16.800629 (+1388486us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.800822 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91031296}
I20260709 12:28:16.814033  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.687561 14451 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.821735  7806 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:16.823734  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.825984  7656 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
W20260709 12:28:16.830013  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.829730  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:16.832278 14579 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:28:16.812930 14341 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:16.853986  7731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1348) took 1263 ms (client timeout 1579 ms). Trace:
I20260709 12:28:16.854486  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3188) took 4286 ms. Trace:
I20260709 12:28:16.854941  7728 rpcz_store.cc:276] 0709 12:28:12.568290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.274138 (+705848us) service_pool.cc:224] Handling call
0709 12:28:16.369464 (+3095326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.387487 (+ 18023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.854236 (+466749us) spinlock_profiling.cc:243] Waited 3.07 s on lock 0x7b54002f1550. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.854410 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7051849984}
W20260709 12:28:16.854552  7731 rpcz_store.cc:269] 0709 12:28:15.591013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.596407 (+  5394us) service_pool.cc:224] Handling call
0709 12:28:16.853713 (+1257306us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:16.853876 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968270720}
I20260709 12:28:16.792656 14574 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:28:16.821529 14460 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:16.867211 14493 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:16.820744 14530 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.873972  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.875854  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.876035  7656 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:16.877041  7659 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
W20260709 12:28:16.878553  7627 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.802s (SENT)
I20260709 12:28:16.879272  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.879746  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:16.880707  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 139240us
I20260709 12:28:16.882944  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:16.889871  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.816s (SENT)
I20260709 12:28:16.892480  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:16.892493  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.875453 14451 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:16.896073  7777 connection.cc:256] RPC call timeout handler was delayed by -1.0355730770002083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2465s for a response
I20260709 12:28:16.896803  7810 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:16.898311  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:16.897928  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:16.900345  8411 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:28:16.900795  8411 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 13
I20260709 12:28:16.901080  8411 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:28:16.901337  8411 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.901661  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:16.898998  7656 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:16.903329  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.904368  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1205) took 2082 ms. Trace:
W20260709 12:28:16.905055  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.873315 14493 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:16.902863  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:16.904964  8412 rpcz_store.cc:276] 0709 12:28:14.822366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.860003 (+ 37637us) service_pool.cc:224] Handling call
0709 12:28:14.944681 (+ 84678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.547553 (+602872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.904103 (+1356550us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54004691d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:16.904299 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280998656}
I20260709 12:28:16.905229  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.907311  7804 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:16.908639  8405 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 12:28:16.911101  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.912187  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.913235  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.915045  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.915335  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.916646  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.917629  7812 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 14: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:16.917972  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:16.895048 14460 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:16.920127  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.923501  7810 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:16.923566  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2883) took 2176 ms. Trace:
I20260709 12:28:16.924057  7656 rpcz_store.cc:276] 0709 12:28:14.747542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.904378 (+2156836us) service_pool.cc:224] Handling call
0709 12:28:16.923028 (+ 18650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.923206 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.923271 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.923314 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:16.923348 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.923402 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.923509 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.921219  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.896306 14524 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:28:16.921270  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.864s (SENT)
W20260709 12:28:16.934401  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.899s (SENT)
I20260709 12:28:16.934875  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:16.936491  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.937456  7810 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:16.908979 14505 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:16.899865 14448 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:16.904553 14282 raft_consensus.cc:2764] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:28:16.909139 14515 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:16.944134  7812 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:16.944423  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:16.900892 14517 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:28:16.902475 14501 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:28:16.921468 14504 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:16.948612  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1220) took 1622 ms. Trace:
I20260709 12:28:16.967974  7803 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:16.967469  7656 rpcz_store.cc:276] 0709 12:28:15.326098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.925015 (+1598917us) service_pool.cc:224] Handling call
0709 12:28:16.947908 (+ 22893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.948154 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.948196 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.948261 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:16.948304 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.948383 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.948535 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.931512 14363 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:16.956682  7812 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:16.939343 14531 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:16.971943  7812 pending_rounds.cc:85] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 9
I20260709 12:28:16.972698  7812 pending_rounds.cc:107] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:16.973115  7812 raft_consensus.cc:2889] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.981611  7661 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:16.981895  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:16.984683  7656 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:28:16.985404  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1221) took 1659 ms. Trace:
I20260709 12:28:16.987581  7656 rpcz_store.cc:276] 0709 12:28:15.326520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.969471 (+1642951us) service_pool.cc:224] Handling call
0709 12:28:16.984336 (+ 14865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.985135 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.985345 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.993275  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2882) took 2246 ms. Trace:
I20260709 12:28:16.966961 14504 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.970187 14416 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:16.952303 14448 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:16.971491 14524 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:16.975829 14531 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.011680  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.002512  7656 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:16.971156 14574 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:16.952503 14415 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 12:28:17.004838  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102136us
I20260709 12:28:16.980206 14115 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:17.007532  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.995496 14498 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
I20260709 12:28:17.008684  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3207) took 2332 ms. Trace:
I20260709 12:28:16.994710 14517 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:17.012876  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:17.061020 14517 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:17.000980 14504 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:17.021379  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:17.023430  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:17.062526  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1219) took 1767 ms (client timeout 1683 ms). Trace:
W20260709 12:28:17.061426  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48695.9us
W20260709 12:28:17.062989  7653 rpcz_store.cc:269] 0709 12:28:15.295155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.427642 (+132487us) service_pool.cc:224] Handling call
0709 12:28:17.062330 (+1634688us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54000f4954. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:17.062455 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186317184}
I20260709 12:28:17.064241  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:17.065387 14574 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:17.066042 14574 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:17.066375  7653 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:17.027135  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.030560 14363 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:17.032657 14448 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.036903 14498 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.004057 14409 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.043201 14588 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:28:17.043555  7806 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:28:17.044037 14584 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:28:17.083313 14363 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:17.045113 14530 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.047413  7808 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:28:17.084064 14363 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:17.085582  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158140us
I20260709 12:28:17.022516  7734 rpcz_store.cc:276] 0709 12:28:14.676848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.973480 (+1296632us) service_pool.cc:224] Handling call
0709 12:28:16.017345 (+ 43865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.017586 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:16.025844 (+  8258us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.912362 (+886518us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b1800352600. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:28:16.984059 (+ 71697us) log.cc:844] Serialized 14 byte log entry
0709 12:28:17.008073 (+ 24014us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:17.008145 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:17.008199 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.008480 (+   281us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.008622 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23932,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":101,"raft.queue_time_us":17284,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":12996,"spinlock_wait_cycles":16601856,"thread_start_us":166766,"threads_started":2,"wal-append.queue_time_us":118340}
I20260709 12:28:17.058296  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:17.059762 14581 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:28:17.024227  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.096691  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.067158 14282 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:17.067401 14416 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:28:17.024166  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.066519 14574 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:17.068073 14505 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:17.025615  7731 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:16.986583 14515 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:28:17.073285  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.104664  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.105110  7728 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:17.072768  7653 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:17.061703 14451 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:17.015743 14531 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:17.076341  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.024163  7656 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:17.108017  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2889) took 1023 ms. Trace:
I20260709 12:28:17.108610  7656 rpcz_store.cc:276] 0709 12:28:16.084232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.014206 (+929974us) service_pool.cc:224] Handling call
0709 12:28:17.023703 (+  9497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.107631 (+ 83928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.107942 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.977782 14460 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.062103 14517 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.077800 14409 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.095089  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.111086  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:17.098062  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 163680us
I20260709 12:28:17.099076  7732 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:17.115891  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.097117  7729 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:17.082793 14448 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:28:17.076397 14527 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:28:17.119489  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:17.120117  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.940s (SENT)
I20260709 12:28:17.115589  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.120924  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:17.134099  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.954s (SENT)
I20260709 12:28:17.134758  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.111802  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.116959  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.115283  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.124470  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.097635  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 189606us
W20260709 12:28:17.115283  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 183243us
I20260709 12:28:17.143350  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:17.143437  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.144045  8411 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:17.128301 14498 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:17.148770  7656 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:28:17.150863  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76539.9us
I20260709 12:28:17.129047 14596 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:17.144428  8414 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:17.113497 14517 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:17.151983  7804 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:28:17.152024  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.155509  8406 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:17.160259  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:17.165274  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.165702  7732 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:17.173877  7656 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:17.174187  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:17.112604 14363 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:17.182736  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47839.6us
W20260709 12:28:17.190202  7700 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.869s (SENT)
I20260709 12:28:17.190871  7700 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.175010  8412 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.193324  8411 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.193284  7808 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:17.202505  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:17.202983  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2892) took 1117 ms. Trace:
I20260709 12:28:17.203544  7655 rpcz_store.cc:276] 0709 12:28:16.085651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.073757 (+988106us) service_pool.cc:224] Handling call
0709 12:28:17.092138 (+ 18381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.092377 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:17.102536 (+ 10159us) log.cc:844] Serialized 14 byte log entry
0709 12:28:17.202600 (+100064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:17.202681 (+    81us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:17.202734 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.202806 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.202926 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41122,"prepare.queue_time_us":10522,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"thread_start_us":58350,"threads_started":1,"wal-append.queue_time_us":53791}
W20260709 12:28:17.202958  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45765.2us
I20260709 12:28:17.151120 14258 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:28:17.166802 14581 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:28:17.173923 14594 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:17.194484 14580 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:17.173264 14449 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 12:28:17.236337  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 142152us
I20260709 12:28:17.211035 14498 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:17.217567 14115 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:17.248898  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1340) took 3252 ms. Trace:
I20260709 12:28:17.249578  7735 rpcz_store.cc:276] 0709 12:28:13.996405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.777655 (+1781250us) service_pool.cc:224] Handling call
0709 12:28:15.779089 (+  1434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.275447 (+496358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.248460 (+973013us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b540037e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:17.248804 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134294400}
I20260709 12:28:17.242982  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.209897 14591 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
W20260709 12:28:17.262249  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92550.6us
I20260709 12:28:17.241678 14498 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:17.261535  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.253291 14115 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:17.285859  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.224750 14574 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.286612  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:17.266759 14594 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.269655 14498 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:17.295143 14597 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:17.267361 14451 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.239449 14580 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.291778 14594 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.262650 14505 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:28:17.277588 14416 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:28:17.310922 14451 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.312428 14608 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:17.334648  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 144603us
I20260709 12:28:17.314447 14580 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:17.317577 14594 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.280536 14599 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:28:17.317988  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.314860  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.320600  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.321094 14516 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.322284 14115 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.323863  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.347476  8412 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:17.348143 14165 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.349478  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:17.335196 14608 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.350533 14608 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:17.350778 14594 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:17.351980 14594 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.352504 14594 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:17.335480 14409 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:17.357918  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.358336  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.359351  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.359519  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.361377  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.362146  7732 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.363842  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.364638  7733 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:17.364377 14493 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.365864  7728 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:17.325215 14448 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:17.367080 14448 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.367667 14448 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:17.369024  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.369868  7731 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.370712  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:17.302107 14600 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.005s
I20260709 12:28:17.325691 14596 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:17.371515  7731 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:17.325944 14602 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:17.371599  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.329552  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.329936  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.372490  7733 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.373050  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.373530  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.374419  7733 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.331322  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.290366 14603 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:17.375211  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.336513  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1227) took 1530 ms. Trace:
I20260709 12:28:17.375662  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.375885  7733 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16. Re-sending same reply.
I20260709 12:28:17.376505  7735 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.341470 14505 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:17.376091  7661 rpcz_store.cc:276] 0709 12:28:15.805775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.997357 (+1191582us) service_pool.cc:224] Handling call
0709 12:28:16.997818 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.998247 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.998284 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.017838 (+ 19554us) log.cc:844] Serialized 26 byte log entry
0709 12:28:17.143352 (+125514us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:17.143393 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.143471 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.336193 (+192722us) raft_consensus.cc:1685] finished
0709 12:28:17.336225 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.336423 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12989,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":2955,"thread_start_us":47664,"threads_started":1,"wal-append.queue_time_us":101751}
I20260709 12:28:17.377399 14286 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:17.378062 14286 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:17.378479  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.376952  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.380431  7733 raft_consensus.cc:2368] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:17.381994  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.365829  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.375720  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2723) took 2753 ms. Trace:
I20260709 12:28:16.993963  7659 rpcz_store.cc:276] 0709 12:28:14.746975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.878223 (+2131248us) service_pool.cc:224] Handling call
0709 12:28:16.900026 (+ 21803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.900200 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:16.925926 (+ 25726us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.992851 (+ 66925us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.992917 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:16.992974 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.993057 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.993208 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7964,"prepare.queue_time_us":26015,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3798,"raft.queue_time_us":9019,"thread_start_us":58561,"threads_started":1}
I20260709 12:28:17.339906  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:17.385778  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.384034  7807 rpcz_store.cc:276] 0709 12:28:14.622655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.637384 (+ 14729us) service_pool.cc:224] Handling call
0709 12:28:14.704783 (+ 67399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.062683 (+1357900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.701548 (+638865us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:17.375485 (+673937us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b54003732d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:17.375623 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3205862016}
I20260709 12:28:17.324791 14531 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.391897  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.393889  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.394433  7731 raft_consensus.cc:2368] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:17.395828  7730 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:17.399739  8412 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:28:17.401317  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.403399  8406 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:28:17.408668  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.414476  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 183703us
I20260709 12:28:17.418097  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.371739 14596 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.424588  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:17.426112  7810 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:17.427211  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.431346  8405 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:17.438578  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.397540 14459 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.438144  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.446280  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:17.479120  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.523960  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.523649  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.530659  7733 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:17.530120  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.534101  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:17.526247  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80077.9us
W20260709 12:28:17.524690  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.709s (SENT)
I20260709 12:28:17.379032 14286 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.419574 14435 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:17.539279  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.711s (SENT)
I20260709 12:28:17.540458  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.542788  7731 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:17.545221  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 212422us
W20260709 12:28:17.496078 14584 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:28:17.442631 14258 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:17.549374  7627 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.988s (SENT)
I20260709 12:28:17.553733  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.450178 14415 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:17.544420  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:17.550381  7731 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:17.558125  8405 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:17.562758  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.564132  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.566219  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 425049us
I20260709 12:28:17.568398  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:17.589769  7731 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:17.592526 14523 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.597434  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.599884  7734 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 12.
I20260709 12:28:17.449589 14612 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:28:17.524180 14459 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.602324  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.575591  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.602945  8405 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:17.588546 14504 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.000s
I20260709 12:28:17.621342  8411 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:28:17.614221  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.610356  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 76342.1us
I20260709 12:28:17.573757 14115 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:17.588758 14516 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:17.622812  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.622938  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.640465  7730 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:17.615130  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.597906 14603 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:17.641786  7731 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:17.628842  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.642971  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:17.628609 14498 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.629788  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.644270  7734 raft_consensus.cc:2368] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:17.635910 14531 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:17.636648 14258 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.596660 14451 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:17.647325 14603 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.652220 14603 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:17.652768 14459 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:17.653508 14459 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:17.656297 14606 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:17.656925 14606 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.658192 14606 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.592525 14610 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:28:17.618507 14464 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:17.643082  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.664451  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.667589  7731 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.662739  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.647150 14585 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:17.668814  7734 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:17.668429  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.670653 14606 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:17.672396 14610 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:17.673004 14610 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.676832 14498 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.680310  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:17.681687 14448 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.674266 14610 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:17.684764  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 144409us
I20260709 12:28:17.685978  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.686709  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.687630 14531 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:17.639425 14516 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:17.689402  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:17.649626 14580 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:17.706470  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.706773 14616 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:28:17.706416  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63397.9us
I20260709 12:28:17.622218 14504 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.712733  7732 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:17.700081  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.701793  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:17.702450  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:17.704559  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1382) took 1467 ms (client timeout 1554 ms). Trace:
I20260709 12:28:17.665606 14598 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:17.702919  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.695863  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.719368  7810 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:17.710074  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.712719  7653 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:17.647118 14493 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:17.715467  8408 rpcz_store.cc:269] 0709 12:28:16.237059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.239103 (+  2044us) service_pool.cc:224] Handling call
0709 12:28:17.704341 (+1465238us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:17.704480 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355635456}
W20260709 12:28:17.714882  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 164650us
I20260709 12:28:17.714764  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.720014  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.689160 14464 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:17.729238  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.733376  7735 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.729771  7729 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.706108 14516 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:17.730865  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.707261 14616 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.739008  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.740828  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:17.741811  8408 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.619145 14596 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.749390 14460 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:17.738528  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.745833  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:17.751444  7735 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.754532  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:17.754905 14606 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:17.755867 14606 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:17.756172  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:17.756404  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.757551 14606 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:28:17.763077  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.893s (SENT)
I20260709 12:28:17.763873  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.765024 14534 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:28:17.765056  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45110.6us
I20260709 12:28:17.718024 14598 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.770931  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:17.774380  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.771167  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.775467  7810 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.771313 14459 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:17.780215 14459 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:17.781530 14459 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.789067  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.790149  7809 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:17.790714  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.791808  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.770330  8408 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:28:17.792775  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:17.795049  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.795888  7811 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:17.798024 14527 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:17.798727 14527 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.799743  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2932) took 1448 ms. Trace:
I20260709 12:28:17.818549  7662 rpcz_store.cc:276] 0709 12:28:16.351413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.406571 (+ 55158us) service_pool.cc:224] Handling call
0709 12:28:17.799464 (+1392893us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400398fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:17.799659 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632316160}
W20260709 12:28:17.807807  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.821058  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.821767  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:17.823141  7735 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:17.745570 14616 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:17.827162  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:17.744501 14580 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:17.828356  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.832000  7807 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:17.743829 14623 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:17.833827  7803 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [19.14-20.15]
I20260709 12:28:17.770613 14612 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:17.799181 14527 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:17.847755  7654 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:17.843859  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:17.850827  7654 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:17.850947  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:17.851284  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.117s (SENT)
I20260709 12:28:17.855082  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.852578  8411 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:17.808809 14501 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:17.816067 14524 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:17.856396  7732 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:28:17.836299 14623 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.859333 14501 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:17.864955 14534 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:17.867427 14602 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:17.871901 14464 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.852485 14628 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:17.876351  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:17.858770  7735 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:17.848486 14531 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:17.855643 14615 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:17.819713  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46875.1us
I20260709 12:28:17.840274 14612 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:17.865983 14620 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:17.882094  7658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 2935) took 1526 ms (client timeout 1683 ms). Trace:
W20260709 12:28:17.896929  7658 rpcz_store.cc:269] 0709 12:28:16.355695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.427278 (+ 71583us) service_pool.cc:224] Handling call
0709 12:28:17.881555 (+1454277us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54003999d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:17.881974 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630117632}
I20260709 12:28:17.878039 14622 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:17.887575  8406 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:17.888721  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:17.902941  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:17.903404  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:17.879199 14623 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.880224  7659 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:17.890815  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:17.899479  7803 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:28:17.894411 14611 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:28:17.892650 14498 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:17.920296 14498 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:17.865340 14621 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:28:17.868172 14115 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:28:17.911492  7732 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:17.890875 14628 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.894832 14612 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:17.913683 14517 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.759s	user 0.005s	sys 0.010s
I20260709 12:28:17.865823 14598 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.915406 14449 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:17.918430  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2938) took 1533 ms. Trace:
I20260709 12:28:17.954558  7653 rpcz_store.cc:276] 0709 12:28:16.385331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.107137 (+721806us) service_pool.cc:224] Handling call
0709 12:28:17.107606 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.728525 (+620919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.918128 (+189603us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54003985d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:17.918356 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1391088000}
I20260709 12:28:17.890633 14385 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:17.891801  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:17.927676  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 171614us
I20260709 12:28:17.890763 14531 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:17.931484  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 194930us
I20260709 12:28:17.875437 14451 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:28:17.898739 14620 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.893537  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:17.944782  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.984179  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.142s (SENT)
I20260709 12:28:17.984767  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:17.945860  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.988000  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.901504 14534 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.957584 14594 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.008s
I20260709 12:28:17.992115 14534 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.992225 14594 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.929335 14628 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:17.994732 14534 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:17.959409 14115 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:17.989622 14620 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:17.956096 14603 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:17.997014  7809 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:28:17.876478 14524 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:17.979537 14599 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:17.999689 14607 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:18.000001 14599 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:18.000175 14607 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:18.000386 14524 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:18.000733 14165 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:17.976830  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:18.003533  7735 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:17.994796 14594 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:18.008329  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:17.981061 14608 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.004s
I20260709 12:28:18.009347  7735 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:17.979060  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 87355.2us
I20260709 12:28:17.980410  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:17.991243  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97797.3us
I20260709 12:28:17.954262 14449 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:17.978160 14531 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:17.998279 14603 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:17.929414 14622 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:18.022212  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:17.988914 14451 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:18.041126  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:18.042982  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:17.989768 14633 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:18.034037  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:18.053470  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:18.034799 14611 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:18.027643 14637 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:28:18.067183  8412 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:18.009681 14498 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:18.068818  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:18.051193  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:18.020445 14534 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.015503 14504 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.062296  7735 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:18.192278  7735 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:28:18.022665 14608 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.067276  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:18.145779 14435 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.008s
I20260709 12:28:18.026579 14523 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:18.147207  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:18.161832 14415 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:28:18.116742  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:18.146109  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 86236.2us
I20260709 12:28:18.025521 14385 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 12:28:18.060586 14633 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.059194 14493 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:18.117857 14580 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:18.025126 14624 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:18.179520  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:18.094015  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.048377 14165 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
W20260709 12:28:18.182471  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 131415us
W20260709 12:28:18.198114  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130949us
W20260709 12:28:18.269141  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 89713.9us
I20260709 12:28:18.270533  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:18.271868  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:18.019644 14115 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:18.187201 14594 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:18.188123 14615 raft_consensus.cc:2764] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:28:18.187008 14286 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:18.248853  7806 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:18.306389  7659 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:18.251925 14633 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.176396 14607 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.277729  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:18.308511  7658 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:18.279088  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.279771  7807 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:18.257153 14580 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.280778  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.283291  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.283820  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:18.327607  7805 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:18.327777 14115 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.329169 14612 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.240942 14622 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.333621 14611 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:18.307202 14630 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:18.182960 14463 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:28:18.212826 14524 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:18.289387  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.210467 14435 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:18.167438 14449 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.310516  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.280421  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.311862  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:18.312146 14615 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:18.289309 14517 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:18.318596 14610 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:18.318951 14644 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:28:18.289552 14594 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.319761 14282 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:18.321072  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3229) took 1263 ms. Trace:
I20260709 12:28:18.323613 14638 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.325968 14608 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.196283 14534 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.326743 14580 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:18.337806  7804 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:28:18.343848  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.348249  7805 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:18.336275 14630 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:18.255023 14504 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:28:18.289187 14651 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:18.442723  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:18.365139  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.370441  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.397066  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.498723  7662 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:18.397109  7728 rpcz_store.cc:276] 0709 12:28:17.058346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.106214 (+ 47868us) service_pool.cc:224] Handling call
0709 12:28:17.112126 (+  5912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.366579 (+254453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.320780 (+954201us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002490d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:18.321001 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583033216}
I20260709 12:28:18.409173  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1386) took 1520 ms. Trace:
I20260709 12:28:18.499984  7654 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:18.499840  8415 rpcz_store.cc:276] 0709 12:28:16.888811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.888942 (+   131us) service_pool.cc:224] Handling call
0709 12:28:16.893099 (+  4157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.893322 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:16.919496 (+ 26174us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.408101 (+1488605us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b180045f460. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:28:18.408440 (+   339us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.408655 (+   215us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:18.408767 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.408884 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.409081 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26467,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":8740,"raft.queue_time_us":3039,"spinlock_wait_cycles":14463744,"thread_start_us":30436,"threads_started":1,"wal-append.queue_time_us":59244,"wal-append.run_cpu_time_us":1778,"wal-append.run_wall_time_us":1045120}
I20260709 12:28:18.289433 14523 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:28:18.346021 14524 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:18.296085 14258 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:28:18.355366 14647 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:18.363870 14517 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:18.418637  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.591s (SENT) [suppressed 45 similar messages]
W20260709 12:28:18.519296  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100683us
W20260709 12:28:18.520274  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.618s (SENT)
I20260709 12:28:18.520787  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:18.443743  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:18.442579  7805 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 12:28:18.417173 14630 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:18.355109  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 148771us
W20260709 12:28:18.522156  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78548.1us
I20260709 12:28:18.501447  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.503082  7657 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:18.525192  7654 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:18.505381  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.506053  7655 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:18.506960  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 230040us
I20260709 12:28:18.302095 14620 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.415374 14607 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:18.509584  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 238411us
I20260709 12:28:18.395728 14282 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:18.395623 14594 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.351936 14616 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.526978  7654 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:28:18.539767  7654 pending_rounds.cc:85] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 9
I20260709 12:28:18.540143  7654 pending_rounds.cc:107] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:28:18.540560  7654 raft_consensus.cc:2889] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.538774  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:18.540339  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:18.515638 14644 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:28:18.511983 14524 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:18.514732 14624 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:18.582305  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.635s (SENT)
I20260709 12:28:18.527364 14517 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.587597 14624 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.537156 14608 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:18.593727  8415 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:28:18.597488 14616 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.414908 14638 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.604238  7812 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:18.606640 14258 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:18.607307 14258 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:18.608407 14258 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:18.616089 14628 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.000s
I20260709 12:28:18.615864 14282 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.616694 14628 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.549401 14449 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:18.624092  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73114.5us
I20260709 12:28:18.568583 14523 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:18.638119  7662 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:18.573119 14651 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:18.647887 14652 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:28:18.618111 14282 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.655344 14501 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.658151 14585 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:18.658758 14534 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:18.660728 14643 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.349s	user 0.000s	sys 0.000s
W20260709 12:28:18.661974 14646 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 12:28:18.553952 14647 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.619208 14628 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.666026  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:18.658927 14527 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.797s	user 0.007s	sys 0.002s
W20260709 12:28:18.675302  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 232633us
I20260709 12:28:18.638736 14523 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.672034 14385 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:18.676126 14585 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:18.706187 14460 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.003s
I20260709 12:28:18.706828 14460 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:18.645519 14612 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:18.625273 14656 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 12:28:18.708873 14647 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:18.687042  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:18.681780 14661 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:28:18.711030 14598 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:18.714682  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3239) took 1010 ms. Trace:
I20260709 12:28:18.682543 14501 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:18.640395 14658 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.688860 14523 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:18.690934 14623 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.755s	user 0.013s	sys 0.000s
I20260709 12:28:18.692804 14599 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.796036 14599 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.796800 14658 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.797839  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:18.798806  8409 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:28:18.703954 14516 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.958s	user 0.014s	sys 0.001s
I20260709 12:28:18.807431 14516 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.809878 14516 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:18.759259 14648 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.448s	user 0.000s	sys 0.001s
I20260709 12:28:18.810956 14585 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.765964 14448 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.715252  7733 rpcz_store.cc:276] 0709 12:28:17.704427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.704666 (+   239us) service_pool.cc:224] Handling call
0709 12:28:17.721227 (+ 16561us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:17.721699 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:17.843380 (+121681us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.064540 (+221160us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:18.142116 (+ 77576us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.142342 (+   226us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:18.142641 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.142854 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.142944 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.714392 (+571448us) raft_consensus.cc:1685] finished
0709 12:28:18.714421 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.714603 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86675,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":6483,"raft.queue_time_us":4064,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":72326,"thread_start_us":277258,"threads_started":2,"wal-append.queue_time_us":342219}
I20260709 12:28:18.756834 14633 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.856232 14585 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:18.750669 14527 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.683046 14517 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.763012  7729 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:18.764353  7732 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:18.767773  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1224) took 1028 ms. Trace:
W20260709 12:28:18.774447  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62580.1us
I20260709 12:28:18.741976 14574 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:18.779151  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:18.901762  7731 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.781051 14524 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:18.787691  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:18.752354 14634 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.610s	user 0.005s	sys 0.000s
W20260709 12:28:18.901382  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 362729us
I20260709 12:28:18.787930  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:18.906826  7653 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:18.746013 14596 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:18.694954 14341 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.800140  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:18.921468  8411 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:18.802578  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:18.803397  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:18.803944 14645 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.698839 14620 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:18.804507  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.801069  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.812837  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 124155us
I20260709 12:28:18.688293 14657 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:18.814811 14638 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:18.815845 14594 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.817899 14658 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:18.818637  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.789851 14580 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:18.820039 14531 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.000s
I20260709 12:28:18.950713  7659 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:18.819974 14523 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.821568  7656 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:28:18.915386 14647 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:18.825698 14630 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:18.814927  7557 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 305) took 57556 ms (client timeout 15000 ms). Trace:
I20260709 12:28:18.832105 14463 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:18.836054 14464 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.836656 14582 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:18.839181  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.125s (SENT)
I20260709 12:28:18.852344  7803 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:28:18.852064  7556 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 15, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 15 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:18.856829 14633 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:18.676343 14649 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.967514  7656 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:18.862334  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:18.966070  7803 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 11
I20260709 12:28:18.968454  7803 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:28:18.968708  7803 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.900226  8412 rpcz_store.cc:276] 0709 12:28:17.739115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.739262 (+   147us) service_pool.cc:224] Handling call
0709 12:28:17.739599 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.068066 (+328467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.767543 (+699477us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540046cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:18.767711 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752598784}
I20260709 12:28:18.902822  7661 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:18.905838  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 335948us
I20260709 12:28:18.795735 14623 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:18.910660  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:28:18.970366  8414 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:18.916594  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:18.920509 14585 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:18.926975  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.941771 14659 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:18.928144  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.929081  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.929736  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.981738  7653 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:28:18.923620  8408 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:18.932921  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.929996  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:18.935106  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:18.983191  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:18.983718  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53819.4us
I20260709 12:28:18.984090  7810 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:18.984201  7806 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:18.984965  8411 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:18.947399  7662 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:18.945259  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:18.948038  7808 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:18.934180  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.942797  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:18.952179  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:18.992450  7659 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:18.827028  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:18.993774  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 166848us
I20260709 12:28:18.920428 14516 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.957042  7655 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:18.997984  7655 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:18.909811 14282 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:28:18.920998 14574 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:19.004133  7660 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:18.966069  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:18.894183  8406 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:18.964814 14510 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.972595  7812 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:18.927351  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:18.917734  8415 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:28:18.966876  7556 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 6 to 10, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 10 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:18.933537 14637 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:28:18.986356  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:18.946745 14620 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:28:18.954087 14460 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:28:18.970003  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:19.059509  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89610.6us
I20260709 12:28:19.060335  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:19.063803  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:19.002800  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:18.956349 14624 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:28:19.034170  7809 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:18.933794 14670 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:28:18.942833 14645 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.076892 14449 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:19.069796  7807 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:19.078616 14645 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:19.033784  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 194627us
I20260709 12:28:19.079710 14638 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:19.080121  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.162s (SENT)
I20260709 12:28:19.040936  7812 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:18.972792 14582 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.103646  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.105284  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.040937  7662 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:19.046034  7806 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:19.107535  7806 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:19.108820  7806 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:28:19.109062  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.039736  7655 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.110643  7655 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:19.049806 14524 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.036650 14534 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:19.052857  7804 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:19.113824  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.117350  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.118232  7807 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:28:19.055576  7659 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:19.066916  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:18.963268 14647 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.028545 14501 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:19.001417 14633 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:19.074168  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:19.133104  7656 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:19.076561 14622 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:18.958315 14531 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.135360  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:19.080189 14638 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.041052  7661 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:19.147485  7729 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 17 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:19.083482  7812 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:28:19.032878  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 90186.5us
I20260709 12:28:19.034545  8405 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:28:19.005887 14651 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:19.088665 14623 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:18.980017 14649 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.090139 14517 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.090474 14115 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.038867 14510 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.069271 14672 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:19.169669  7655 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:28:18.959738  7557 rpcz_store.cc:269] 0709 12:27:21.258719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.270670 (+ 11951us) service_pool.cc:224] Handling call
0709 12:28:18.814692 (+57544022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:05.471531 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:05.484159 (+ 12628us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:05.484226 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:28:05.484248 (+    22us) tablet.cc:662] Decoding operations
0709 12:28:05.485770 (+  1522us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:05.485915 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:05.485951 (+    36us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:28:05.526664 (+ 40713us) tablet.cc:701] Row locks acquired
0709 12:28:05.541315 (+ 14651us) write_op.cc:260] PREPARE: finished
0709 12:28:05.572928 (+ 31613us) write_op.cc:270] Start()
0709 12:28:05.593951 (+ 21023us) write_op.cc:276] Timestamp: P: 1783600085562524 usec, L: 0
0709 12:28:05.598299 (+  4348us) op_driver.cc:348] REPLICATION: starting
0709 12:28:05.748578 (+150279us) log.cc:844] Serialized 5539 byte log entry
0709 12:28:06.764954 (+1016376us) op_driver.cc:464] REPLICATION: finished
0709 12:28:07.354383 (+589429us) write_op.cc:301] APPLY: starting
0709 12:28:07.398025 (+ 43642us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:07.590358 (+192333us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:07.590565 (+   207us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:08.361477 (+770912us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:08.361842 (+   365us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 12:28:08.361903 (+    61us) write_op.cc:312] APPLY: finished
0709 12:28:08.362344 (+   441us) log.cc:844] Serialized 129 byte log entry
0709 12:28:09.173590 (+811246us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:09.174274 (+   684us) write_op.cc:454] Released schema lock
0709 12:28:09.174488 (+   214us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:09.247102 (+ 72614us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:09.247186 (+    84us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3116050,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":512566,"apply.run_cpu_time_us":90652,"apply.run_wall_time_us":1887054,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62647,"num_ops":14,"prepare.queue_time_us":628503,"prepare.run_cpu_time_us":33313,"prepare.run_wall_time_us":544418,"raft.queue_time_us":544170,"raft.run_cpu_time_us":22213,"raft.run_wall_time_us":454973,"replication_time_us":1065758,"thread_start_us":2072666,"threads_started":5,"wal-append.queue_time_us":850615,"wal-append.run_cpu_time_us":24515,"wal-append.run_wall_time_us":2473287}]]}
I20260709 12:28:19.094154 14594 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.174391  7809 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:19.027611 14658 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:28:19.095626  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:18.967927 14464 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.096289 14630 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.096594 14523 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.097605 14580 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.097929 14527 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.098520 14615 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:19.202414 14615 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:19.203897 14615 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.206130 14615 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:19.099565 14574 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.083523  7556 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 14, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:19.213161  7556 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 12, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:19.029011 14516 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:19.069021 14668 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:28:19.108670  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:19.120329  7662 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:19.055429  7653 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:19.131556  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64735us
I20260709 12:28:19.133318  7732 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:19.218971 14516 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.132516  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:19.132525  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:19.225464  7728 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:19.227763  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:19.152166  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:19.231994  7656 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:19.158591  8412 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:19.151394  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:19.158847  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.156471  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:18.991699 14659 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.242327 14659 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.105752 14582 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:19.134671 14607 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:19.118582 14459 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:28:19.175014  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:19.244618  8406 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:19.152118 14651 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:19.170706  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196202us
I20260709 12:28:19.245271 14651 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:19.185714  8409 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:28:19.152230 14633 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:19.158938 14638 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:19.171516 14672 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:19.255677 14672 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.259955 14633 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.264282  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:19.265381  8414 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:19.269678  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:19.170470 14501 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:28:19.275422 14501 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:19.276007 14501 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.276506 14501 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:19.279704 14659 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:19.280840 14659 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.283965  7657 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:19.191938  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.270903 14534 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:28:19.306408  7702 connection.cc:256] RPC call timeout handler was delayed by -3.6605802599997332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
I20260709 12:28:19.223503  7556 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 13, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:28:19.306952  7702 connection.cc:256] RPC call timeout handler was delayed by -3.5624802600000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 12:28:19.310837  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83242.5us
I20260709 12:28:19.315035  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:19.316735  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:19.308046  7702 connection.cc:256] RPC call timeout handler was delayed by -3.5408802599999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 12:28:19.323395  7702 connection.cc:256] RPC call timeout handler was delayed by -3.4238802599998053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 12:28:19.323930  7702 connection.cc:256] RPC call timeout handler was delayed by -3.3788802599997325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
W20260709 12:28:19.324474  7702 connection.cc:256] RPC call timeout handler was delayed by -2.7082787879999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 12:28:19.324927  7702 connection.cc:256] RPC call timeout handler was delayed by -2.597578787999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 12:28:19.325363  7702 connection.cc:256] RPC call timeout handler was delayed by -2.5714787879996948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
W20260709 12:28:19.325871  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4886787880000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:28:19.326455  7702 connection.cc:256] RPC call timeout handler was delayed by -2.420278787999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 12:28:19.326822  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:19.327962  8409 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 13.
I20260709 12:28:19.330523  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.331333  7804 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:19.349751  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4148787879998963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:28:19.354913  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3743787879998308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
W20260709 12:28:19.356446  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3698787879998235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:28:19.357087  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3302787879997595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 12:28:19.357581  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3050787879997188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 12:28:19.357908  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2987787879997086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 12:28:19.358207  7702 connection.cc:256] RPC call timeout handler was delayed by -2.279878787999678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 12:28:19.358500  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2366787880000629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 12:28:19.358793  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2159787880000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 12:28:19.358711  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 136745us
W20260709 12:28:19.359414  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1844787879999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 12:28:19.351662  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:19.359789  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0206787879997137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2295s for a response
I20260709 12:28:19.355748  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:19.360116  7702 connection.cc:256] RPC call timeout handler was delayed by -1.73717878799971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.261s for a response
I20260709 12:28:19.360954  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:19.361249  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.362310  7812 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:19.175925 14115 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.178830 14620 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.143s
I20260709 12:28:19.367522  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:19.370805  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:19.399631  8406 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:19.373114  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:19.373368  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 264762us
I20260709 12:28:19.391616  7803 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:28:19.206161  7805 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:19.392037  7728 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:19.386024  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:19.406152  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:19.409286  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:19.410547  7812 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:19.423189  7804 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
W20260709 12:28:19.427585  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60204.1us
W20260709 12:28:19.429101  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56059.7us
I20260709 12:28:19.432390  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.433120  7810 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:19.434839  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:19.378422 14673 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.341s	user 0.004s	sys 0.000s
I20260709 12:28:19.431640  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.435724  7808 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:19.436421  7807 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:19.441766 14633 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:19.442505 14633 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.442987 14633 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:19.447650 14599 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.448539 14599 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.370280 14574 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:19.400015 14258 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:19.365366 14516 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 12:28:19.415727 14598 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:19.395190 14341 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.423633 14385 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:19.504961  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.513092 14670 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.513842 14670 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.517361 14649 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.533516 14670 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.493635  7810 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:19.416004 14611 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:19.482800 14595 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.546989 14611 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:19.547590 14611 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:19.547140 14595 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.549680 14595 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.470849  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:19.551779  7812 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:19.563864  7804 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:19.436669 14449 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:19.480087 14622 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:19.393891 14672 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.409173 14667 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.194s
I20260709 12:28:19.478588 14574 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:19.490056 14258 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:19.624223 14574 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:19.624603 14258 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:19.609308 14672 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:19.610954 14678 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.431s	user 0.001s	sys 0.000s
W20260709 12:28:19.612020 14677 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
W20260709 12:28:19.582020 14671 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:28:19.616923 14531 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:19.631754 14510 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:19.633011 14464 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:19.597059 14599 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.573307 14598 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:19.623667 14667 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:19.635929 14667 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:19.636082 14598 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.637084 14667 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.638388 14667 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:19.590263 14582 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:19.639631 14582 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.662441 14651 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.708567 14633 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:19.812635  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.816933  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.070s	sys 0.120s
W20260709 12:28:19.817667  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.070s	sys 0.120s
W20260709 12:28:19.832242  7806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.010s	sys 0.030s
W20260709 12:28:19.833083  7806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.010s	sys 0.030s
W20260709 12:28:19.839890  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 400044us
I20260709 12:28:19.841877  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:19.847923  7737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.003s	sys 0.002s
W20260709 12:28:19.848534  7737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.003s	sys 0.002s
I20260709 12:28:19.849463  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:19.821180  7556 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 12, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 12 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:19.852968  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.853585  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 501914us
W20260709 12:28:19.857869  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.865366  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:19.853751  7556 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:19.896890 14664 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.898337 14664 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.904371  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:19.727562 14638 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.919924 14690 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:19.920197 14638 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:19.920583  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 79123us
I20260709 12:28:19.921644 14638 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:19.921108 14690 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.908314 14611 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.712292 14633 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.925354 14611 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.925570 14582 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:19.926235 14464 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.927626 14580 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:19.927452 14464 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.929497 14580 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:19.931674 14282 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.932063 14692 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:19.932837 14282 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:19.934693 14623 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:19.940066 14282 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:19.940852 14693 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:19.941833 14693 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:19.846776  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:19.912515  7556 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 10, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 10 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:19.962416  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 101702us
I20260709 12:28:19.963014 14713 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:28:19.963717  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.966727  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.001s	sys 0.030s
I20260709 12:28:19.965469 14638 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:19.976040  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.002s	sys 0.030s
I20260709 12:28:19.960486 14623 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:19.974529  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:19.973430  7654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.007s	sys 0.004s
W20260709 12:28:19.978018  7660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.001s	sys 0.022s
W20260709 12:28:19.978674  7660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.002s	sys 0.022s
I20260709 12:28:19.980070  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:19.981206 14599 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.981712  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.982054 14534 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:19.983166  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118108us
I20260709 12:28:19.983529 14697 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.983438 14701 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:19.984851 14697 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:19.992039  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:19.993284  8411 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:19.993436 14697 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.996351  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:19.996752  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2752) took 1034 ms. Trace:
I20260709 12:28:19.997354  7809 rpcz_store.cc:276] 0709 12:28:18.962654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.084158 (+121504us) service_pool.cc:224] Handling call
0709 12:28:19.122771 (+ 38613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.174890 (+ 52119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:19.995978 (+821088us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:19.996509 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118120704}
W20260709 12:28:19.975613  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.022s
W20260709 12:28:19.999171  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.004s	sys 0.022s
W20260709 12:28:19.999444  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 153150us
I20260709 12:28:19.999004  7556 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 15, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 15 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:20.000775  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 591623us
I20260709 12:28:20.001986 14695 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:20.002344  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:20.004416 14165 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:20.004964 14623 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:20.006542  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83986.6us
I20260709 12:28:20.009783  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3281) took 1082 ms. Trace:
I20260709 12:28:20.010471  7805 rpcz_store.cc:276] 0709 12:28:18.926893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.927026 (+   133us) service_pool.cc:224] Handling call
0709 12:28:20.009135 (+1082109us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:20.009461 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341472512}
I20260709 12:28:20.002031  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:19.980311 14596 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.059s	user 0.010s	sys 0.007s
I20260709 12:28:20.002976 14695 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:20.015825  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:19.976701  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120804us
W20260709 12:28:20.019024  7654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.008s	sys 0.004s
I20260709 12:28:20.015450 14464 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:20.024416 14527 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.024186  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.029531 14527 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.024272 14531 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:28:20.031649 14599 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.034729 14599 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.035699  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:20.036672  8406 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:20.036882 14594 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:20.037392 14594 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:20.026229  7731 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:20.028296  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.043994 14622 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:20.044924  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:20.046260  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.046842 14622 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:20.048187 14464 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:20.050233 14695 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.051379 14699 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:20.055603 14638 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.047497 14622 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:20.059840 14622 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:20.063310 14659 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.060791 14690 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:20.069284 14664 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:19.993541 14693 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.066900 14651 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.072820  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:20.076650  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.077281  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:20.079350  7655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.009s	sys 0.028s
I20260709 12:28:20.080087 14620 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:20.081095 14620 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:20.078178 14690 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:20.082504 14165 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:20.084141 14670 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:20.084856 14670 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:20.080152  7655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.009s	sys 0.029s
I20260709 12:28:20.086305 14670 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.095059  8406 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:20.098234  8411 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:20.099485  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 97806.7us
I20260709 12:28:20.099869  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.103084  7556 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 10 to 13, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.106443 14574 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.109908 14574 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.105736  7556 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 13, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 13 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:20.038856 14594 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.111563 14594 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.103665  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.112936  7556 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:20.114914 14664 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:20.082782 14620 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:20.120693 14341 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.004s
I20260709 12:28:20.121712 14498 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:20.129133 14501 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.008s
I20260709 12:28:20.122376 14498 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.138119 14633 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:20.139431 14596 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:20.139917 14596 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:20.140693  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:20.140805  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47213.4us
I20260709 12:28:20.141512 14596 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.143877  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.138377  7728 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:20.139799 14697 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.144217 14670 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:20.141976  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:20.161713 14574 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.165230 14574 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.173105  7654 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:28:20.173637  7654 pending_rounds.cc:85] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 14
I20260709 12:28:20.173959  7654 pending_rounds.cc:107] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:28:20.174270  7654 raft_consensus.cc:2889] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.175410 14596 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:20.176182 14258 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.551s	user 0.015s	sys 0.011s
I20260709 12:28:20.176766 14258 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.179276 14574 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.180058 14115 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.179792  7556 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 14, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.181602  7556 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 13, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 13 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.184285 14527 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:20.163786  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 161690us
W20260709 12:28:20.165990 14667 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.014s
I20260709 12:28:20.130152 14501 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:20.187799  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45923.3us
I20260709 12:28:20.185745  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:20.218997 14598 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:20.223032  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.231760  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.238315  8411 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:20.208720 14258 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.233126 14598 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:20.230406 14611 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:20.252409 14460 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:20.251288  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:20.250907  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.253103 14460 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:20.257078 14460 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.257632 14598 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.259311 14633 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:20.260200 14711 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:20.260901 14711 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.252410  7556 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:20.260257  7809 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:20.264811  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2921) took 1267 ms. Trace:
I20260709 12:28:20.266291  7658 rpcz_store.cc:276] 0709 12:28:18.999898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.105038 (+105140us) service_pool.cc:224] Handling call
0709 12:28:19.105377 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:19.130849 (+ 25472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:19.130899 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:19.130961 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:19.130993 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.264202 (+1133209us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54000d07d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:20.264311 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.264596 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57947520}
I20260709 12:28:20.263873  7556 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 13, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 13 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:28:20.269701 14702 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.576s	user 0.001s	sys 0.013s
I20260709 12:28:20.276154  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2930) took 1154 ms. Trace:
I20260709 12:28:20.276772  7656 rpcz_store.cc:276] 0709 12:28:19.122097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.232971 (+110874us) service_pool.cc:224] Handling call
0709 12:28:19.233441 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:19.233753 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:19.394177 (+160424us) log.cc:844] Serialized 14 byte log entry
0709 12:28:20.274476 (+880299us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b18004b91c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:28:20.274895 (+   419us) log.cc:844] Serialized 14 byte log entry
0709 12:28:20.275336 (+   441us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:20.275436 (+   100us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:20.275571 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.275715 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.275916 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":873558,"prepare.queue_time_us":997010,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":26495,"raft.queue_time_us":101,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"spinlock_wait_cycles":4902144,"thread_start_us":159866,"threads_started":1}
I20260709 12:28:20.279297  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1383) took 1175 ms. Trace:
I20260709 12:28:20.273226  7556 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.281288 14696 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:20.280030  7737 rpcz_store.cc:276] 0709 12:28:19.103880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.104041 (+   161us) service_pool.cc:224] Handling call
0709 12:28:19.104395 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:19.104824 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:19.104858 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:19.105466 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 12:28:19.849528 (+744062us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:19.952839 (+103311us) log.cc:844] Serialized 14 byte log entry
0709 12:28:20.003781 (+ 50942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.003965 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.268528 (+264563us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:20.278782 (+ 10254us) raft_consensus.cc:1685] finished
0709 12:28:20.278921 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.279169 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111478,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":51597,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":58822,"spinlock_wait_cycles":11064704,"thread_start_us":840761,"threads_started":2,"wal-append.queue_time_us":584974}
I20260709 12:28:20.281680 14498 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.282030 14696 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:20.282234 14639 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:20.283408 14696 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.284169  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1239) took 1009 ms. Trace:
I20260709 12:28:20.284878  8414 rpcz_store.cc:276] 0709 12:28:19.274990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.287675 (+ 12685us) service_pool.cc:224] Handling call
0709 12:28:19.288309 (+   634us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:19.423486 (+135177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:19.423520 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:19.424177 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:28:20.122568 (+698391us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b18002744e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:20.251683 (+129115us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:20.256423 (+  4740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.283616 (+ 27193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.283679 (+    63us) raft_consensus.cc:1685] finished
0709 12:28:20.283692 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.283980 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141883,"raft.queue_time_us":143790,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":44850,"spinlock_wait_cycles":16633216,"thread_start_us":115113,"threads_started":1,"wal-append.queue_time_us":779240}
W20260709 12:28:20.287408  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.273238  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1391) took 2652 ms. Trace:
I20260709 12:28:20.237536 14697 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.289733  8405 rpcz_store.cc:276] 0709 12:28:17.621186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.621375 (+   189us) service_pool.cc:224] Handling call
0709 12:28:17.808043 (+186668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.149257 (+1341214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.271670 (+1122413us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:20.272807 (+  1137us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400183b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:20.273022 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3182019584}
I20260709 12:28:20.281590  7556 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.297263 14611 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.299631  7556 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 13, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 13 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.300433 14611 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.324162  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.325120  8405 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:20.327138  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:20.302747  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109520us
I20260709 12:28:20.328763  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.335263 14574 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:20.335563  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.335942 14574 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:20.336474 14501 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:20.306535  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.303464 14667 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:28:20.336557  7658 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:20.307611  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2755) took 1343 ms. Trace:
I20260709 12:28:20.339855  7803 rpcz_store.cc:276] 0709 12:28:18.964092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.088324 (+124232us) service_pool.cc:224] Handling call
0709 12:28:19.088705 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.401627 (+312922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.259022 (+857395us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400134cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:20.307367 (+ 48345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695692672}
I20260709 12:28:20.317027 14655 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:28:20.342413  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:20.342051  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.343186  8414 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:20.345273  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.346581  7658 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:20.347870  7654 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:28:20.303074  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 177883us
W20260709 12:28:20.329202 14687 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.847s	user 0.000s	sys 0.001s
I20260709 12:28:20.322815 14647 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:20.353648  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55832.9us
I20260709 12:28:20.360440  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.363271  8405 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:20.301750  7556 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 14, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.349716 14693 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:20.336468 14574 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.351044 14701 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:20.316785 14711 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:20.381002  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.382102  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.384171  7805 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [19.14-21.16]
I20260709 12:28:20.388582  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.389113  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2929) took 1268 ms. Trace:
I20260709 12:28:20.392402 14585 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:20.391574  7556 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 10 to 15, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 15 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.392903 14585 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:20.393850  7556 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.367489 14695 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:20.396538  8415 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:20.404003  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.405810  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:20.413681  8414 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:20.389670  7659 rpcz_store.cc:276] 0709 12:28:19.120781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.225866 (+105085us) service_pool.cc:224] Handling call
0709 12:28:19.226353 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.452683 (+226330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:19.452738 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:19.453434 (+   696us) log.cc:844] Serialized 26 byte log entry
0709 12:28:20.021688 (+568254us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:20.021795 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.388573 (+366778us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540017b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:20.388634 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.388694 (+    60us) raft_consensus.cc:1685] finished
0709 12:28:20.388709 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.389020 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45742,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":58737,"spinlock_wait_cycles":519329024,"thread_start_us":522962,"threads_started":1,"wal-append.queue_time_us":500963}
I20260709 12:28:20.378185 14701 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:20.399971  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.415838  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:20.423779  8405 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:20.425268  7803 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:20.426496 14607 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:20.385128 14460 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.431643  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:20.432843  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:20.437215  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.376806 14622 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.421931 14701 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:20.431460  7556 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 15, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 15 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:28:20.441722  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53274.1us
W20260709 12:28:20.444408 14703 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.752s	user 0.000s	sys 0.015s
I20260709 12:28:20.417054 14527 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:20.449170 14711 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.450052  7660 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:28:20.394243 14585 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.491482 14585 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:20.491919  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.493896  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:20.435923 14713 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:20.495769 14713 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:20.496532  7810 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:28:20.497377 14713 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.464088  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.498363  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:20.466857 14622 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.499616  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1251) took 1326 ms. Trace:
I20260709 12:28:20.465883 14582 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:20.469398 14460 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:20.470708 14658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:20.470903  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70496.4us
I20260709 12:28:20.510573  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.510774  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:20.474354  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.475251  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.475544  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:20.475760  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.476320  7812 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:20.481264  8406 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:20.474210  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.490082  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:20.464509 14697 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:20.503661  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.505131  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.505532  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2758) took 1530 ms. Trace:
I20260709 12:28:20.507759 14630 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:20.508072  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.511715  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1249) took 1425 ms. Trace:
W20260709 12:28:20.515409  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 138017us
I20260709 12:28:20.516220 14670 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:20.499545  7556 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:20.526202  7808 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:20.527452  7811 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:20.537667  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.538823  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:20.539328  7732 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:20.539640  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:20.540378  7728 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:20.540798 14115 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:20.537972 14523 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:20.503129  7653 rpcz_store.cc:276] 0709 12:28:19.102770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.221960 (+119190us) service_pool.cc:224] Handling call
0709 12:28:19.222481 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.222964 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:19.222996 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:19.223657 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:28:19.242112 (+ 18455us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:19.242242 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:19.242453 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.327994 (+1085541us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:20.394813 (+ 66819us) raft_consensus.cc:1685] finished
0709 12:28:20.394835 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.428991 (+ 34156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"spinlock_wait_cycles":17131776,"thread_start_us":17617,"threads_started":1,"wal-append.queue_time_us":212667}
I20260709 12:28:20.540921 14711 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.524524 14630 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:20.549841 14721 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:28:20.550312 14718 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:28:20.551925  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.552973  7729 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:20.554070 14657 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:28:20.544042 14115 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:20.556834 14115 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.560746  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:20.561668  7812 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:20.567384 14459 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:20.567919 14459 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:20.520951  7806 rpcz_store.cc:276] 0709 12:28:18.974711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.109624 (+134913us) service_pool.cc:224] Handling call
0709 12:28:19.109947 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.129330 (+ 19383us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:19.129366 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:19.129925 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:28:19.834886 (+704961us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:19.859001 (+ 24115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:19.859167 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.386682 (+527515us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:20.505094 (+118412us) raft_consensus.cc:1685] finished
0709 12:28:20.505119 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.505460 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21465,"raft.queue_time_us":22218,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17274,"spinlock_wait_cycles":12072704,"thread_start_us":662541,"threads_started":1,"wal-append.queue_time_us":554902}
I20260709 12:28:20.522572  7655 rpcz_store.cc:276] 0709 12:28:19.086160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.182971 (+ 96811us) service_pool.cc:224] Handling call
0709 12:28:19.308797 (+125826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:19.309187 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:20.097377 (+788190us) log.cc:844] Serialized 14 byte log entry
0709 12:28:20.221160 (+123783us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b4800080358. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:20.287375 (+ 66215us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:20.288102 (+   727us) log.cc:844] Serialized 26 byte log entry
0709 12:28:20.288262 (+   160us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:20.288542 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:28:20.288776 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.298561 (+  9785us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:20.511332 (+212771us) raft_consensus.cc:1685] finished
0709 12:28:20.511372 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:20.511635 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":968210,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":15522,"raft.run_cpu_time_us":5794,"raft.run_wall_time_us":933878,"spinlock_wait_cycles":31987968,"thread_start_us":891712,"threads_started":2,"wal-append.queue_time_us":248108}
I20260709 12:28:20.585023  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:20.589607 14711 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:20.592751 14574 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.548837 14630 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:20.595911 14534 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:20.596638 14534 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.599485 14574 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.568352 14459 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:20.568337 14598 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:20.599086 14534 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.616000  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:20.616844  8414 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.617695 14658 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:20.619913  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.620751  7731 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:20.602873  7556 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 15, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 15 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.626147  8414 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:20.632546  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:20.638618  7810 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:20.640601 14711 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:20.642442  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:20.622779 14598 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:20.650103  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.650300  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.651747  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:20.647341  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.645902 14711 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:20.657657 14690 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:20.649861 14534 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:20.659199 14534 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:20.658044 14711 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:20.667757  7653 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:28:20.663396  7556 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 13, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:20.682128  7659 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:20.686560  7662 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:20.688400  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:20.685947  7658 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:20.689512 14695 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.727560 14695 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.731956  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.732887  7660 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:20.734031  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:20.710209  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 163752us
I20260709 12:28:20.706364 14658 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:20.713711  7731 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:20.714607  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:20.751051  7730 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.660856 14534 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.722719 14598 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.717574  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:20.715809 14729 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:28:20.756242  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:20.706924 14696 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:20.730520 14695 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:20.709268  7556 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 16, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 16 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:20.721146 14630 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.766168  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:20.770248  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:20.770571  7731 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:20.772372  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:20.772866 14598 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.754370 14611 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.776505 14611 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.762475 14658 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:20.794221  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 76780.7us
I20260709 12:28:20.789165  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:20.792488  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:20.793799  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:20.786854  7735 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:20.816148  7657 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:28:20.817909  7657 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:28:20.818284  7657 pending_rounds.cc:85] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 8
I20260709 12:28:20.818527  7657 pending_rounds.cc:107] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:28:20.818763  7657 raft_consensus.cc:2889] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.813730  7735 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:20.824952  7737 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:20.830708  7734 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:20.831542  7658 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:20.779186 14611 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.781920 14658 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.850529 14711 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:20.851495 14684 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:20.852422 14684 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:20.855427  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.857324 14611 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:20.819849 14670 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:28:20.867566  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95295.8us
I20260709 12:28:20.865183  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:20.801488 14696 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:20.869693  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.875900  8405 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:20.868332  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:20.842443  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2495223820001229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:28:20.876536  7660 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:28:20.878321  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2423223820001112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 12:28:20.910784  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1892223820000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:28:20.912182  7627 connection.cc:256] RPC call timeout handler was delayed by -1.146922381999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 12:28:20.912813  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1100223819998973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 12:28:20.913365  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0596223820002706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 12:28:20.913827  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0389223820002371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
I20260709 12:28:20.914696  7734 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:20.820226 14701 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.873417 14596 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:20.921245  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.868060 14670 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:20.923733  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85190.2us
I20260709 12:28:20.863798 14684 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:20.921270  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157604us
I20260709 12:28:20.929316  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:20.906311 14341 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:20.915424 14693 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:20.972710  7734 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:20.982080  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:20.977365 14607 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:20.992496  7732 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:20.993301  7732 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:20.967499  7735 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:20.964462 14670 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:21.000700 14701 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.964859 14658 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:21.005389  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:21.006026 14696 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:20.975816 14341 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:21.007725 14341 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:20.994480  7732 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.012084  7732 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:21.013919 14574 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:21.014571 14574 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:21.015642 14574 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.024044 14711 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:21.024102 14630 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:20.993166 14607 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.025159 14711 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:21.025995 14711 consensus_queue.cc:1243] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:21.027967 14693 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:21.029143 14693 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.025487 14607 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:21.036326  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49466.5us
I20260709 12:28:21.038213  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.039887 14598 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:21.040555  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.039948  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:21.046398  7655 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:21.049525  7662 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:21.051396  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:21.052253  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45081.7us
I20260709 12:28:21.053138  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:21.053999 14658 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:21.054677 14658 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:21.056090 14684 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:21.056794 14684 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:21.040910 14598 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.097347  7659 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:21.101089  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:21.098596 14713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.012s	sys 0.014s
I20260709 12:28:21.099095 14598 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:21.099321 14701 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:21.120734  7732 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
W20260709 12:28:21.106588 14713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.013s	sys 0.014s
W20260709 12:28:21.114085  7733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.028s
W20260709 12:28:21.125344  7733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.009s	sys 0.028s
W20260709 12:28:21.151199  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 293186us
I20260709 12:28:21.121282  7732 pending_rounds.cc:85] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:21.163784  7732 pending_rounds.cc:107] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:28:21.106935 14647 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:21.166225  7655 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:21.122494 14701 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:21.162740 14459 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.007s
W20260709 12:28:21.168422  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.553s (SENT)
I20260709 12:28:21.181746  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:21.179649  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.164851  7732 raft_consensus.cc:2889] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:21.190745  7730 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:21.122084 14498 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:28:21.194728  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:21.199878  8406 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:21.181070 14611 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:21.195763  7812 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:21.176950 14701 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:21.206087  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:21.210822  7730 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:21.219075  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.217001  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:21.205156 14258 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:21.227372  7806 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:21.163727 14684 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.209597 14594 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:21.208325 14611 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:21.191992 14459 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.255865  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:21.256641  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:21.198886 14647 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.259183  7734 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:28:21.277742  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 113708us
I20260709 12:28:21.278927  7737 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:28:21.284232  7730 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:21.290993  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:21.299448 14749 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:21.299630 14611 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.301455 14611 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:21.283546 14607 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.310094 14607 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.311347  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:21.312011 14742 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:28:21.316493 14459 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:21.322610  7810 raft_consensus.cc:2368] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16.
W20260709 12:28:21.324612 14741 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:28:21.327243  7735 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:21.327608  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:21.329744 14745 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:28:21.331260 14740 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:28:21.332329 14647 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:21.332652 14737 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 12:28:21.293730  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 339609us
I20260709 12:28:21.346711  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.352771  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:21.352653  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:21.362485  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:21.364596  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.561s (SENT)
W20260709 12:28:21.366040  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.239602 14658 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.383363  8405 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:21.390532  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.366230 14639 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:21.321532 14598 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.367643 14647 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:21.412226  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.413223  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:21.416013  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:21.348855 14693 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:28:21.418169  8408 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:21.421365  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.433782  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:21.437903  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.417219 14647 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:21.416816 14756 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:21.443439  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.445379  7735 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:21.424098 14639 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:21.421240 14755 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:21.458200  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:21.440236 14747 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:21.447185 14647 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:21.471493  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 125677us
I20260709 12:28:21.487327 14757 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.488091 14757 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.489185 14758 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:21.490969 14527 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:21.491611 14639 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.490989 14637 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 12:28:21.493254 14739 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
W20260709 12:28:21.493512 14754 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:28:21.498771  7732 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:21.499603 14598 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:21.500363 14598 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:21.500748 14598 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:21.460512 14341 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:21.503551  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110471us
I20260709 12:28:21.504431  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:21.474860 14755 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:21.505739 14755 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:21.506193 14755 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:21.506901 14750 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:28:21.507401 14750 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:21.507804 14750 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:21.468439 14607 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:21.516325 14607 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.516834 14607 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:21.524981  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:21.525887  7658 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:21.526073  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:21.506067  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.527599 14753 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:21.528115 14753 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.528491 14753 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:21.527007  7731 raft_consensus.cc:2376] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19. Re-sending same reply.
I20260709 12:28:21.538822 14695 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:21.540092  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:21.540812 14670 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.000s
I20260709 12:28:21.504583 14341 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:21.456255 14756 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.555382 14611 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:21.564044  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.564239 14693 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:21.573266  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.574041 14693 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:21.568938  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.575582 14611 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:21.576215 14611 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:21.576670 14611 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:21.578277 14746 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
I20260709 12:28:21.579349 14711 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:21.580405 14341 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:21.581830 14341 consensus_queue.cc:1243] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:21.569979 14701 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:21.588721  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.589635  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.592377  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:21.593137  7731 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:21.575126  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.594897  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:21.596865  7809 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:21.609512  7732 raft_consensus.cc:2410] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:28:21.614133  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 138844us
I20260709 12:28:21.619665  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:21.501658 14598 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:21.637516  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.644110  7737 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:21.646366  7737 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:21.648972  8379 leader_election.cc:400] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16.
I20260709 12:28:21.611601 14711 raft_consensus.cc:2764] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:28:21.663535  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.667318  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.670105 14756 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.657910 14765 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.560281 14670 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.674394  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:21.635113 14747 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:28:21.675853 14762 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:21.648371 14749 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.681216 14534 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:21.676985  8406 pending_rounds.cc:85] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 13
I20260709 12:28:21.703100  8406 pending_rounds.cc:107] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:28:21.703413  8406 raft_consensus.cc:2889] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:21.661650 14501 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:21.684119 14701 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:21.683198 14765 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.719396 14749 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.724561  7728 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
I20260709 12:28:21.668541  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:21.647610 14574 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:21.701967  8408 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:21.671085 14594 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:21.705011  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:21.709918  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.697423  7737 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.711825  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:21.745239  8405 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:21.712386  7660 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:21.713024  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:21.715710 14760 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:21.679562  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104625us
I20260709 12:28:21.746809  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:21.746719  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:28:21.751703  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:21.753980  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 214441us
W20260709 12:28:21.751744  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83672.4us
I20260709 12:28:21.758596  7737 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:21.727619 14757 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.762692  8414 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:21.765347  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.728827 14696 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:21.768819  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.770442  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:21.770830  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1261) took 1141 ms. Trace:
I20260709 12:28:21.771278  7661 rpcz_store.cc:276] 0709 12:28:20.629418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.629560 (+   142us) service_pool.cc:224] Handling call
0709 12:28:20.678826 (+ 49266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:21.760897 (+1082071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.760946 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.761017 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:21.761066 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.770488 (+  9422us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540021b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:21.770532 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.770670 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468283264}
I20260709 12:28:21.776573  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.730175 14598 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:21.728211 14574 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.724512 14749 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.807752  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:21.810503 14624 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:21.811455 14624 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.812671 14747 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:21.815618 14624 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:21.816520  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:21.778620 14638 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 12:28:21.819761  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49477.1us
I20260709 12:28:21.821178 14638 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:21.821547 14594 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.822340 14594 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.793471 14760 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.825510 14594 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:21.825881 14760 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:21.798633  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:21.821419  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:21.829545  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.844149  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:21.845489  7733 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:21.844254  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:21.845968  7733 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:21.827076 14637 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:28:21.849146  7729 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:21.850497 14750 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.854854 14670 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.778121 14701 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:28:21.859295 14624 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.860006 14624 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.797418 14756 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:21.862452 14624 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:21.864035  7734 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:21.844275  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:21.864392  7732 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 12. Current term is 13. Ops: []
W20260709 12:28:21.866926  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.870401  7737 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:28:21.866959  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:21.794193 14757 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:21.778802 14695 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:21.873441  7731 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:21.846829  7733 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.896340  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:21.890777  7737 pending_rounds.cc:85] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 8
I20260709 12:28:21.897403  7737 pending_rounds.cc:107] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:28:21.895327  7728 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:21.797312 14574 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:21.899616  7737 raft_consensus.cc:2889] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:21.899410  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:21.900231  7812 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:21.897706  7731 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:21.894363 14639 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:21.901847  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:21.911509  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.899750  7658 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:21.911716  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:21.903553 14684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.025s
I20260709 12:28:21.917285  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:21.922326 14611 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.865065 14596 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:28:21.936201  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:21.923336 14594 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:21.924141  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 102836us
I20260709 12:28:21.926054 14750 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:21.926469  8405 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:21.898684 14695 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.902495 14697 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.883342 14647 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:21.890508 14531 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:28:21.937831  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:21.938656  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71776.9us
I20260709 12:28:21.934381  7729 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:21.941275  7733 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:21.909041 14749 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:21.908411 14757 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:21.929461 14684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.026s
I20260709 12:28:21.904387 14638 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:28:21.951047  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:21.958072  8414 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:28:21.935110 14756 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:21.935328 14690 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:21.941025 14637 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.943804 14760 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:21.940917 14695 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:21.947598 14701 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:21.985411  7812 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:21.919819 14658 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
W20260709 12:28:21.991626  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 498262us
W20260709 12:28:21.993574  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.947327 14594 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:22.011370  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.983191 14756 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.942770 14697 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:21.986027 14690 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:21.990662 14749 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:22.015394 14755 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.002s
I20260709 12:28:22.003412 14773 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:22.071723  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48681.7us
I20260709 12:28:22.061170  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.078171  7736 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:22.080595 14661 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 12:28:22.088191 14633 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:22.088986 14633 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:22.092113  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.097030 14769 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:22.098320 14769 consensus_queue.cc:1243] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:22.099771 14757 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.104588 14459 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:22.105264 14459 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:22.110728 14282 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:22.117230 14527 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:22.118023 14527 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:22.063936 14658 consensus_queue.cc:1243] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:22.075472 14755 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.122670 14594 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:22.132675  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.029819 14756 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:22.177198  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 101681us
I20260709 12:28:21.998963 14607 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.028076 14701 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:22.128297 14755 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:22.188777  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:22.200368  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.065114 14690 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:22.079674 14773 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:22.212297  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 263537us
W20260709 12:28:22.204610  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 249998us
I20260709 12:28:22.130553 14694 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:22.106534 14459 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.071995 14760 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:22.014271 14637 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.090476 14633 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.063246 14775 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.179s
I20260709 12:28:22.247509  7733 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:28:22.265285  7733 pending_rounds.cc:85] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:28:22.265614  7733 pending_rounds.cc:107] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 13.11
W20260709 12:28:22.238230  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.591s (SENT)
I20260709 12:28:22.251024  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:22.213312 14773 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:22.203315 14753 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.002s
I20260709 12:28:22.188104 14501 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:22.239365  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:22.203815 14661 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:28:22.262583  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.037s	sys 0.053s
W20260709 12:28:22.337328  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.037s	sys 0.053s
I20260709 12:28:22.201431 14670 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:28:22.284595  7731 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:22.287642  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:22.299563  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.231745 14777 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:22.207803 14611 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:22.333205  7810 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.013s	sys 0.032s
W20260709 12:28:22.354525  7810 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.013s	sys 0.032s
W20260709 12:28:22.332813  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 370108us
I20260709 12:28:22.334354  8408 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:22.338418  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.265892  7733 raft_consensus.cc:2889] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:22.360275  7733 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:22.360437  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:22.341199  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104247us
I20260709 12:28:22.341804  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:22.331331  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:22.343240  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:22.181412 14527 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.371414  7662 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:22.380148  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:22.389837  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:22.390959  7812 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:22.295845 14639 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:22.337190 14661 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:22.415192  7803 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:28:22.420447  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.358403 14777 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:22.333884 14758 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 12:28:22.366370 14778 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:22.402179 14742 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:28:22.385228 14459 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:28:22.466389  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 255895us
I20260709 12:28:22.346256 14753 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.257402 14694 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.445401 14633 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:22.334887 14749 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.478576 14527 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:22.511538  7728 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:22.512486  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 298082us
I20260709 12:28:22.484630 14624 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:22.528234  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.531260  7702 leader_election.cc:400] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18.
W20260709 12:28:22.584586  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46423.7us
I20260709 12:28:22.477320 14701 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.582343 14696 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:22.502962 14501 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:22.643337  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:22.582585 14779 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:28:22.545301 14607 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:22.541014 14781 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.528913 14638 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 12:28:22.585983 14776 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:28:22.503029 14760 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.556856 14792 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:22.665735 14637 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:22.541997 14624 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.663688 14798 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:22.680934  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 178469us
W20260709 12:28:22.688872  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157267us
I20260709 12:28:22.523200 14661 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:22.700140  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.687299 14791 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.717162  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:22.717916 14611 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:28:22.718410 14598 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.921s	user 0.016s	sys 0.000s
W20260709 12:28:22.698800  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 334885us
I20260709 12:28:22.740950  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.741945  7731 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:22.533926 14639 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.700106 14781 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.699415 14693 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.763974 14693 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:22.766301 14804 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:28:22.766659 14693 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:22.768729 14792 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:22.772461 14624 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:22.776451  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:22.777155  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:22.747408  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.667s (SENT)
I20260709 12:28:22.778285  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:22.779438  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44257.3us
I20260709 12:28:22.779961 14789 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:22.780179  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.708s (SENT)
W20260709 12:28:22.778443 14574 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.864s	user 0.010s	sys 0.001s
I20260709 12:28:22.780696  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:22.781961  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:22.787592  7734 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.897s	user 0.011s	sys 0.004s
I20260709 12:28:22.788187  7734 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:22.788290  7733 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:22.786443  7736 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
I20260709 12:28:22.789624  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:22.792081  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.793114  7736 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:22.794786  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.797192  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.798905  7736 raft_consensus.cc:2368] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
W20260709 12:28:22.800998 14801 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:28:22.801127 14783 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:22.733897 14791 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.801729 14753 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:22.626784 14777 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:22.803740  7627 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.831s (SENT)
I20260709 12:28:22.804098 14791 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:22.804304  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:22.806073  7627 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.899s (SENT)
I20260709 12:28:22.763123 14670 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:22.734697 14607 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 12:28:22.758490 14695 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.000s
I20260709 12:28:22.808051 14534 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:22.808199 14695 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.809060 14534 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:22.754081 14805 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:28:22.810285  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1127) took 1151 ms. Trace:
I20260709 12:28:22.812312  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.813223  7729 raft_consensus.cc:2368] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:22.770268 14792 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:22.734319 14765 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:22.807293 14670 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:22.754846 14694 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:22.810828 14695 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:22.806538  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:22.829946  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.831423  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:22.828200  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 455865us
I20260709 12:28:22.837965  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.736359 14757 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.846819  7733 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:22.848230  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.843715 14777 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:22.775034 14781 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:22.845002  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1411) took 1347 ms. Trace:
I20260709 12:28:22.857224  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.857749  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.858472  7736 raft_consensus.cc:2368] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:22.858582  7733 raft_consensus.cc:2368] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 15.
W20260709 12:28:22.857326  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:22.752660 14770 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.448s	user 0.002s	sys 0.000s
I20260709 12:28:22.726539 14623 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:28:22.857627  7735 rpcz_store.cc:276] 0709 12:28:21.498111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.548683 (+ 50572us) service_pool.cc:224] Handling call
0709 12:28:21.557573 (+  8890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:21.558026 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.558056 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:21.591752 (+ 33696us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.015854 (+424102us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:22.015912 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.015993 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.844658 (+828665us) raft_consensus.cc:1685] finished
0709 12:28:22.844679 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:22.844923 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4751,"thread_start_us":344206,"threads_started":1,"wal-append.queue_time_us":552386}
I20260709 12:28:22.863121  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:22.876924  7736 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:22.880431  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.796875 14598 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.807291 14785 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:22.893290  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.810712  7811 rpcz_store.cc:276] 0709 12:28:21.659303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.670591 (+ 11288us) service_pool.cc:224] Handling call
0709 12:28:21.682217 (+ 11626us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:22.003768 (+321551us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:22.086257 (+ 82489us) log.cc:844] Serialized 14 byte log entry
0709 12:28:22.534662 (+448405us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:22.568890 (+ 34228us) log.cc:844] Serialized 52 byte log entry
0709 12:28:22.569406 (+   516us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:22.569468 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.569590 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.761684 (+192094us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b180035e420. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:22.809949 (+ 48265us) raft_consensus.cc:1685] finished
0709 12:28:22.810039 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:22.810217 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19566,"prepare.queue_time_us":99678,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":56,"raft.queue_time_us":190004,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":168,"spinlock_wait_cycles":14694272,"thread_start_us":752618,"threads_started":3,"wal-append.queue_time_us":448425}
I20260709 12:28:22.900779  7729 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:22.931879  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.780601 14789 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:22.934993  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 103748us
I20260709 12:28:22.762109 14775 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
I20260709 12:28:22.815616 14574 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:22.975907  7729 raft_consensus.cc:1217] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 12:28:22.880468 14783 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:22.980232  7729 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:28:23.024726  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:22.946839  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:22.955605  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.862859 14757 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:23.036950  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 230900us
I20260709 12:28:22.934197 14765 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.934019 14769 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.038175  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.046157  7736 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.043968  7731 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:22.975318 14624 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:23.051605  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 111650us
I20260709 12:28:22.958629 14781 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.975155 14695 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:28:23.054569  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.013s	sys 0.025s
W20260709 12:28:23.057426  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.013s	sys 0.025s
I20260709 12:28:23.056046  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:23.058128  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 714976us
W20260709 12:28:22.814057 14778 consensus_queue.cc:1175] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 15 index: 10 } }
I20260709 12:28:23.060317  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:23.019860 14611 consensus_queue.cc:1243] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:23.067584  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:23.045647 14527 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.004s
I20260709 12:28:23.036456 14807 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:23.096901  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:23.097195  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.951s (SENT)
I20260709 12:28:22.959525 14598 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:23.102633  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.019s (SENT)
I20260709 12:28:23.103291  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:23.108839  7658 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:23.109315  7658 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:23.061301 14695 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:23.113528  7812 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:28:23.106012 14773 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.001s
I20260709 12:28:22.947904 14785 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.129567  7731 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:23.104638 14811 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.115751  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:23.114588 14756 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.001s
I20260709 12:28:23.059010 14765 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.134578  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:23.067061 14624 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.121562 14527 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:23.157379  8409 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:23.153502  8412 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:23.065781 14781 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.055063 14769 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.158049  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.111308 14783 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:23.159643  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.160405  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.151262  7658 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.051818 14670 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.167851  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.117959 14775 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:23.169930  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:23.184022  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81438.3us
I20260709 12:28:23.187100  7658 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:23.144887 14807 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:23.195096  7812 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:28:23.151429 14791 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.152630 14809 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:23.173561 14527 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:23.202574  7627 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.107s (SENT)
I20260709 12:28:23.183511 14796 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:23.138854 14773 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.183794 14701 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:23.212625  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 115761us
I20260709 12:28:23.214411  7731 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:23.214850  7731 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:23.158535 14756 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.154373 14811 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.207103 14527 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:23.219442 14815 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 12:28:23.238904  7729 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:23.214092 14796 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.204488 14611 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.242029  7731 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:23.235183 14793 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
W20260709 12:28:23.260106  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57606.2us
I20260709 12:28:23.260375  7733 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.207185 14791 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.266501  7658 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:23.241365 14789 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.220141 14775 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.250458 14796 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:23.300202  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.301136  7729 raft_consensus.cc:2368] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 13. Current term is 15.
W20260709 12:28:23.301384  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 106169us
I20260709 12:28:23.307428 14765 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.309330 14773 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.310441 14624 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:23.311678 14784 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:23.312402 14781 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:23.313513 14820 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:28:23.314085  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1126) took 1655 ms. Trace:
W20260709 12:28:23.314559 14459 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.002s
W20260709 12:28:23.316390 14812 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
W20260709 12:28:23.316955  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.317649 14757 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.317708 14783 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.319438 14783 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.322104 14783 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.259825 14574 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:23.325476 14819 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:28:23.325825 14774 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.308s	user 0.003s	sys 0.000s
W20260709 12:28:23.325973 14806 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.221s	user 0.001s	sys 0.001s
I20260709 12:28:23.327808 14756 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.328851 14785 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.339051  7731 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:23.341008  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1254) took 1657 ms. Trace:
I20260709 12:28:23.344383  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:23.344776 14633 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.002s
W20260709 12:28:23.345464  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.181s (SENT)
I20260709 12:28:23.345664  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.250291 14817 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:28:23.314618  7804 rpcz_store.cc:276] 0709 12:28:21.658804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.659564 (+   760us) service_pool.cc:224] Handling call
0709 12:28:21.665359 (+  5795us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:21.669303 (+  3944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.669335 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:21.670025 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 12:28:21.986291 (+316266us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:22.023344 (+ 37053us) log.cc:844] Serialized 14 byte log entry
0709 12:28:23.313444 (+1290100us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b18003d8520. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:28:23.313607 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.313692 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.313741 (+    49us) raft_consensus.cc:1685] finished
0709 12:28:23.313843 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.313991 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48990,"prepare.queue_time_us":36822,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":28360,"raft.queue_time_us":278729,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":78232,"spinlock_wait_cycles":43403648,"thread_start_us":587397,"threads_started":2,"wal-append.queue_time_us":508450}
I20260709 12:28:23.346957  7811 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:23.350080  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.351145  7736 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:23.353879  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.341467  8406 rpcz_store.cc:276] 0709 12:28:21.673288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.673432 (+   144us) service_pool.cc:224] Handling call
0709 12:28:21.676745 (+  3313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:21.729336 (+ 52591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.729393 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:21.743320 (+ 13927us) log.cc:844] Serialized 52 byte log entry
0709 12:28:22.266889 (+523569us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:22.266951 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.267057 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.330245 (+1063188us) raft_consensus.cc:1685] finished
0709 12:28:23.330271 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.330473 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2610,"raft.queue_time_us":5305,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6934,"thread_start_us":496136,"threads_started":1,"wal-append.queue_time_us":1449716}
I20260709 12:28:23.361629  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.364090  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.365955  7732 raft_consensus.cc:2368] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:23.376184  7729 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.272840 14611 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.387938  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:23.388468  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:23.387447 14690 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.149s	user 0.010s	sys 0.005s
I20260709 12:28:23.392122 14690 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.393985  8411 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:23.386299 14817 consensus_queue.cc:1243] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:23.397626 14647 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.399562 14769 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:23.402133  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97237.4us
W20260709 12:28:23.397753 14813 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.350s	user 0.003s	sys 0.000s
I20260709 12:28:23.362751 14598 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.404678  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.407804  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:23.409195  8405 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:23.412503  7729 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.275867 14807 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.354275 14789 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 12:28:23.412962 14777 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.003s
I20260709 12:28:23.423477  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.424196  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.422022  7731 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:23.428213  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:23.430902  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.242s (SENT)
I20260709 12:28:23.435418  7735 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.411480 14639 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.435690  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:23.414485 14690 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.438664  8412 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:28:23.440466 14775 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.444833  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:23.453485  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:23.454593  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.421377 14817 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:23.458042  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:23.458405  7729 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:23.464570  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.466073  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:23.469106  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.468758 14817 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.469142  7735 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:28:23.478824  7736 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:23.457261 14838 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:23.460040  7731 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:23.485970 14791 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.487360 14838 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:28:23.487879 14838 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:23.488813  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.490130  7732 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.490422 14611 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.492061  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.493532  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.488960 14838 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.497217 14838 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:23.498998 14498 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:23.495127  7735 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:23.504413  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82138.5us
I20260709 12:28:23.510865  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.512689  7732 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
W20260709 12:28:23.480098 14832 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:28:23.517872  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.515044 14527 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.445533 14639 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.596928  8406 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:23.595767 14796 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.598601 14639 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.600167 14796 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.543699 14817 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.601441 14115 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:23.577308  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.580058  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:23.580355  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.567814 14611 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:23.601518  7806 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:23.481521 14647 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
W20260709 12:28:23.606017  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8115134829999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31970000000000004s for a response
I20260709 12:28:23.608351  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.609205  7735 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:23.610101  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:23.610605  7735 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:23.612645  7732 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:23.612278 14611 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.624858  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.625105 14760 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.625727 14760 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.626228  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:23.626765 14611 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:23.627668  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.629842 14809 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:23.631845 14527 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.632014 14836 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.632867 14836 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.634351  7735 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
W20260709 12:28:23.643779  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4504886459999398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.165s for a response
W20260709 12:28:23.644632  7627 connection.cc:256] RPC call timeout handler was delayed by -1.380288646000281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 12:28:23.645066  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3541886460002388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:28:23.649444  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1768886459999521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260709 12:28:23.649839  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1138886459998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
W20260709 12:28:23.650100  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0724886460002381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 12:28:23.653666 14697 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:23.656153  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.656508  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.657686 14695 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.661019 14459 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:23.661504 14459 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:23.663507 14594 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.664170 14594 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.643860 14647 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:23.656153  7803 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:28:23.657727  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.621678 14785 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.673327  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.673645 14785 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.674146  8407 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:23.675173  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.675679 14661 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:23.677066  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.676420  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:23.685389 14845 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:28:23.669973 14498 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:23.688853  7728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3297) took 1792 ms (client timeout 1570 ms). Trace:
I20260709 12:28:23.689304 14498 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:23.689407  7728 rpcz_store.cc:269] 0709 12:28:21.896475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.915775 (+ 19300us) service_pool.cc:224] Handling call
0709 12:28:23.688562 (+1772787us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002aba54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:23.688733 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286062848}
I20260709 12:28:23.683023 14647 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.695124 14647 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.630936 14809 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.700834 14836 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.702343  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.704129  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.666608 14594 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.705140  7803 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:23.706276  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.706918  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.710402  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1063) took 1457 ms. Trace:
I20260709 12:28:23.710914  8405 rpcz_store.cc:276] 0709 12:28:22.252552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.252694 (+   142us) service_pool.cc:224] Handling call
0709 12:28:23.710167 (+1457473us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:23.710303 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254684544}
W20260709 12:28:23.714340  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.659102 14695 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:23.712796  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.650s (SENT) [suppressed 9 similar messages]
I20260709 12:28:23.682896 14701 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:23.719973 14701 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.721064  7809 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:23.676047 14785 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.728003  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:23.729846  8405 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:23.662515 14459 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.727882  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.732088  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.733996  7809 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:23.735496  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:23.723760 14825 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.480s	user 0.000s	sys 0.004s
I20260709 12:28:23.740350  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.716830 14781 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.741916 14781 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:23.738629 14684 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:28:23.744395 14781 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.744774 14760 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.727974 14282 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:23.746999 14849 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:28:23.751940  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.757046  8405 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:23.761202  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:23.763440  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:23.768663  7702 connection.cc:256] RPC call timeout handler was delayed by -2.7850659110004017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 12:28:23.769387  7702 connection.cc:256] RPC call timeout handler was delayed by -1.8346659110002292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3098s for a response
W20260709 12:28:23.769948  7702 connection.cc:256] RPC call timeout handler was delayed by -1.6294659110003522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3326s for a response
I20260709 12:28:23.750630 14459 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:23.778136  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:23.741422 14750 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:23.783349  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70448.5us
I20260709 12:28:23.788441 14498 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.771108 14670 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:23.790796  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.317s (SENT)
I20260709 12:28:23.793735 14670 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.795964  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1129) took 1438 ms. Trace:
I20260709 12:28:23.796547 14670 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:23.796447  7808 rpcz_store.cc:276] 0709 12:28:22.358059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.358208 (+   149us) service_pool.cc:224] Handling call
0709 12:28:22.391663 (+ 33455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:22.485911 (+ 94248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.485951 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:22.486021 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:22.486069 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.795728 (+1309659us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400295754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:23.795783 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.795904 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216009472}
I20260709 12:28:23.787202 14777 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:23.760155 14282 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.804113  7803 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:23.809057  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.804674  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:23.806017  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.765944  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "268874518d1d4544a8500be048628b67"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:23.814173  7809 raft_consensus.cc:606] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:23.818354  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:23.821310  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.821924  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.827607  7804 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:23.829247  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.830164  7804 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:23.834918  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.786145 14750 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.836385  7807 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:23.838254 14750 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.840883  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.842701  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.849040  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:23.843977 14282 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.806141 14777 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.862314  8407 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:28:23.863633  8407 pending_rounds.cc:85] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 11
I20260709 12:28:23.863901  8407 pending_rounds.cc:107] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:28:23.864176  8407 raft_consensus.cc:2889] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:23.864454  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:23.865281  7808 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:23.870080 14750 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:23.825932 14611 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:23.874506  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:23.876040  7804 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
W20260709 12:28:23.871512 14854 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:28:23.879083  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:23.880750  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1417) took 1186 ms. Trace:
I20260709 12:28:23.881788  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:23.881158  8415 rpcz_store.cc:276] 0709 12:28:22.694525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.695198 (+   673us) service_pool.cc:224] Handling call
0709 12:28:22.706797 (+ 11599us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:22.776586 (+ 69789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.776617 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:23.685816 (+909199us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b2c000e5b88. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:23.686513 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 12:28:23.771462 (+ 84949us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:23.771520 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.771625 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.880391 (+108766us) raft_consensus.cc:1685] finished
0709 12:28:23.880417 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.880685 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57701,"raft.queue_time_us":69643,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":24611200,"thread_start_us":80691,"threads_started":1,"wal-append.queue_time_us":111258}
I20260709 12:28:23.883503 14756 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:23.871716 14750 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:23.860538 14777 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:23.892347  7806 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:28:23.891567 14750 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:23.873430 14282 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:23.896154  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.896811 14282 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.897058 14750 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:23.898056 14785 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.898481 14750 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:23.893260 14498 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.899811 14498 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.902189 14498 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.903687  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.904903  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:23.905514  7653 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:23.905766  7662 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:23.893333 14684 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:23.907495  7657 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:28:23.907810  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.908182  7654 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:28:23.907922 14684 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.908766  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:23.909426  7657 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:23.910238  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:23.910444 14684 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:23.910898  7658 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:23.901942 14823 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:23.871435 14670 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:23.916013  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.909301  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.917279  7806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:23.917999  7806 raft_consensus.cc:606] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:23.917899  7660 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:23.920682  7806 raft_consensus.cc:614] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:23.914690  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.922523  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:23.924376  7655 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:23.892597 14658 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:23.928726  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 46161.6us
I20260709 12:28:23.898778 14785 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:23.894248 14814 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:23.922042  7661 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:23.934232  7656 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:23.894057 14755 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 12:28:23.942028  7734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3291) took 2104 ms (client timeout 2233 ms). Trace:
I20260709 12:28:23.929724  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:23.923161 14807 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:23.944715  7662 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:23.945565 14860 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:28:23.943042  7734 rpcz_store.cc:269] 0709 12:28:21.833692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.834561 (+   869us) service_pool.cc:224] Handling call
0709 12:28:23.937866 (+2103305us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54000721d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:23.938006 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43384064}
I20260709 12:28:23.961129  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:23.962209  7661 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:23.966674  7658 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:23.919485 14498 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:28:23.949898 14814 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:23.988925  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:23.993114  7658 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:28:23.996090  7657 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:23.996402  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74234.5us
I20260709 12:28:23.951810 14860 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:23.997742  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.001528  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:23.969080 14282 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:24.009974  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.975626 14807 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.016467  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:24.017122  8415 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:24.017933 14807 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:24.025543  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.026577  7657 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:24.032112  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:24.032475  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.034332  8413 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:24.041549  7658 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
W20260709 12:28:24.056070  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 234999us
W20260709 12:28:24.056859  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59214.5us
I20260709 12:28:24.060873  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.064123  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.062738  7658 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [14.9-18.11]
W20260709 12:28:24.067075  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.037415 14684 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:24.067924  8405 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:24.070426  7657 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:24.071975  7803 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:24.055757 14756 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:24.078244  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:24.079000  8414 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:28:24.080026  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55824.7us
I20260709 12:28:24.095886  8414 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7112151d65374f6690a117439cc2dc33"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:28:24.096905  8414 raft_consensus.cc:606] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:24.097309  8414 raft_consensus.cc:614] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:24.078368 14867 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.106300  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.108840  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:28:24.102404 14753 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:28:24.101226 14756 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:24.103993 14684 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.102195 14862 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:28:24.101527 14860 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:24.142068 14858 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:28:24.142336 14860 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.068265 14814 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.147719 14498 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.148418 14498 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:24.149893 14780 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.666s	user 0.000s	sys 0.003s
I20260709 12:28:24.151880 14868 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:24.152189  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1412) took 2469 ms. Trace:
I20260709 12:28:24.153033 14785 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.152563  7730 rpcz_store.cc:276] 0709 12:28:21.682311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.685571 (+  3260us) service_pool.cc:224] Handling call
0709 12:28:21.686055 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:21.686441 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.686471 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:21.687023 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:28:21.999757 (+312734us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:21.999827 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.999930 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:24.148831 (+2148901us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:24.151823 (+  2992us) raft_consensus.cc:1685] finished
0709 12:28:24.151844 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.152106 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1310,"raft.queue_time_us":3596,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":14019840,"thread_start_us":311533,"threads_started":1,"wal-append.queue_time_us":652207}
I20260709 12:28:24.155120  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.156239  7657 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:28:24.157863  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 93869us
I20260709 12:28:24.158794  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.099123 14867 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:24.136929 14861 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:28:24.164093  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.165043  7662 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:24.168946 14498 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.170079  7658 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:24.170931  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.172750  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.173377  7653 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.176036  7655 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:24.177305  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.178611  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.164994 14115 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.179864 14115 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.182598 14115 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.184088  7662 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:24.187610  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.188257 14750 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.189443 14750 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.192075  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.197731  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.200270  7808 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:24.201911  7661 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 13.
I20260709 12:28:24.158087 14684 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.313166  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:24.275866  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.313048  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:24.280854  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.330380  7731 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.280458  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.329231  7736 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:24.213268  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.317088  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.279811  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.276835  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.281219  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1134) took 1152 ms. Trace:
I20260709 12:28:24.276420  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.333573  7812 rpcz_store.cc:276] 0709 12:28:23.128632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.143810 (+ 15178us) service_pool.cc:224] Handling call
0709 12:28:24.280875 (+1137065us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b540042a254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:24.281124 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50429184}
I20260709 12:28:24.270800  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.160777 14753 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:24.234056  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.340893  7656 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:24.337549  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:24.342065  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.334673  7729 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:24.345638  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.347023  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.352562  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1255) took 1573 ms. Trace:
I20260709 12:28:24.356513  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:24.357889  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.358263  7804 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 14: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:24.347946  7812 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:24.360980  7735 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:28:24.274753 14869 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:24.192893 14785 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:24.353127  8411 rpcz_store.cc:276] 0709 12:28:22.778900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.782282 (+  3382us) service_pool.cc:224] Handling call
0709 12:28:22.802920 (+ 20638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:23.394406 (+591486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.352204 (+957798us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54004ba854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:24.352469 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358839552}
I20260709 12:28:24.365432  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.366526  7730 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:24.197078 14611 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:24.368135  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 209431us
I20260709 12:28:24.309736 14870 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.384088  7728 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:24.388463  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:24.389348  7806 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:24.395946 14863 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:28:24.397210  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.401798  7807 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:24.403893  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3305) took 1565 ms. Trace:
I20260709 12:28:24.405473  7811 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:28:24.413869 14684 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.416072  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.417949  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.419442 14661 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:24.419628 14582 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:24.419965 14661 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:24.420442 14661 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:24.424129 14867 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.427404 14785 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.430298 14785 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.423060  7806 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:24.404503  7733 rpcz_store.cc:276] 0709 12:28:22.838155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.880040 (+ 41885us) service_pool.cc:224] Handling call
0709 12:28:24.403605 (+1523565us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:24.403799 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850856192}
I20260709 12:28:24.449805  7807 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:24.428411 14750 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.371028  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.368251  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.465158  7731 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:24.161972 14756 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.310027 14868 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:24.383417  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.427865  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.362509 14866 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:28:24.432123  7805 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:24.432888 14701 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.362357 14869 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.437640 14611 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.491647  7736 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:24.446342  7810 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:24.373165  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:24.452397  7729 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:24.494222 14869 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.499660 14868 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:24.500926 14777 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.572s	user 0.014s	sys 0.000s
I20260709 12:28:24.501403 14777 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.502739 14756 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:24.496809  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.373329  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:24.473275  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109550us
I20260709 12:28:24.521420  7658 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:24.502977  7809 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:24.520901  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75778.9us
I20260709 12:28:24.549856  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:24.508255  7655 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:24.374495  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.472436  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.480319  7732 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:24.445338  7662 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:24.519675  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.547391  7729 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.555791  7657 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:24.453784  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:24.520234  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.557240  7654 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:24.549036  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:24.563195  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 90861.4us
I20260709 12:28:24.564373  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.457693  7653 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:24.453367 14862 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.476748 14867 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.507449 14697 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.596555  7729 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.452965 14870 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.602255  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.572232 14695 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:24.505445 14693 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:24.512048 14871 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.503424 14866 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.614706  7729 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:24.616237  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:24.580652 14785 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:24.596252 14847 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.013s	sys 0.006s
I20260709 12:28:24.591611  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:24.619841  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:24.674350  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82842.3us
I20260709 12:28:24.525543 14876 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:28:24.622918 14527 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:24.678077 14860 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.004s
I20260709 12:28:24.601609 14867 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.587466 14823 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.601069 14697 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.679718 14823 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.680634 14867 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.614725 14757 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.681128 14697 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:24.587916 14594 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 12:28:24.681639 14757 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.682518 14785 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.684495 14757 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.617941 14695 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.685599 14695 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.686595 14785 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:24.521209 14758 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 12:28:24.692152 14866 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.503542 14701 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.524361 14811 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:24.645382  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.695716 14701 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:24.673681 14847 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.013s	sys 0.007s
I20260709 12:28:24.620760 14459 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:24.697330 14459 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:24.698135 14836 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:24.698743 14459 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:24.699708  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83596.8us
I20260709 12:28:24.700678  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:24.690402 14869 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.599483 14624 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:24.703435  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.704171  7661 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:24.704826  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.705576  7737 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:28:24.581238 14756 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 12:28:24.620708 14874 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:24.696502  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.698767 14836 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.715242  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.614452 14877 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:24.708655 14607 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:24.716265  7655 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:24.718662 14836 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:24.724136  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.728160 14282 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:24.728833 14282 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.729246 14282 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.729933 14756 consensus_queue.cc:1243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:24.715672 14785 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.673835 14871 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.732074 14785 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.670837 14693 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.715040 14498 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:24.731427 14527 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.752666  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:24.754302  7729 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:28:24.754611  7733 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:28:24.754985  7733 pending_rounds.cc:85] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:28:24.755331  7733 pending_rounds.cc:107] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:28:24.755661  7733 raft_consensus.cc:2889] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:24.754760  7729 pending_rounds.cc:85] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:28:24.756343  7729 pending_rounds.cc:107] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:28:24.756645  7729 raft_consensus.cc:2889] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:24.758778  7658 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:24.741331  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.735849 14498 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:24.760319  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:24.761014  7661 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:24.707842 14624 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:24.761965  7653 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:24.764350  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 387752us
I20260709 12:28:24.765596  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.769006  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:24.769115  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:24.748466 14829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.120s	user 0.020s	sys 0.029s
I20260709 12:28:24.773238  7730 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:24.722992 14877 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.775766  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.779526  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.779428  7653 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:24.773712  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.736461 14785 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.784972  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:24.751066 14867 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:28:24.774137 14670 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:28:24.748773 14874 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.764973 14624 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:24.761970 14527 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.786978 14756 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:24.807144 14756 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:24.807735  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.808487  7655 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:24.778079 14829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.020s	sys 0.029s
I20260709 12:28:24.791453  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:28:24.803356  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.811896  7661 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:24.812583  7810 raft_consensus.cc:606] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:24.813055  7810 raft_consensus.cc:614] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:24.821926  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:24.809808 14756 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:24.782819 14877 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:24.787022 14867 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.834692  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.842388  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.853296  7808 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:24.857035  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.862713  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.863878  7807 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:24.793545 14753 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:24.918644  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.919319 14753 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:24.920696  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.921558  7661 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:24.826921 14582 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:24.918931  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.819579 14860 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.949528  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.959301  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:28:24.993479  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.994231  7654 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:24.996320  7661 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:24.998531  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:24.971840  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:28:25.022243  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 156520us
I20260709 12:28:24.957748 14611 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:25.065029  8378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.032s	sys 0.058s
W20260709 12:28:25.067121  8378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.032s	sys 0.058s
W20260709 12:28:25.072193  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.091187  7661 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:25.091856  7661 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:25.099540  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:24.960294 14860 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.103909  7654 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:25.104022  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:25.104139  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:25.110772  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.112807  8405 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 13: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:25.108990  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:25.117163  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:25.119797  8405 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:24.730540 14862 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.136348  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:25.066655 14611 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.133509  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.110682  7661 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.138497  7661 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:25.148974  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:25.111825 14757 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:25.147075  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:25.123749 14658 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.253286  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.025s	sys 0.032s
W20260709 12:28:25.253999  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.025s	sys 0.033s
W20260709 12:28:25.254485  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 801252us
I20260709 12:28:25.257205  7658 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:25.123086 14758 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:25.264151 14758 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.263823 14658 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.264693 14758 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:25.266781 14701 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:25.267354 14701 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:25.267753 14701 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:25.268152 14498 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.229548  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.279397 14498 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:25.264155 14871 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.133239 14696 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
W20260709 12:28:25.284852 14695 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.005s
I20260709 12:28:25.142426 14596 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.133s
I20260709 12:28:25.277822 14658 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:25.147378 14803 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.526s	user 0.016s	sys 0.020s
I20260709 12:28:25.285580 14695 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.286540 14803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.665s	user 0.016s	sys 0.021s
W20260709 12:28:25.295085  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.295984  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1430) took 1475 ms. Trace:
I20260709 12:28:25.297353  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:25.298363  7654 raft_consensus.cc:2376] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18. Re-sending same reply.
W20260709 12:28:25.248934  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 101982us
I20260709 12:28:25.301286  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:25.284916  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:25.303983  8405 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:25.133704 14881 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:25.315554  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:25.316776  7654 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18. Re-sending same reply.
I20260709 12:28:25.296547  8407 rpcz_store.cc:276] 0709 12:28:23.820690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.832805 (+ 12115us) service_pool.cc:224] Handling call
0709 12:28:23.834021 (+  1216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:24.583703 (+749682us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b54005e09a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99b06 00007fc001f2514a 00007fc001f24fb7 00007fc001f25470 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91
0709 12:28:24.583892 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.294708 (+710816us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:25.295905 (+  1197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61371648}
I20260709 12:28:25.152581 14791 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:25.323330  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.048s	sys 0.065s
I20260709 12:28:25.153247 14777 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:25.326551  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.049s	sys 0.065s
I20260709 12:28:25.326903  7626 leader_election.cc:400] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
W20260709 12:28:25.327456  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 778456us
I20260709 12:28:25.327710  7626 leader_election.cc:403] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:28:25.332448  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.333626  8406 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:25.335251  7661 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:28:25.336011  7661 pending_rounds.cc:85] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 10
I20260709 12:28:25.336485  7661 pending_rounds.cc:107] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:28:25.336916  7661 raft_consensus.cc:2889] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:25.312517  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:25.356348  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43952.2us
I20260709 12:28:25.359422  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:25.375630  7731 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:25.375905  7735 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:25.393340  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:25.394330  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:25.057133 14874 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.417285  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:25.417694  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77406.3us
I20260709 12:28:25.417402  7730 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:25.439462  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.360368  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:25.437628  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:25.416914  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95305.6us
I20260709 12:28:25.440768  7731 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:25.300246 14880 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.442008 14115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.005s	sys 0.013s
W20260709 12:28:25.437541  8413 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.015s	sys 0.042s
W20260709 12:28:25.446432  8413 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.016s	sys 0.043s
I20260709 12:28:25.447840  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.326972  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.453488  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:25.316397 14803 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:25.468034  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.134871 14867 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.487946  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:25.471300 14633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.001s	sys 0.017s
I20260709 12:28:25.155599 14611 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.442489 14115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.005s	sys 0.013s
W20260709 12:28:25.450719 14816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.000s	sys 0.018s
I20260709 12:28:25.437543 14790 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:25.443609 14785 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:25.440161 14791 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:25.490067 14808 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.006s	sys 0.023s
I20260709 12:28:25.449803 14814 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.496923 14633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.002s	sys 0.018s
I20260709 12:28:25.477025 14869 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.416782 14829 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:28:25.437731 14757 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:28:25.535660  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:25.522316 14816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.000s	sys 0.019s
W20260709 12:28:25.529620  7657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.041s	sys 0.044s
W20260709 12:28:25.577031  7657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.041s	sys 0.045s
I20260709 12:28:25.542011 14630 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:28:25.537164 14498 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:25.551895 14860 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:25.552980 14884 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:28:25.482045 14696 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:28:25.596784  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1294) took 1135 ms. Trace:
I20260709 12:28:25.523574 14870 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.597173  7656 rpcz_store.cc:276] 0709 12:28:24.461214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.461384 (+   170us) service_pool.cc:224] Handling call
0709 12:28:24.461799 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.462235 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.462267 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.462801 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:28:24.824650 (+361849us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:24.824738 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.824875 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.596534 (+771659us) raft_consensus.cc:1685] finished
0709 12:28:25.596560 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.596716 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9595,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":17935,"thread_start_us":361481,"threads_started":1,"wal-append.queue_time_us":843058}
W20260709 12:28:25.531181 14661 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.107s	user 0.014s	sys 0.002s
I20260709 12:28:25.599046 14661 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.601734 14661 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:25.499617 14807 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:28:25.558151 14836 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.839s	user 0.013s	sys 0.000s
I20260709 12:28:25.604334 14836 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.605609  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2963) took 1334 ms. Trace:
W20260709 12:28:25.539294 14808 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.006s	sys 0.023s
I20260709 12:28:25.458822 14658 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:28:25.608589  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:25.609485  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.556366 14611 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.324934 14881 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.584077 14879 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:28:25.587901 14791 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.592684 14498 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:25.504357 14670 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:28:25.601591  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:25.633610  8412 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:25.570708  7811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.018s	sys 0.012s
W20260709 12:28:25.634723  7811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.019s	sys 0.012s
I20260709 12:28:25.598933 14695 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.606079  7659 rpcz_store.cc:276] 0709 12:28:24.271469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.368707 (+ 97238us) service_pool.cc:224] Handling call
0709 12:28:25.116913 (+748206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.117129 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:25.117177 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:25.117246 (+    69us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:25.117283 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.117342 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.605419 (+488077us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400218ed0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:25.605548 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528433408}
I20260709 12:28:25.615195  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:25.615705  7732 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.031s	sys 0.085s
I20260709 12:28:25.641031  7654 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:28:25.641466  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.028s	sys 0.062s
I20260709 12:28:25.616283  7661 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:28:25.602509 14886 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:28:25.482671 14880 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.607124 14836 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:25.624500 14882 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.484s	user 0.001s	sys 0.000s
I20260709 12:28:25.626554 14881 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.627894 14498 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:28:25.641996  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.028s	sys 0.062s
W20260709 12:28:25.663455  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 635627us
W20260709 12:28:25.641021  7732 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.031s	sys 0.085s
I20260709 12:28:25.566007 14790 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:25.646891  8411 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:25.674196  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:25.631058 14693 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:25.571383 14814 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.638468 14697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.012s	sys 0.035s
I20260709 12:28:25.700875 14790 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:25.701957 14697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.012s	sys 0.035s
I20260709 12:28:25.702138 14814 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:25.664069 14866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.016s	sys 0.015s
W20260709 12:28:25.696055  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.696424  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:25.696839  7733 raft_consensus.cc:2376] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16. Re-sending same reply.
I20260709 12:28:25.572264 14701 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:25.632488 14459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.006s	sys 0.036s
W20260709 12:28:25.733843  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.699559 14836 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:25.732708 14866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.017s	sys 0.016s
W20260709 12:28:25.746754  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50477.9us
I20260709 12:28:25.749003  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:25.740110 14459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.006s	sys 0.037s
I20260709 12:28:25.750434  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.718969 14881 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.741847 14901 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:25.758996  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.758092  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2964) took 1240 ms. Trace:
I20260709 12:28:25.763514  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1418) took 3043 ms. Trace:
I20260709 12:28:25.764175  8408 rpcz_store.cc:276] 0709 12:28:22.720000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.720121 (+   121us) service_pool.cc:224] Handling call
0709 12:28:22.720574 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.853088 (+132514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.853143 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.366536 (+1513393us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b18003ceaa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:28:24.388437 (+ 21901us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b2c000e5b88. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:24.389114 (+   677us) log.cc:844] Serialized 26 byte log entry
0709 12:28:24.825258 (+436144us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:24.825317 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.762914 (+937597us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:25.762990 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.763068 (+    78us) raft_consensus.cc:1685] finished
0709 12:28:25.763089 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.763403 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52665,"raft.queue_time_us":57952,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12078,"spinlock_wait_cycles":161763584,"thread_start_us":429878,"threads_started":1,"wal-append.queue_time_us":865605}
I20260709 12:28:25.768018  8407 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:25.763540  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.772872 14611 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:25.773557 14611 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:25.755358 14836 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:25.774107 14611 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:25.759996  7660 rpcz_store.cc:276] 0709 12:28:24.517624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.517814 (+   190us) service_pool.cc:224] Handling call
0709 12:28:24.518353 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.518913 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.518961 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.519828 (+   867us) log.cc:844] Serialized 26 byte log entry
0709 12:28:25.007439 (+487611us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:25.007512 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.007632 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.601858 (+594226us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:25.749704 (+147846us) raft_consensus.cc:1685] finished
0709 12:28:25.749729 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.758012 (+  8283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1631,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":10241280,"thread_start_us":487024,"threads_started":1,"wal-append.queue_time_us":955117}
I20260709 12:28:25.787667  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:25.754105 14860 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:25.780517 14624 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.948s	user 0.011s	sys 0.003s
I20260709 12:28:25.795431 14901 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:25.762544 14881 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.812770 14527 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.815841  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:25.832571  7733 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:25.817075 14696 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:25.817636 14633 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:25.770074 14866 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:28:25.836038  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:25.829578 14901 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:25.830518 14807 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.853467  8409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.000s	sys 0.017s
W20260709 12:28:25.854199  8409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.000s	sys 0.018s
W20260709 12:28:25.839560 14877 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.003s
W20260709 12:28:25.860157  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 331786us
W20260709 12:28:25.860325  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 412582us
I20260709 12:28:25.848903 14880 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:25.831614  7627 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.005s	sys 0.018s
W20260709 12:28:25.865901  7627 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.006s	sys 0.018s
W20260709 12:28:25.866412  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 828962us
W20260709 12:28:25.868491  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.585s (SENT)
W20260709 12:28:25.876749  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89187.5us
I20260709 12:28:25.879276  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:25.863897  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:25.881309 14755 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:25.881815 14755 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:25.882249 14755 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:25.882668  8412 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:28:25.775254 14836 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.832948 14624 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.852412 14901 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:25.916349  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:25.925323  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.862053 14881 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:25.896171  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:25.928485  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60043.2us
W20260709 12:28:25.929540  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.587s (SENT)
W20260709 12:28:25.865118 14282 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.135s	user 0.016s	sys 0.000s
I20260709 12:28:25.876888  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:25.842092 14696 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.850564 14633 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.930359  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:25.936285  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.620s (SENT)
I20260709 12:28:25.936527  7733 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:28:25.933885  8406 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:25.891776 14866 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:28:25.928258 14879 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:28:25.926733 14880 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:25.866081 14877 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:25.860342 14807 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.942638  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1436) took 1167 ms. Trace:
I20260709 12:28:25.952184 14880 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:25.953958 14696 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.956097 14633 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:25.935878 14282 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.957170 14611 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:25.933367 14881 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:25.958559  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.628s (SENT)
I20260709 12:28:25.944872 14876 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:25.959373 14755 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:25.959765 14758 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.005s
I20260709 12:28:25.928967 14684 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 12:28:25.988389  7703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.013s	sys 0.012s
W20260709 12:28:25.988783  7703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.013s	sys 0.012s
W20260709 12:28:25.989264  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:25.969151  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 481297us
W20260709 12:28:25.969811  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 530258us
I20260709 12:28:25.973961  7736 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:25.945382 14658 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:25.965083 14866 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:25.938651  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:25.971413 14701 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:25.974926  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.010174  7812 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:26.026456  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1431) took 1273 ms. Trace:
W20260709 12:28:26.010651  7728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.010s	sys 0.034s
W20260709 12:28:26.028829  7728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.034s
I20260709 12:28:25.962643  8413 rpcz_store.cc:276] 0709 12:28:24.774832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.774980 (+   148us) service_pool.cc:224] Handling call
0709 12:28:24.775324 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:24.775558 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:25.450202 (+674644us) log.cc:844] Serialized 14 byte log entry
0709 12:28:25.919625 (+469423us) log.cc:844] Serialized 14 byte log entry
0709 12:28:25.942054 (+ 22429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:25.942145 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:25.942217 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.942318 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.942533 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21273,"prepare.queue_time_us":851459,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":88,"raft.queue_time_us":31126,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":20339,"spinlock_wait_cycles":6502656,"thread_start_us":1129177,"threads_started":2}
I20260709 12:28:26.011637  7733 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:25.977777 14876 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:25.975915 14881 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:26.066756  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.075464  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.076579  7733 raft_consensus.cc:2368] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:26.077594  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:26.089049  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.006650 14115 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:28:26.036077  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:26.022367 14862 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.025617 14753 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.093256  7733 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:26.099953  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.101058  7733 raft_consensus.cc:2368] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17.
I20260709 12:28:26.102164  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.032431 14860 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.031810 14873 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.009788 14758 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.077669 14881 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:26.027168 14693 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:28:26.073592 14876 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:26.183869  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:26.132409  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96447.3us
I20260709 12:28:26.036232 14633 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:26.210301  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 285060us
I20260709 12:28:26.075688 14661 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 12:28:26.089833 14814 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.092990 14701 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:26.155049 14897 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 12:28:26.226938  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.650s (SENT)
I20260709 12:28:26.230486  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:26.096909 14907 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:26.091502 14911 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:26.180577 14670 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.083678 14889 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.186s
I20260709 12:28:26.272004  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.273240  7731 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:26.107645 14753 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.105052 14862 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:26.249095 14896 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.313s	user 0.001s	sys 0.001s
W20260709 12:28:26.292902  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 416082us
I20260709 12:28:26.238283 14909 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:26.210582 14873 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.154970 14860 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.297710  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:26.306330  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.225235 14633 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.272215 14624 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.308606  7731 raft_consensus.cc:2368] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:26.105019 14880 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.241636 14814 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.244659 14701 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.255189 14658 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:26.324683  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 134087us
I20260709 12:28:26.260720 14670 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:26.331015  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104146us
I20260709 12:28:26.217222 14866 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.336648  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:26.338965  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.341537  7733 raft_consensus.cc:2368] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:26.359140  7731 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
W20260709 12:28:26.349972  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.728s (SENT)
I20260709 12:28:26.334360  8405 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:28:26.342012  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.310082 14807 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:26.314599 14115 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:26.368881  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:26.322165 14755 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:26.379948  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.757s (SENT)
I20260709 12:28:26.389147  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:26.334245 14877 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.408687  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.400889  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3339) took 1006 ms. Trace:
I20260709 12:28:26.322712 14684 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 12:28:26.417479  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 189617us
W20260709 12:28:26.422627  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 180912us
I20260709 12:28:26.314666 14911 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:28:26.397411  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:26.433585  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:26.387140 14896 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:28:26.366612 14929 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:26.373380 14282 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:26.448385  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3315) took 2173 ms (client timeout 2280 ms). Trace:
I20260709 12:28:26.449146  7807 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:28:26.448902  7737 rpcz_store.cc:269] 0709 12:28:24.275249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.275471 (+   222us) service_pool.cc:224] Handling call
0709 12:28:26.448144 (+2172673us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400212854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:26.448312 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863263488}
I20260709 12:28:26.437862  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.414810  7728 rpcz_store.cc:276] 0709 12:28:25.394009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.445277 (+ 51268us) service_pool.cc:224] Handling call
0709 12:28:25.489766 (+ 44489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.490062 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:26.029615 (+539553us) log.cc:844] Serialized 14 byte log entry
0709 12:28:26.400257 (+370642us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:26.400354 (+    97us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:26.400434 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.400554 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:26.400769 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12883,"prepare.queue_time_us":335357,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":205,"spinlock_wait_cycles":53376,"thread_start_us":896181,"threads_started":2}
I20260709 12:28:26.436968  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.451153  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:26.451076  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.451960  7731 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:26.441761  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.442077  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.442212  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.453552  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:26.454138  7809 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:26.441447  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.392630 14624 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:26.447170  7811 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:26.436883  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.457170  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.456423  7807 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:26.458222  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.463932 14594 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:26.464097  7737 raft_consensus.cc:2368] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:26.423837 14808 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:26.434854 14755 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.418624 14934 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:26.466172  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.449723 14661 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:26.475131  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.483925  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.468092  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.477873  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:26.538085  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:26.543073  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65355.8us
I20260709 12:28:26.543241  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.548894  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.550469  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.546528  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:26.484129 14924 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 12:28:26.551628  7737 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:26.552358  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:26.556524  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:26.557161 14926 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:28:26.557315 14498 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:26.557881 14902 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:26.495855 14934 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:26.558905 14919 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:28:26.557637  7661 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:26.484768 14658 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.559816 14684 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:26.560539 14931 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:26.589303  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.560781  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:26.464565 14594 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:26.561470 14933 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:28:26.614867 14860 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.615361 14930 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:26.616216 14930 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:28:26.616906 14941 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:26.617574 14941 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.596405 14902 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:26.621729 14902 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.579841 14939 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.614049 14701 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:26.623927 14939 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.563407  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.568744  7805 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:26.568918  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:26.571133  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 191250us
I20260709 12:28:26.629221  7737 raft_consensus.cc:2410] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:28:26.462342 14911 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.574393  7810 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:26.459124 14624 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:26.473068 14115 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.580598  7661 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:26.455137 14929 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.652977 14929 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.653940  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.448599 14889 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
W20260709 12:28:26.562281 14917 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.322s	user 0.002s	sys 0.000s
W20260709 12:28:26.634419 14904 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
I20260709 12:28:26.532461 14935 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:26.597340  7728 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:26.665601  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.665778  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:26.665062  7805 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:26.596176  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.664490  7626 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:26.601581 14918 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
W20260709 12:28:26.603863 14928 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:28:26.606056 14594 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.606320  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.608516  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1272) took 1182 ms. Trace:
I20260709 12:28:26.608938 14753 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.609973 14873 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.498448 14808 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.580199 14934 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:26.537274 14755 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:26.586632 14913 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:28:26.574523 14906 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:26.575098 14498 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:26.614061 14862 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:26.582700 14916 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
W20260709 12:28:26.629144  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.764s (SENT)
I20260709 12:28:26.569684  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:26.637468 14911 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.638054  7661 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.641255  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:26.636570 14624 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:26.655442  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.516515 14661 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:26.656584  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 133446us
I20260709 12:28:26.663245  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:26.664995  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64516.6us
I20260709 12:28:26.667518  7810 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:28:26.666719  7655 raft_consensus.cc:2376] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17. Re-sending same reply.
I20260709 12:28:26.691730  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:26.746095 14917 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:28:26.732020  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67694.5us
I20260709 12:28:26.759943  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:26.655684 14943 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.735682  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:26.626170 14684 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.663652 14633 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:26.740883  7809 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:26.637208 14115 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.773576  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3343) took 1364 ms. Trace:
I20260709 12:28:26.747383 14947 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:28:26.748365  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119281us
W20260709 12:28:26.748683  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 179133us
I20260709 12:28:26.749598  7653 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:26.749716  7661 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:26.656890  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.744678  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.753312  7654 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:26.669200 14929 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:28:26.694314  8406 rpcz_store.cc:276] 0709 12:28:25.426304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.443429 (+ 17125us) service_pool.cc:224] Handling call
0709 12:28:25.497806 (+ 54377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.937536 (+439730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.608198 (+670662us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:26.608447 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002185984}
I20260709 12:28:26.754765  7656 raft_consensus.cc:2376] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16. Re-sending same reply.
I20260709 12:28:26.655176 14876 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.771873  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:26.744390  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.774207  7735 rpcz_store.cc:276] 0709 12:28:25.408889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.604658 (+195769us) service_pool.cc:224] Handling call
0709 12:28:25.605050 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:25.605743 (+   693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:25.605778 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:25.606352 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:28:25.940146 (+333794us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:25.940242 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.940331 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:26.773216 (+832885us) raft_consensus.cc:1685] finished
0709 12:28:26.773249 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:26.773487 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":326165,"threads_started":1,"wal-append.queue_time_us":615713}
I20260709 12:28:26.768853  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:26.756604 14661 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:26.801672 14939 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.767266 14941 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.803902 14808 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.805765 14670 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:26.758232 14881 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.001s
I20260709 12:28:26.753978 14873 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:26.777281  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.753736 14814 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:26.811131  7659 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:26.808259  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1434) took 2040 ms. Trace:
W20260709 12:28:26.812160  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.766940 14935 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:26.814394  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:26.815721 14938 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 12:28:26.027176  7732 rpcz_store.cc:276] 0709 12:28:24.752752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.752885 (+   133us) service_pool.cc:224] Handling call
0709 12:28:24.771912 (+ 19027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:24.772145 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:25.665164 (+893019us) log.cc:844] Serialized 14 byte log entry
0709 12:28:26.025779 (+360615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:26.025858 (+    79us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:26.025911 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.026137 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:26.026374 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11792,"prepare.queue_time_us":1052686,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":13021,"thread_start_us":1212150,"threads_started":2}
W20260709 12:28:26.818241 14901 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.886s	user 0.010s	sys 0.006s
I20260709 12:28:26.818643  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.823858  7732 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:26.825145  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.778276 14885 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:28:26.826562  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.799404 14914 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:26.799751  7731 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:26.805631 14921 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.507s	user 0.000s	sys 0.003s
I20260709 12:28:26.809906  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.811549 14873 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:26.796756 14936 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:28:26.796641 14950 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:28:26.827206  7736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1429) took 2075 ms (client timeout 1939 ms). Trace:
I20260709 12:28:26.807580 14881 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.827407  7734 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:28:26.787467 14498 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.856624  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:26.827728 14935 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.828545  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.799404 14876 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.860548 14498 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:26.862190 14911 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.853006 14914 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:26.864318 14914 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.771620 14943 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.766610 14836 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.832288  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.833700 14939 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:26.787410 14944 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:28:26.836447  7735 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:28:26.778726  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.787s (SENT)
I20260709 12:28:26.840848  7809 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:26.848440  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.876827  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:26.875419  7805 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:26.811859  8409 rpcz_store.cc:276] 0709 12:28:24.768030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.768158 (+   128us) service_pool.cc:224] Handling call
0709 12:28:24.784523 (+ 16365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.804012 (+ 19489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.804049 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.823744 (+ 19695us) log.cc:844] Serialized 26 byte log entry
0709 12:28:25.855031 (+1031287us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:25.855095 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.855195 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:26.808010 (+952815us) raft_consensus.cc:1685] finished
0709 12:28:26.808037 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:26.808190 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":196,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9815,"thread_start_us":934389,"threads_started":1,"wal-append.queue_time_us":1173191}
I20260709 12:28:26.818707 14901 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:26.852098  7736 rpcz_store.cc:269] 0709 12:28:24.752120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.752394 (+   274us) service_pool.cc:224] Handling call
0709 12:28:26.826931 (+2074537us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540037cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:26.827124 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2806059392}
I20260709 12:28:26.858856  7656 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 13.
I20260709 12:28:26.773213 14594 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:28:26.849115 14873 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:26.869200  7728 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:26.874917  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.881976  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.872900 14939 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:26.889405  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.890027 14939 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.891957  7807 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:28:26.892421  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.892730  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.893914  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.894641  7658 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:26.895354 14929 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:26.895955 14929 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:26.896179 14946 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.896656  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.897428  7803 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:26.897042 14929 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.898399 14929 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:26.898797 14881 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.883286  7661 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:26.868001 14935 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:26.890981  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.901651  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.912289  7661 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:26.913254  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 134589us
I20260709 12:28:26.913833  7808 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:28:26.900936  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:26.904006  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.904660  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.920204  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.923507  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.926223  7736 raft_consensus.cc:2410] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:26.924140  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:26.916750  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.897s (SENT)
I20260709 12:28:26.906448  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.928411  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:26.904983  7656 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:26.904660  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.929723  7728 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:26.930336  7658 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:26.899863 14785 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
W20260709 12:28:26.932267  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.933764  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.934062  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.934881  7805 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:26.936174  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.936976  7803 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:26.937825  8414 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:28:26.938441  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.939347  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.940289  7661 raft_consensus.cc:2410] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:28:26.941598  7661 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:28:26.941308 14934 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.904428 14911 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:26.944082 14891 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.320s	user 0.009s	sys 0.001s
I20260709 12:28:26.944711  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.948019 14901 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.928957 14939 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:26.949309 14670 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.950037 14670 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.950851 14611 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:26.952630 14670 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.952730 14661 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.954586 14866 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.954949  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:26.966857  7732 raft_consensus.cc:2376] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15. Re-sending same reply.
I20260709 12:28:26.965785  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:26.957561 14927 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.327s	user 0.003s	sys 0.000s
I20260709 12:28:26.957724 14881 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:26.959601  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:26.961788  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:26.962137  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:26.876422 14836 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:26.962889  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.960178  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:26.955932  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:26.971431  7656 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:26.972396 14901 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:26.972944  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:26.973199 14901 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.973635 14901 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:26.973768 14836 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.896976 14946 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.977460 14881 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:26.894667 14943 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:26.974069  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:26.955135 14876 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.899461 14873 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.968252 14866 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:26.981791  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:27.024914  7806 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:27.382416  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:26.968916 14756 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.415s
I20260709 12:28:27.383539  7736 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:27.384150 14756 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:27.384665 14756 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:27.385106  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.385748 14756 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:27.075769  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:27.188434 14902 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:27.260150  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3345) took 1817 ms. Trace:
I20260709 12:28:27.388665  7728 rpcz_store.cc:276] 0709 12:28:25.442490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.945741 (+1503251us) service_pool.cc:224] Handling call
0709 12:28:26.946087 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:26.946297 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:26.946654 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.259708 (+313054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.259778 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:27.259832 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.259926 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.260079 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83423,"prepare.queue_time_us":2483,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":93634,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":20463,"thread_start_us":229016,"threads_started":1}
W20260709 12:28:27.024060  7812 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2780) took 2182 ms (client timeout 2618 ms). Trace:
I20260709 12:28:27.391314 14876 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.092587 14906 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.995803  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1274) took 1503 ms. Trace:
I20260709 12:28:27.062042  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3340) took 1662 ms. Trace:
I20260709 12:28:27.011755 14943 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:26.998153  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:27.015619  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3344) took 1579 ms. Trace:
I20260709 12:28:27.397544  7661 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:27.035099  8408 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:27.014271  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.020304  7732 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:28:26.985898  7777 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.570s (SENT)
W20260709 12:28:27.248493 14898 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.310s	user 0.000s	sys 0.005s
I20260709 12:28:27.394371  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:27.388168 14867 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:27.026270  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3233) took 1041 ms. Trace:
I20260709 12:28:27.034099 14115 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:27.273449  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 356700us
I20260709 12:28:27.034627  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:27.033172  8409 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:27.411057  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.954s (SENT)
I20260709 12:28:27.399536  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:27.387357 14756 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:27.386010  7736 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:27.394534  8407 rpcz_store.cc:276] 0709 12:28:25.492171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.492444 (+   273us) service_pool.cc:224] Handling call
0709 12:28:25.492894 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.776258 (+283364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.995457 (+1219199us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54004ba854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:26.995713 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631941376}
W20260709 12:28:27.391979  7812 rpcz_store.cc:269] 0709 12:28:24.841824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.841982 (+   158us) service_pool.cc:224] Handling call
0709 12:28:27.023706 (+2181724us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000686d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:27.023927 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2620987776}
I20260709 12:28:27.404886  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:27.395051  7730 rpcz_store.cc:276] 0709 12:28:25.399820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.489095 (+ 89275us) service_pool.cc:224] Handling call
0709 12:28:25.489614 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.610910 (+121296us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:25.975923 (+365013us) log.cc:844] Serialized 14 byte log entry
0709 12:28:26.252366 (+276443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:26.252437 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:26.252493 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.061670 (+809177us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540037df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:27.061752 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.061950 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214753,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":8575,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":14321,"spinlock_wait_cycles":278331776,"thread_start_us":639984,"threads_started":2,"wal-append.queue_time_us":449103}
W20260709 12:28:27.420979  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 435100us
I20260709 12:28:26.968730 14791 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.385227 14946 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:27.399298  7734 rpcz_store.cc:276] 0709 12:28:25.436401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.793426 (+357025us) service_pool.cc:224] Handling call
0709 12:28:26.827098 (+1033672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:26.855180 (+ 28082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.015355 (+160175us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540037cb50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:27.015532 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353622272}
I20260709 12:28:27.411955  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:27.411602  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:27.409705  7733 rpcz_store.cc:276] 0709 12:28:25.984767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.101827 (+117060us) service_pool.cc:224] Handling call
0709 12:28:27.025945 (+924118us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54002477d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:27.026163 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548375424}
I20260709 12:28:27.396324 14943 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.043111 14624 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.414534  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.371030 14880 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.428061  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:27.063288 14611 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:27.432729 14633 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:27.433537  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.443084  7733 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:27.406059  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3346) took 1601 ms. Trace:
I20260709 12:28:27.442397  7661 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:27.453301  7732 rpcz_store.cc:276] 0709 12:28:25.804216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.970519 (+1166303us) service_pool.cc:224] Handling call
0709 12:28:27.019919 (+ 49400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.405620 (+385701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.405969 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.428431  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:27.020695 14758 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:27.011263 14670 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:27.021168 14836 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:26.979200 14881 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:27.404277 14934 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:27.446497  7736 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:27.396347 14873 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:27.450493  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.433540 14701 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 12:28:27.458635 14956 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 12:28:27.441654 14582 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.460718  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.425468 14808 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:27.469444  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.431970 14115 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:27.433719 14876 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:27.441313 14791 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.498960  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.500517 14791 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:27.451560 14880 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.514631 14880 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:27.488451  7653 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:27.474951  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3223) took 1537 ms. Trace:
I20260709 12:28:27.518714  7736 rpcz_store.cc:276] 0709 12:28:25.937310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.419987 (+1482677us) service_pool.cc:224] Handling call
0709 12:28:27.446135 (+ 26148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.474655 (+ 28520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.474884 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.519347  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:27.519898 14866 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:27.520027  7653 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:27.520149 14888 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:27.520452  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1436) took 1486 ms. Trace:
I20260709 12:28:27.520998  7736 rpcz_store.cc:276] 0709 12:28:26.033863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.519585 (+1485722us) service_pool.cc:224] Handling call
0709 12:28:27.519934 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.520129 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.520153 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.520197 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:27.520228 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.520284 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.520390 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.523320  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1437) took 1488 ms. Trace:
I20260709 12:28:27.523803  7736 rpcz_store.cc:276] 0709 12:28:26.034788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.522191 (+1487403us) service_pool.cc:224] Handling call
0709 12:28:27.522700 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.522935 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.522965 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.523016 (+    51us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:27.523051 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.523109 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.523260 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:27.517922 14901 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.004s
I20260709 12:28:27.527654 14901 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:27.530577  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3332) took 1080 ms. Trace:
I20260709 12:28:27.531075  7653 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:27.502143  7734 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:27.461113 14611 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:27.532155  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3234) took 1534 ms. Trace:
I20260709 12:28:27.532675  7734 rpcz_store.cc:276] 0709 12:28:25.998188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.501542 (+1503354us) service_pool.cc:224] Handling call
0709 12:28:27.501925 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.531864 (+ 29939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.532092 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.534124  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:27.534612  7734 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:28:27.535279  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1441) took 1445 ms. Trace:
I20260709 12:28:27.485256  7732 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:27.535661  7734 rpcz_store.cc:276] 0709 12:28:26.090251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.533957 (+1443706us) service_pool.cc:224] Handling call
0709 12:28:27.534285 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.534984 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.535169 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.536247  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:27.536404  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3230) took 1568 ms. Trace:
I20260709 12:28:27.537462  7734 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:28:27.537050  7662 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:27.538339  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3239) took 1289 ms. Trace:
I20260709 12:28:27.538767  7734 rpcz_store.cc:276] 0709 12:28:26.249387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.536770 (+1287383us) service_pool.cc:224] Handling call
0709 12:28:27.537113 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.538038 (+   925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.538279 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.540817  7734 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 12:28:27.541249  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.541536  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3252) took 1225 ms. Trace:
W20260709 12:28:27.541785  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.541900  7734 rpcz_store.cc:276] 0709 12:28:26.316258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.540225 (+1223967us) service_pool.cc:224] Handling call
0709 12:28:27.540585 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.541260 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.541484 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.517020 14624 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:27.388751 14902 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:27.393512 14906 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:27.532884  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.552106  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2982) took 2783 ms. Trace:
I20260709 12:28:27.533213  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.531083  7806 rpcz_store.cc:276] 0709 12:28:26.450225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.450453 (+   228us) service_pool.cc:224] Handling call
0709 12:28:27.024547 (+574094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.382711 (+358164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.530351 (+147640us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54000686d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:27.530507 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1319360896}
I20260709 12:28:27.537060  7732 rpcz_store.cc:276] 0709 12:28:25.968421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.455012 (+1486591us) service_pool.cc:224] Handling call
0709 12:28:27.478192 (+ 23180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.535962 (+ 57770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.536316 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.557220  7656 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:27.563606  7653 raft_consensus.cc:2410] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:28:27.559298  7657 rpcz_store.cc:276] 0709 12:28:24.768376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.788498 (+ 20122us) service_pool.cc:224] Handling call
0709 12:28:24.788975 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.801424 (+ 12449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.801500 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.802151 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:28:25.577696 (+775545us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:25.577762 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.577866 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:27.041396 (+1463530us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:27.551641 (+510245us) raft_consensus.cc:1685] finished
0709 12:28:27.551686 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.551974 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13861,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":11494272,"thread_start_us":767574,"threads_started":1,"wal-append.queue_time_us":1055170}
I20260709 12:28:27.571436  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.467005 14670 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:27.572999  7656 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:27.474833 14881 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.581365  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3348) took 1776 ms. Trace:
I20260709 12:28:27.557648  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:27.586486  7735 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:27.560364  7733 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:27.587558  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:27.583901  7731 rpcz_store.cc:276] 0709 12:28:25.805062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.189028 (+1383966us) service_pool.cc:224] Handling call
0709 12:28:27.389681 (+200653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.390077 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.390106 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:27.390815 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 12:28:27.471613 (+ 80798us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:27.471657 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.471727 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:27.581051 (+109324us) raft_consensus.cc:1685] finished
0709 12:28:27.581085 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.581277 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73326,"spinlock_wait_cycles":6238080,"thread_start_us":80321,"threads_started":1,"wal-append.queue_time_us":188161}
I20260709 12:28:27.588888  7733 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:27.590016  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3256) took 1225 ms. Trace:
I20260709 12:28:27.590619  7733 rpcz_store.cc:276] 0709 12:28:26.364820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.588385 (+1223565us) service_pool.cc:224] Handling call
0709 12:28:27.588702 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.589640 (+   938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.589923 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:27.548964 14582 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:27.555019 14902 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:27.592114  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:27.544340 14611 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:27.594277  7733 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:27.604924  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:27.581523 14670 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:27.589596 14755 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:27.591317 14862 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:27.594491 14881 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:27.714131  7805 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:27.640512  7806 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:27.651396  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:27.719938  7812 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:27.611163 14873 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:27.740486  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:27.630383 14611 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:27.743319  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:27.744819  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:27.749030  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.627760 14901 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:27.746197  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 315177us
I20260709 12:28:27.743360  7805 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:27.745810  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 2997) took 1107 ms. Trace:
I20260709 12:28:27.757368  7660 rpcz_store.cc:276] 0709 12:28:26.638421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.639615 (+  1194us) service_pool.cc:224] Handling call
0709 12:28:26.640066 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:26.733453 (+ 93387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:26.733505 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:26.733575 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:26.733624 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.745293 (+1011669us) spinlock_profiling.cc:243] Waited 93 ms on lock 0x7b540017b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:27.745390 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.745685 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213998464}
I20260709 12:28:27.742431  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:27.758780  7812 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:27.768846  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 340526us
I20260709 12:28:27.770503  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.771550  7737 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:27.773169  7737 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:27.771332  7660 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:27.779395  7735 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:27.781280  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:27.785347  7812 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:27.790748  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.796574  7662 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:27.798455  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.833518  7660 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:27.835039  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:27.835876  7660 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:27.846984  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:27.847952  7655 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:27.849644  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:27.850548  7807 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:27.812346  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 371482us
I20260709 12:28:27.793319 14808 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:28:27.815769  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:27.820207  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:27.862902  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.877031  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:27.918195  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 102538us
I20260709 12:28:27.919445  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:27.920876  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:27.922314  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:27.924453  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65324.8us
I20260709 12:28:27.820996  7737 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:27.740945 14880 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:28:27.856554 14624 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 12:28:27.967751  7655 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:27.956444  7737 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:27.989629  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2983) took 1100 ms. Trace:
I20260709 12:28:27.990257  7655 rpcz_store.cc:276] 0709 12:28:26.888874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.942583 (+1053709us) service_pool.cc:224] Handling call
0709 12:28:27.967531 (+ 24948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.989253 (+ 21722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.989551 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:27.958184 14929 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.059s	user 0.015s	sys 0.005s
W20260709 12:28:27.995519  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.949923 14701 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:27.948304 14949 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:28:27.998971  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 326070us
I20260709 12:28:27.949442 14791 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 12:28:28.032084  7809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.014s	sys 0.023s
W20260709 12:28:28.047339  7809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.014s	sys 0.028s
I20260709 12:28:28.049155  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1307) took 1240 ms. Trace:
I20260709 12:28:27.949671 14633 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:28.039695  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2986) took 1137 ms. Trace:
I20260709 12:28:28.053666  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.051332 14582 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.057968  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.065634  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:28.056213 14758 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:28.022552 14961 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:28.081084  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158857us
I20260709 12:28:28.081456  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:27.967445 14914 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 12:28:28.053445  7655 rpcz_store.cc:276] 0709 12:28:26.902381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.002042 (+1099661us) service_pool.cc:224] Handling call
0709 12:28:28.038793 (+ 36751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.039327 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.039366 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:28.039408 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:28.039436 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.039494 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.039627 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2517,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":10208}
I20260709 12:28:28.042905 14949 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:28.043185 14701 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.089784  7702 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:28.087729 14823 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.060264 14959 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:28.108451  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:28.107610  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158322us
I20260709 12:28:28.075841 14963 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:28.088436 14880 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:28.097764 14902 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:28.114567 14935 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.207s	user 0.017s	sys 0.001s
I20260709 12:28:28.069125 14965 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:28:28.107558 14961 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:28:28.090410 14633 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:28:28.006675 14937 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:28:28.041522 14906 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.221925  7659 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:28.223074  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2993) took 1251 ms. Trace:
I20260709 12:28:28.199427 14949 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:28.202268  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1300) took 1637 ms (client timeout 1892 ms). Trace:
W20260709 12:28:28.206995  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 117359us
I20260709 12:28:28.205714 14701 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:28.185002 14914 consensus_queue.cc:1243] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:28:28.223958  7659 rpcz_store.cc:276] 0709 12:28:26.971312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.200601 (+1229289us) service_pool.cc:224] Handling call
0709 12:28:28.221435 (+ 20834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.222675 (+  1240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.222983 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:28.197471 14755 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.216346  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1445) took 1257 ms. Trace:
I20260709 12:28:28.243824  7731 rpcz_store.cc:276] 0709 12:28:26.959143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.609507 (+650364us) service_pool.cc:224] Handling call
0709 12:28:27.634354 (+ 24847us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:27.638604 (+  4250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.638649 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:27.740792 (+102143us) log.cc:844] Serialized 78 byte log entry
0709 12:28:27.998670 (+257878us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:28.207968 (+209298us) log.cc:844] Serialized 14 byte log entry
0709 12:28:28.215500 (+  7532us) log.cc:844] Serialized 14 byte log entry
0709 12:28:28.215834 (+   334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.215920 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.215973 (+    53us) raft_consensus.cc:1685] finished
0709 12:28:28.215992 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.216254 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15228,"raft.queue_time_us":113266,"raft.run_cpu_time_us":652,"raft.run_wall_time_us":46202,"thread_start_us":367782,"threads_started":2,"wal-append.queue_time_us":330371}
W20260709 12:28:28.245999  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158017us
I20260709 12:28:28.195775  7660 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:28.230484  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.211580  7807 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:28.229346  7654 rpcz_store.cc:269] 0709 12:28:26.548490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.548617 (+   127us) service_pool.cc:224] Handling call
0709 12:28:28.185770 (+1637153us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54000f3554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:28.185996 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467990912}
I20260709 12:28:28.216027 14880 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.215929 14957 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:28.214676 14902 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.248478  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.260856  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.266896  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:28.215060 14929 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:28:28.256907  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.225253 14823 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:28.257498  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:28.281088  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:28.282199  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.248803  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:28.257476 14880 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:28.286347  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.287343  7654 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:28.253665 14969 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:28.282956  7660 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:28.293663  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.295519  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.296463  7660 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:28:28.274250  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 331022us
W20260709 12:28:28.262753 14888 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.742s	user 0.015s	sys 0.000s
I20260709 12:28:28.222422 14935 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.265663 14785 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.290261  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.276955 14963 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:28.288347 14867 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.006s
I20260709 12:28:28.279912  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.315048  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.317365  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.299059  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2988) took 1344 ms. Trace:
I20260709 12:28:28.257943 14957 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.275314 14960 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:28.301867  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.304186  7654 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:28.264619 14902 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:28.257896 14755 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:28.301366 14967 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
I20260709 12:28:28.312496  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.297741  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:28.319159  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 889254us
I20260709 12:28:28.320539  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:28.301366 14970 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.183s	user 0.000s	sys 0.002s
I20260709 12:28:28.285588 14906 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:28.330359  7729 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:28:28.291661 14929 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:28.299450 14979 raft_consensus.cc:2764] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:28:28.330535  7658 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:28.334178  7729 pending_rounds.cc:85] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 13
I20260709 12:28:28.337596  7729 pending_rounds.cc:107] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:28:28.338155  7729 raft_consensus.cc:2889] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:28.332054  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:28.338824  7627 connection.cc:256] RPC call timeout handler was delayed by -2.102501704000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2796s for a response
I20260709 12:28:28.323808  7655 rpcz_store.cc:276] 0709 12:28:26.955073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.197290 (+1242217us) service_pool.cc:224] Handling call
0709 12:28:28.218616 (+ 21326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.298642 (+ 80026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.298682 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:28.298740 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:28.298774 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.298860 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.298993 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24014,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":79511,"threads_started":1}
I20260709 12:28:28.311762 14888 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:28.324945 14969 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:28.322667 14959 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:28.341686  8406 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 12:28:28.341987  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:28.288678 14823 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.348423  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.326435 14957 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:28.352093  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3350) took 1265 ms. Trace:
W20260709 12:28:28.355810 14971 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.180s	user 0.000s	sys 0.002s
I20260709 12:28:28.345911 14867 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:28.360713 14755 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.328682 14960 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.374598 14960 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:28.318584 14785 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:28.375860  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78254.8us
I20260709 12:28:28.351974 14987 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.363904  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:28.364427  7732 rpcz_store.cc:276] 0709 12:28:27.087113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.636717 (+549604us) service_pool.cc:224] Handling call
0709 12:28:27.742804 (+106087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.957294 (+214490us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:28.186886 (+229592us) log.cc:844] Serialized 14 byte log entry
0709 12:28:28.290964 (+104078us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:28.298526 (+  7562us) log.cc:844] Serialized 26 byte log entry
0709 12:28:28.298708 (+   182us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:28.298760 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.298856 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.351814 (+ 52958us) raft_consensus.cc:1685] finished
0709 12:28:28.351843 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.352006 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3679,"prepare.queue_time_us":263800,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4262,"raft.queue_time_us":69663,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":11858,"thread_start_us":455484,"threads_started":3,"wal-append.queue_time_us":135918}
I20260709 12:28:28.370415  7655 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:28:28.329855 14983 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:28.344929 14964 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.200s	user 0.000s	sys 0.002s
W20260709 12:28:28.363731  7627 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.024s (SENT)
I20260709 12:28:28.381474  7655 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:28.336000 14929 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:28.340580 14984 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:28.345147 14906 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:28.345753 14874 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.381902  8415 raft_consensus.cc:2410] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:28:28.360733 14935 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.381656  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.387087  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1446) took 1420 ms. Trace:
I20260709 12:28:28.376400 14876 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:28.387629  7733 rpcz_store.cc:276] 0709 12:28:26.966243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.610218 (+643975us) service_pool.cc:224] Handling call
0709 12:28:27.610541 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.629521 (+ 18980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.629585 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:27.630462 (+   877us) log.cc:844] Serialized 26 byte log entry
0709 12:28:27.849031 (+218569us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:27.849083 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.849161 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.386777 (+537616us) raft_consensus.cc:1685] finished
0709 12:28:28.386803 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.387009 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18396,"raft.queue_time_us":27590,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":26611,"thread_start_us":79078,"threads_started":1,"wal-append.queue_time_us":327436}
I20260709 12:28:28.376590 14968 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:28.375362 14965 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.362999 14969 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.389526  7659 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:28.384330  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.373636 14867 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.428052  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.394636 14874 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.404593 14906 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:28.405923  7627 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.050s (SENT)
I20260709 12:28:28.406286 14823 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:28.408387  7656 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:28:28.481482  7656 pending_rounds.cc:85] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 10
I20260709 12:28:28.481864  7656 pending_rounds.cc:107] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:28:28.482184  7656 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:28.409634 14758 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:28.049835  7657 rpcz_store.cc:276] 0709 12:28:26.808966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.584471 (+775505us) service_pool.cc:224] Handling call
0709 12:28:27.584974 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.585294 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:27.874796 (+289502us) log.cc:844] Serialized 14 byte log entry
0709 12:28:28.047815 (+173019us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:28.047962 (+   147us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:28.048738 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.048847 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.049065 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":420381,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":62,"raft.queue_time_us":434,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5870,"thread_start_us":460617,"threads_started":2}
I20260709 12:28:28.413700 14968 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.417124  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1309) took 1604 ms. Trace:
I20260709 12:28:28.417799  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2803) took 1430 ms. Trace:
I20260709 12:28:28.419184 14785 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.420434  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1304) took 1623 ms. Trace:
I20260709 12:28:28.492897  7658 rpcz_store.cc:276] 0709 12:28:26.796992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.930978 (+133986us) service_pool.cc:224] Handling call
0709 12:28:26.931374 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.336861 (+1405487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.420165 (+ 83304us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000f3f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:28.420370 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3217361792}
I20260709 12:28:28.399204 14755 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:28.422427  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.383358 14983 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.423025 14937 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:28.424412 14902 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.426581 14661 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.430620 14684 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.427666  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.428298  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.433643  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 2999) took 1037 ms. Trace:
I20260709 12:28:28.507473  7659 rpcz_store.cc:276] 0709 12:28:27.396409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.378647 (+982238us) service_pool.cc:224] Handling call
0709 12:28:28.389009 (+ 10362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.433250 (+ 44241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.433571 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:28.436468  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:28.508656  7804 raft_consensus.cc:606] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.509155  7804 raft_consensus.cc:614] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:28.436590  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:28.438052  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53793.8us
I20260709 12:28:28.511294  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:28.384912  7655 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
W20260709 12:28:28.514137  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 77699.9us
I20260709 12:28:28.439889 14867 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:28.382205 14987 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.444181 14929 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.445564 14965 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.389582 14984 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.393013 14876 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.471910  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.473124  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.474243  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.526499  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.475323  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:28.508008 14823 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:28.480502  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.531904  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.532714  8411 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 13.
I20260709 12:28:28.497069  7809 rpcz_store.cc:276] 0709 12:28:26.987155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.987374 (+   219us) service_pool.cc:224] Handling call
0709 12:28:27.398334 (+410960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.398886 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.398917 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:27.405189 (+  6272us) log.cc:844] Serialized 26 byte log entry
0709 12:28:28.047811 (+642622us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:28.047873 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.048014 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.417537 (+369523us) raft_consensus.cc:1685] finished
0709 12:28:28.417558 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.417723 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":11882,"thread_start_us":582193,"threads_started":1,"wal-append.queue_time_us":664846}
W20260709 12:28:28.532801  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126906us
I20260709 12:28:28.491533 14758 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:28.517055  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 175158us
I20260709 12:28:28.514149  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:28.433776 14969 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:28.532454 14876 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:28.545634  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.547533  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.548265  8409 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:28.527726 14594 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:28.522950 14959 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:28.484131 14836 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:28.499472 14968 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:28.484907 14994 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.528002  8412 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:28.478506  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.530284  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:28.520331 14808 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:28.509232 14937 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:28.495076  7653 rpcz_store.cc:276] 0709 12:28:26.813164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.588175 (+775011us) service_pool.cc:224] Handling call
0709 12:28:27.609838 (+ 21663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.610281 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.610324 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:27.625763 (+ 15439us) log.cc:844] Serialized 26 byte log entry
0709 12:28:27.959533 (+333770us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:27.959585 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.959664 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.416863 (+457199us) raft_consensus.cc:1685] finished
0709 12:28:28.416887 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.417055 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":17282,"thread_start_us":185706,"threads_started":1,"wal-append.queue_time_us":405404}
W20260709 12:28:28.500468  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 78151.9us
W20260709 12:28:28.534427  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.070s (SENT)
I20260709 12:28:28.586064  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:28.537870 14997 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:28.520953 14975 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:28.553333 14866 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:28.554381 14987 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:28.527462 14935 consensus_queue.cc:1175] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d72176ccaed049958ac68f706d4f052d" responder_term: 17 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:28.557505 14965 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.538312 14755 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:28.560011  8440 heartbeater.cc:646] Failed to heartbeat to 127.6.76.254:33941 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.76.254:33941 timed out after 15.000s (SENT)
I20260709 12:28:28.563658 14594 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:28.546530 14823 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:28.565223  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1435) took 3795 ms. Trace:
W20260709 12:28:28.566645 14670 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.929s	user 0.010s	sys 0.002s
I20260709 12:28:28.622991  7729 rpcz_store.cc:276] 0709 12:28:24.770318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.777841 (+  7523us) service_pool.cc:224] Handling call
0709 12:28:24.791484 (+ 13643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.344313 (+3552829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.564851 (+220538us) spinlock_profiling.cc:243] Waited 3.54 s on lock 0x7b540037e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:28.565126 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8137775872}
I20260709 12:28:28.543977 14929 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:28.528105 14902 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.519693  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:28.571951  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.18094e+06us
I20260709 12:28:28.539988 14961 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:28.519543  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 95895.5us
I20260709 12:28:28.573993  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:28.588126  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:28.542338 14984 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.597685  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3349) took 2792 ms. Trace:
I20260709 12:28:28.601276 14937 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.601775  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.585281 14808 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.594664 14753 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:28:28.603291 14959 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.604912 14873 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:28.607486 14874 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:28.613842 14995 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:28:28.615658 14881 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.870s	user 0.015s	sys 0.002s
I20260709 12:28:28.595041 14994 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:28.631054  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 111442us
W20260709 12:28:28.615808  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81428.8us
W20260709 12:28:28.655956  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.656342  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:28.656694  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5182014549995984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2889s for a response
W20260709 12:28:28.657189  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5106020319994968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26480000000000004s for a response
I20260709 12:28:28.658308 14937 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:28.658674 14785 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.650820  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:28.664301  7731 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:28.664526 14983 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:28.652201 14874 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:28.665242 14836 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:28.658788 15002 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:28.640105  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.653007 14866 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:28.667001 14866 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.667644 14866 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:28.641233  8414 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:28.612222 15000 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:28.647317 14755 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:28:28.647581 14594 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.649837  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:28.596663 14996 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:28:28.663973 14785 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.629944 14670 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.665947 14874 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:28.641803 14902 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:28.638994  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50931.1us
W20260709 12:28:28.623838  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 109817us
W20260709 12:28:28.617736  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.694537  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:28.694955  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:28.650055  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.697698  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.650446  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:28.638564  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 64698.6us
I20260709 12:28:28.699141  7733 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:28.638270 14975 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:28.684790  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:28.659348  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3337204639997253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3145s for a response
W20260709 12:28:28.710251  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2464204639995842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32420000000000004s for a response
I20260709 12:28:28.711982  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.634382  7728 rpcz_store.cc:276] 0709 12:28:25.805356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.390755 (+1585399us) service_pool.cc:224] Handling call
0709 12:28:27.407280 (+ 16525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:27.822834 (+415554us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:27.955201 (+132367us) log.cc:844] Serialized 14 byte log entry
0709 12:28:28.336385 (+381184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:28.369229 (+ 32844us) log.cc:844] Serialized 26 byte log entry
0709 12:28:28.369413 (+   184us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:28.369447 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.369523 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.556876 (+187353us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180048fa60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:28.597133 (+ 40257us) raft_consensus.cc:1685] finished
0709 12:28:28.597288 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.597558 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10818,"prepare.queue_time_us":334140,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":58,"raft.queue_time_us":71226,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":11986,"spinlock_wait_cycles":4105472,"thread_start_us":693743,"threads_started":3,"wal-append.queue_time_us":282956}
I20260709 12:28:28.701268  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:28.644855 14965 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:28:28.698163  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:28.702739  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:28.703329 14993 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:28.703459 14873 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:28.764302  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:28.678254 14943 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.766410  7736 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:28.704245 14960 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.705243 14701 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.705783 14753 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:28.706943  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:28.826421  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 119561us
I20260709 12:28:28.644649 14808 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:28.632283 14961 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.676003 15000 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.656347 14994 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.754642  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.770222 15011 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:28.840909  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:28.799960  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 141713us
I20260709 12:28:28.669847 14949 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.617141 14997 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.831312  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:28.835855  7626 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.020s (SENT) [suppressed 42 similar messages]
W20260709 12:28:28.836808  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 110314us
I20260709 12:28:28.837266 15012 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:28.838294  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 128436us
I20260709 12:28:28.914222  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:28.851521 14943 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:28.678752 14983 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 12:28:28.652143 14881 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.700862 14755 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.857169  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:28.857282 15015 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.831477 15001 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:28.829584 14975 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:28.843005 14961 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:28.853175 14993 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:28:28.828629 14902 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:28.690910 14984 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.142s
I20260709 12:28:28.916713  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:28.916859  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:28.841991 15000 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:28.706981 14957 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.920019  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:28.907626 14582 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:28.855978 14753 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:28.922529  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 168025us
I20260709 12:28:28.942885 15015 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:28.838892  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 136219us
I20260709 12:28:28.924998  7803 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:28.956257  7812 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:28.956564  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:28.957249  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:28.925834  8407 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:28:28.927310  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.807658 14630 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:28.930680  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:28.934355  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.937286  7807 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:28.936935  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:28.839318 14758 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.831744 14999 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.158s
I20260709 12:28:28.919262 14943 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.955075  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:28.922873  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:28.995357  7737 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
W20260709 12:28:28.994240  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63652.2us
I20260709 12:28:28.962780  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:28.830574 14823 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:28.934731 15016 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:29.009078  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:29.011353  7659 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:28:29.011991  7659 pending_rounds.cc:85] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 8
I20260709 12:28:29.011459  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:29.012560  7659 pending_rounds.cc:107] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:28:28.961969  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.994397  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:29.016628  7803 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:28.938789 15001 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:28.995285 14753 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.019457 15001 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:28.998195  7655 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:28.965852  7662 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:29.023571  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.965480  7804 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:29.005252  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.924424 14969 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.025593  7653 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:28:28.939510 14755 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.013525  7659 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:29.030567 15015 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:29.031211 15005 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:28:29.031684 15016 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:28.956257 14582 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:29.032416 14758 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:29.032871 14582 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:29.033320 14881 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:29.033947  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:29.035538 14968 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.039472  7730 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 19. Ops: []
I20260709 12:28:29.040493 15016 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:29.041342 15016 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:29.041435 14975 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.043061  7655 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:29.044472  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:29.045758  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:28.997948 14943 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:28.964099 14630 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:29.021790  7660 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:28.941285 14994 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:28:28.940912 15011 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:29.028753  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:29.080818  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:29.056658  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 112740us
I20260709 12:28:29.025439 14993 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:29.021755 14785 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.090287  7656 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:29.030886 14753 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:28.944015 15014 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:29.106246  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:29.104406  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 132257us
I20260709 12:28:28.975106 14902 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:29.073808 14630 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:29.072914 14968 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:29.114218  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 117643us
W20260709 12:28:29.107349  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 412867us
I20260709 12:28:29.118734  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:29.134017  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:29.143842  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.144538  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:29.145164  7658 raft_consensus.cc:2410] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:28:29.145300  7804 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:29.151897  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:29.152789  8414 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:29.157024  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:29.157591  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.103739 14670 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.108654 14862 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.142567 14993 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:29.173092  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:29.182121  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 227159us
I20260709 12:28:29.192786  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:29.194972  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:29.197917  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:29.167300 14701 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:29.203079  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:29.202014 15014 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:29.214620 15023 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:29.186371 15027 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:28:29.219606 14758 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:29.220425 14994 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:29.105032 15011 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.242883  7804 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:29.246763  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:29.175616 14975 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 12:28:29.281672  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3356) took 1878 ms (client timeout 1778 ms). Trace:
I20260709 12:28:29.283349  7734 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:28:29.277858  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:29.172318 14862 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.214676 14949 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:29.282275  7735 rpcz_store.cc:269] 0709 12:28:27.403439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.411680 (+  8241us) service_pool.cc:224] Handling call
0709 12:28:29.281426 (+1869746us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400282b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:29.281592 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365576576}
I20260709 12:28:29.289911  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.290915  7735 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.248562 15014 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:29.295449  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.299782  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:29.311414  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:29.312070  7659 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:29.301673 14957 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.306545  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.316030  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.317037  7735 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.292514 14785 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:29.279325 15023 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:28:29.182884 14902 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.332711  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:29.350668  8412 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:29.263708 15016 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
W20260709 12:28:29.337329  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.976s (SENT)
I20260709 12:28:29.374519  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:29.329710 14755 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.324690 14785 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:29.383607 15029 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:28:29.383857 14785 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:29.320689 14876 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.003s
I20260709 12:28:29.386631 14876 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.389796 14876 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:29.404055  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:29.321149 14867 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.796s	user 0.013s	sys 0.001s
W20260709 12:28:29.417429 15030 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:28:29.319869 15024 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:29.424388  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:29.452970  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:29.457007 14969 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:29.458092  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.445194 15024 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:29.484517 15024 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:29.484767  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3253) took 3161 ms. Trace:
I20260709 12:28:29.485594  7734 rpcz_store.cc:276] 0709 12:28:26.323279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.542755 (+1219476us) service_pool.cc:224] Handling call
0709 12:28:29.283033 (+1740278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:29.285913 (+  2880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.484343 (+198430us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400282b50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:29.484650 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4001447168}
I20260709 12:28:29.490121 14889 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:29.490775 14889 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:29.491782 14889 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.492874 14889 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:29.495610  7660 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:28:29.497159  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:29.498031  7660 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:29.466830 14992 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:29.535698 14992 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:29.537101 14997 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:29.472858  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1087) took 1151 ms. Trace:
I20260709 12:28:29.538386  8407 rpcz_store.cc:276] 0709 12:28:28.321714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.321997 (+   283us) service_pool.cc:224] Handling call
0709 12:28:28.322554 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:28.959145 (+636591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.472532 (+513387us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:29.472779 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386516864}
I20260709 12:28:29.474898  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1088) took 1140 ms. Trace:
I20260709 12:28:29.539728  8408 rpcz_store.cc:276] 0709 12:28:28.334809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.334956 (+   147us) service_pool.cc:224] Handling call
0709 12:28:28.346354 (+ 11398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:28.522649 (+176295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.522706 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:28.528764 (+  6058us) log.cc:844] Serialized 52 byte log entry
0709 12:28:28.598559 (+ 69795us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:28.598641 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.474469 (+875828us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400587854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:29.474522 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:29.474568 (+    46us) raft_consensus.cc:1685] finished
0709 12:28:29.474583 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:29.474825 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7707,"spinlock_wait_cycles":404372864,"thread_start_us":38880,"threads_started":1,"wal-append.queue_time_us":48205}
I20260709 12:28:29.344365 14993 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:29.535004  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.601094  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.602236  7728 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.559966  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.595404 14997 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.595992 14969 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:29.596879 14963 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.644318  7734 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.446103 14867 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.658973  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.660316  7728 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.647399  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.649158 14987 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:29.595000 14960 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:29.663087 15026 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.537032 14992 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.731681  7654 pending_rounds.cc:85] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 15
I20260709 12:28:29.748242  7654 pending_rounds.cc:107] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:28:29.748670  7654 raft_consensus.cc:2889] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:29.734167  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.730147  7734 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:28:29.752116  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.753595  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.754352  7728 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:29.755556  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.755692  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.756605  7728 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
W20260709 12:28:29.757424  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.696941 14987 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:29.760874  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.774497  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:29.758890  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.763123  7737 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:29.645155 14997 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.643560 14969 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:29.770267  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.018s	sys 0.017s
W20260709 12:28:29.781962  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.018s	sys 0.018s
I20260709 12:28:29.769173 14992 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:29.788935  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.008s
W20260709 12:28:29.792184  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.008s	sys 0.009s
W20260709 12:28:29.792655  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 564011us
I20260709 12:28:29.798324  7660 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:29.799912  7659 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:29.801239  7734 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:28:29.800689  7811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.009s	sys 0.031s
W20260709 12:28:29.801923  7811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.031s
W20260709 12:28:29.789155 15000 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.793s	user 0.018s	sys 0.007s
I20260709 12:28:29.818089 14684 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.758279 14984 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:28:29.820318  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:29.799099 15032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.007s	sys 0.034s
I20260709 12:28:29.831009  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:29.839185  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:29.843358  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:29.846590  7656 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.009s
I20260709 12:28:29.847059  7656 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:29.852617  8407 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:29.823447 14960 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:29.829828 15032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.008s	sys 0.034s
W20260709 12:28:29.838016 14880 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.539s	user 0.050s	sys 0.016s
I20260709 12:28:29.822542 14901 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:29.859712  8412 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:29.825883 14684 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.861392  8408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
W20260709 12:28:29.862195 14866 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.194s	user 0.030s	sys 0.013s
I20260709 12:28:29.863999 14785 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.866215  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:29.915177  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.866470 14957 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:29.824280 15000 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.868537 14867 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:29.827932 14937 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.169s	user 0.023s	sys 0.002s
I20260709 12:28:29.882791  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:29.870873 14876 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:29.932060  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:29.933396  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.933696  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3227) took 3988 ms. Trace:
I20260709 12:28:29.940950  8415 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:29.941448  7730 rpcz_store.cc:276] 0709 12:28:25.945285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.421681 (+1476396us) service_pool.cc:224] Handling call
0709 12:28:28.599000 (+1177319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:29.040172 (+441172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.664468 (+624296us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54002477d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:29.933445 (+268977us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002477d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:29.933635 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718576640}
W20260709 12:28:29.926326  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62509.8us
I20260709 12:28:29.823668 14987 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:29.964213  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:29.886801 14901 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:29.963529  7653 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 14: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:29.943079  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:29.969936  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:29.968226  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1283) took 1499 ms. Trace:
I20260709 12:28:29.933053 15011 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:29.982237  8405 rpcz_store.cc:276] 0709 12:28:28.468491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.532987 (+ 64496us) service_pool.cc:224] Handling call
0709 12:28:28.533334 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.545760 (+ 12426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.545785 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:28.545819 (+    34us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:28.545840 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.967885 (+1422045us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b54007b3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:29.967952 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:29.968133 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28112640}
W20260709 12:28:29.961268 15035 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 12:28:29.983582  7730 raft_consensus.cc:2368] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 19.
I20260709 12:28:29.922928 14866 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.968694 14984 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.013223  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.021317  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.022400  7730 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:30.024058  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.025745  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.026818  7730 raft_consensus.cc:2393] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:30.028051  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:29.988423 14961 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.031s	user 0.011s	sys 0.009s
W20260709 12:28:30.001513 14943 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.904s	user 0.030s	sys 0.008s
I20260709 12:28:29.896510 14880 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:30.008643 14582 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.975s	user 0.034s	sys 0.008s
I20260709 12:28:30.041478  7733 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:30.041957  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:30.011061 14929 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.379s	user 0.014s	sys 0.017s
I20260709 12:28:30.000485 14901 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:30.046041  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3011) took 1084 ms. Trace:
W20260709 12:28:30.047786  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82817.6us
I20260709 12:28:30.057886  7658 rpcz_store.cc:276] 0709 12:28:28.961476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.961592 (+   116us) service_pool.cc:224] Handling call
0709 12:28:30.045706 (+1084114us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540028da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.045948 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295602048}
I20260709 12:28:29.984419 14935 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:28:30.055411 14969 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:30.055411  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 94793.9us
I20260709 12:28:29.740036 15026 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:30.068189  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.680s (SENT)
W20260709 12:28:30.068811 15041 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 12:28:30.054774 14808 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.201s	user 0.006s	sys 0.039s
I20260709 12:28:30.074707  7732 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:30.050804  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88514.2us
I20260709 12:28:30.009074 14876 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:30.058285  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:30.037034  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.068753  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:30.040885 14943 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:29.645416 14963 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.080791  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
W20260709 12:28:30.081378 15051 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:28:30.038194 14961 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.065320 14997 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.037040 14984 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:30.068811 15043 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:30.059681 14902 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:28:30.081156 14937 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:28:30.087378 15024 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.602s	user 0.025s	sys 0.009s
I20260709 12:28:30.061384 14582 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.085021 14785 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:28:30.111451 14866 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:30.111598 15039 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:28:30.112149 14874 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.116469 14684 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.119904 15054 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.148523 15024 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.155568 15054 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:30.158784  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.159390 14949 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:30.159898 14949 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:30.120549 14876 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.124061 15000 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.124660 14961 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:30.116808 14963 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.158600 15024 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.129061 14862 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:30.130241 15048 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.138s	user 0.000s	sys 0.004s
W20260709 12:28:30.130968 15010 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.134s	user 0.002s	sys 0.000s
I20260709 12:28:30.185235 14987 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:30.133808 14984 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:30.137859 14889 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.645s	user 0.018s	sys 0.001s
I20260709 12:28:30.080634 14808 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.153991 14880 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.154510 14937 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:30.202747 14937 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.203963 14808 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:30.143908 14630 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.989s	user 0.018s	sys 0.005s
I20260709 12:28:30.128607 14943 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.160992 14949 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.158700 15054 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.176436 14874 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:30.216957 14630 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.223824 14949 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:30.226385 15026 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:30.210592 15001 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.190s	user 0.029s	sys 0.016s
I20260709 12:28:30.248481 15001 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.252494 15001 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:30.264339 15027 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.009s	sys 0.010s
W20260709 12:28:30.270502 15027 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.010s	sys 0.010s
W20260709 12:28:30.319478  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.319834  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1318) took 1352 ms. Trace:
I20260709 12:28:30.320709  7654 rpcz_store.cc:276] 0709 12:28:28.966678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.730138 (+763460us) service_pool.cc:224] Handling call
0709 12:28:29.730604 (+   466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:29.749510 (+ 18906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:29.749544 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:29.750144 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 12:28:29.931696 (+181552us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:29.931740 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.931807 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.318634 (+386827us) raft_consensus.cc:1685] finished
0709 12:28:30.318731 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.319452 (+   721us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12604,"thread_start_us":181199,"threads_started":1,"wal-append.queue_time_us":332160}
W20260709 12:28:30.336634  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.339144  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1460) took 1225 ms. Trace:
I20260709 12:28:30.339807  7729 rpcz_store.cc:276] 0709 12:28:29.113751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.114668 (+   917us) service_pool.cc:224] Handling call
0709 12:28:29.115159 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:29.115700 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:29.115746 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:29.116363 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:28:29.786323 (+669960us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:29.984008 (+197685us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.077249 (+ 93241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.077375 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.338473 (+261098us) raft_consensus.cc:1685] finished
0709 12:28:30.338530 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.338914 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20629,"prepare.queue_time_us":249566,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":52,"raft.queue_time_us":52666,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":38393,"thread_start_us":887409,"threads_started":3,"wal-append.queue_time_us":683288}
I20260709 12:28:30.326234  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3019) took 1191 ms. Trace:
I20260709 12:28:30.344837  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1438) took 4303 ms. Trace:
I20260709 12:28:30.343513  7661 rpcz_store.cc:276] 0709 12:28:29.135133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.818198 (+683065us) service_pool.cc:224] Handling call
0709 12:28:29.818569 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:29.819003 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:29.819059 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:29.823494 (+  4435us) log.cc:844] Serialized 52 byte log entry
0709 12:28:29.995643 (+172149us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:30.051992 (+ 56349us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.052705 (+   713us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.058263 (+  5558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.058363 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.193877 (+135514us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b1800350b60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:30.325326 (+131449us) raft_consensus.cc:1685] finished
0709 12:28:30.325404 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.325938 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5260,"prepare.queue_time_us":228914,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"raft.queue_time_us":14126,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":14173,"spinlock_wait_cycles":14508928,"thread_start_us":199878,"threads_started":2,"wal-append.queue_time_us":153042}
I20260709 12:28:30.345449  7736 rpcz_store.cc:276] 0709 12:28:26.041690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.524877 (+1483187us) service_pool.cc:224] Handling call
0709 12:28:27.525258 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.808016 (+1282758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.344128 (+1536112us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:30.344610 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2853863680}
W20260709 12:28:30.085628  7627 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.754s (SENT)
I20260709 12:28:30.348280  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:30.348896  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:30.349773  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.352468  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2819) took 1423 ms. Trace:
W20260709 12:28:30.353943  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:30.354964  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 270851us
W20260709 12:28:30.136691  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1447) took 1712 ms (client timeout 1904 ms). Trace:
I20260709 12:28:30.152814 14582 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:30.357190  8406 rpcz_store.cc:269] 0709 12:28:28.424493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.427281 (+  2788us) service_pool.cc:224] Handling call
0709 12:28:30.136322 (+1709041us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.136589 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3513197824}
I20260709 12:28:30.141500  8414 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:30.329087  7662 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:30.343549  7735 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.007s	sys 0.010s
W20260709 12:28:30.368055  7735 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.009s	sys 0.010s
I20260709 12:28:30.368810  7730 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
W20260709 12:28:30.364395  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.352559  7805 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:30.380990 14935 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.380723 14906 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:30.382211 14935 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.384177 14943 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:30.385550 14670 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:30.371976  7734 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:28:30.377410 14901 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.389580  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1312) took 1427 ms. Trace:
I20260709 12:28:30.390805  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1455) took 1582 ms. Trace:
I20260709 12:28:30.394076  7734 pending_rounds.cc:85] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:28:30.401432 14670 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.403910 14902 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.405543 14670 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.405184 14902 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.406585  8408 raft_consensus.cc:606] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:30.407374  8408 raft_consensus.cc:614] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:30.390821  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1458) took 1307 ms. Trace:
I20260709 12:28:30.408797  7732 rpcz_store.cc:276] 0709 12:28:29.083325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.083552 (+   227us) service_pool.cc:224] Handling call
0709 12:28:29.084014 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.075402 (+991388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.390188 (+314786us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:30.390591 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266835840}
I20260709 12:28:30.414880  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:30.419473  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.394491  7657 rpcz_store.cc:276] 0709 12:28:28.962091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.963126 (+  1035us) service_pool.cc:224] Handling call
0709 12:28:28.963667 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.066284 (+1102617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.066333 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.066406 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:30.066470 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.220272 (+153802us) spinlock_profiling.cc:243] Waited 904 us on lock 0x7b4800080358. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:30.380460 (+160188us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400398ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:30.380594 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.389311 (+  8717us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6863,"raft.queue_time_us":9797,"raft.run_cpu_time_us":3446,"raft.run_wall_time_us":151374,"spinlock_wait_cycles":2520830080}
I20260709 12:28:30.424913 14624 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:30.426039  7657 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:30.397995  7734 pending_rounds.cc:107] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:28:30.394872  7733 rpcz_store.cc:276] 0709 12:28:28.808387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.809184 (+   797us) service_pool.cc:224] Handling call
0709 12:28:28.809720 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.041970 (+1232250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.339069 (+297099us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400212d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:30.390603 (+ 51534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832302848}
W20260709 12:28:30.428331  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 342713us
I20260709 12:28:30.434033  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:30.435799  7730 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:28:30.435731 15054 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:30.437311  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 332519us
I20260709 12:28:30.440229  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3026) took 1301 ms. Trace:
I20260709 12:28:30.440665 14823 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:30.446266  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:30.444272 14969 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:30.445971  7657 rpcz_store.cc:276] 0709 12:28:29.138444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.424679 (+1286235us) service_pool.cc:224] Handling call
0709 12:28:30.425526 (+   847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:30.431610 (+  6084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.440086 (+  8476us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:30.448570 14630 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.448906  7728 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:30.430068  7734 raft_consensus.cc:2889] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:30.445145 14901 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.450680  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:30.442221 14876 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:30.431294 14935 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.353345  7807 rpcz_store.cc:276] 0709 12:28:28.928195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.960624 (+ 32429us) service_pool.cc:224] Handling call
0709 12:28:28.961015 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:28.998778 (+ 37763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.998813 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:28.999648 (+   835us) log.cc:844] Serialized 52 byte log entry
0709 12:28:29.541089 (+541441us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b18003b91a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:29.821403 (+280314us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:29.825813 (+  4410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.825891 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.350874 (+524983us) raft_consensus.cc:1685] finished
0709 12:28:30.350941 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.351914 (+   973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32124,"raft.queue_time_us":33242,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":11961,"spinlock_wait_cycles":15221632,"thread_start_us":279978,"threads_started":1,"wal-append.queue_time_us":864810}
I20260709 12:28:30.482692  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3368) took 1273 ms. Trace:
I20260709 12:28:30.479158  7731 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:30.480690  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3023) took 1339 ms. Trace:
I20260709 12:28:30.486860  7661 rpcz_store.cc:276] 0709 12:28:29.137516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.346735 (+1209219us) service_pool.cc:224] Handling call
0709 12:28:30.394655 (+ 47920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.474965 (+ 80310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.475001 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.476790 (+  1789us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:30.476853 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.476932 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.477105 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:30.490097  7661 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:30.491762  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1311) took 1529 ms (client timeout 1690 ms). Trace:
W20260709 12:28:30.492529  7656 rpcz_store.cc:269] 0709 12:28:28.961795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.962996 (+  1201us) service_pool.cc:224] Handling call
0709 12:28:30.484826 (+1521830us) spinlock_profiling.cc:243] Waited 80.2 ms on lock 0x7b54000f3054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.491424 (+  6598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184348416}
I20260709 12:28:30.496894  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:30.498190  7728 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:30.500411  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3028) took 1351 ms. Trace:
I20260709 12:28:30.503858  7661 rpcz_store.cc:276] 0709 12:28:29.140043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.488721 (+1348678us) service_pool.cc:224] Handling call
0709 12:28:30.489586 (+   865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.490788 (+  1202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.491125 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:30.511511  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:30.511778  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:30.513075  8407 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:30.519062  7658 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:28:30.502521  7735 rpcz_store.cc:276] 0709 12:28:29.209552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.661967 (+452415us) service_pool.cc:224] Handling call
0709 12:28:29.662370 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:29.662612 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:30.368994 (+706382us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.481920 (+112926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.482042 (+   122us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:30.482131 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.482267 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.482572 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":320110,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":8248,"raft.queue_time_us":129,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":335,"thread_start_us":741207,"threads_started":2,"wal-append.queue_time_us":112781}
W20260709 12:28:30.518476  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.447805 14969 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.534278  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3029) took 1393 ms. Trace:
I20260709 12:28:30.541740  7656 rpcz_store.cc:276] 0709 12:28:29.140382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.494396 (+1354014us) service_pool.cc:224] Handling call
0709 12:28:30.501050 (+  6654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.533716 (+ 32666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.533758 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.533849 (+    91us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.533889 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.533989 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.534171 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:30.545873  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.513028  8406 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:30.552944  8407 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:28:30.559337  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3025) took 1364 ms. Trace:
I20260709 12:28:30.560249 14633 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:30.560820 14633 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:30.561913  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:30.563455 14630 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:30.565799  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:30.567550  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:30.567880 14791 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.557828 14624 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:28:30.568843 14791 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.569631 14624 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:30.570142 14624 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:30.571650 14876 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:30.572474 14594 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:30.576489  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3021) took 1439 ms. Trace:
I20260709 12:28:30.576960  7658 rpcz_store.cc:276] 0709 12:28:29.136844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.058933 (+922089us) service_pool.cc:224] Handling call
0709 12:28:30.518174 (+459241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:30.519941 (+  1767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.576218 (+ 56277us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54000f3050. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.576421 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013871360}
I20260709 12:28:30.537534 14901 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:28:30.576478  7656 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:30.575084 14808 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:30.588997  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3032) took 1447 ms. Trace:
I20260709 12:28:30.592444  7658 rpcz_store.cc:276] 0709 12:28:29.141298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.578104 (+1436806us) service_pool.cc:224] Handling call
0709 12:28:30.578770 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.579101 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.579126 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.579188 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.588682 (+  9494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.588778 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.588930 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:30.560005  7659 rpcz_store.cc:276] 0709 12:28:29.138182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.384699 (+1246517us) service_pool.cc:224] Handling call
0709 12:28:30.385404 (+   705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.386011 (+   607us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:30.386763 (+   752us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.478482 (+ 91719us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.478571 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:30.500826 (+ 22255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.500907 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.502259 (+  1352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39572,"prepare.queue_time_us":4807,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":87,"raft.queue_time_us":10991,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":25539,"thread_start_us":82084,"threads_started":1,"wal-append.queue_time_us":101238}
I20260709 12:28:30.572503 14876 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:30.595898 15013 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:30.596582 15013 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:30.596801 14624 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.597167 14876 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.600142 14999 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:30.598234 15013 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.600451  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:30.602391  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3031) took 1461 ms. Trace:
I20260709 12:28:30.608098  7656 rpcz_store.cc:276] 0709 12:28:29.140976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.558158 (+1417182us) service_pool.cc:224] Handling call
0709 12:28:30.575875 (+ 17717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.590039 (+ 14164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.602321 (+ 12282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:30.600471  8415 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:30.615068 15013 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:30.619918  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58245.9us
I20260709 12:28:30.622218  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:30.623590 14630 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:30.623675 14701 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:30.624289 14630 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:30.624382 14701 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:30.624958 14630 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:30.630355  7731 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
W20260709 12:28:30.633368  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.818s (SENT)
I20260709 12:28:30.562575 14633 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.507270 14823 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 12:28:30.637079 14633 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:30.626351 14701 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.602013 14902 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:30.687213 14701 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:30.689450  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:30.690478  7656 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:30.699469  7624 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:30.702584 14962 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:30.703909  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1160) took 1855 ms. Trace:
I20260709 12:28:30.706563 14960 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.707321 14960 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:30.705618  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.36736e+06us
I20260709 12:28:30.711462 14624 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:30.716305 14791 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:30.718788  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.720223  7656 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:30.720041  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1314) took 1754 ms (client timeout 1632 ms). Trace:
W20260709 12:28:30.721823  7662 rpcz_store.cc:269] 0709 12:28:28.965365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.023241 (+ 57876us) service_pool.cc:224] Handling call
0709 12:28:30.719729 (+1696488us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.719958 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992761728}
I20260709 12:28:30.723780  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3027) took 1584 ms. Trace:
I20260709 12:28:30.724305  7657 rpcz_store.cc:276] 0709 12:28:29.139216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.449081 (+1309865us) service_pool.cc:224] Handling call
0709 12:28:30.449688 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:30.452924 (+  3236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.453012 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:30.454278 (+  1266us) log.cc:844] Serialized 26 byte log entry
0709 12:28:30.508834 (+ 54556us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.508868 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.508929 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.723516 (+214587us) raft_consensus.cc:1685] finished
0709 12:28:30.723540 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.723719 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1223,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":202,"thread_start_us":53803,"threads_started":1,"wal-append.queue_time_us":266905}
I20260709 12:28:30.729830  7657 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:30.732729  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:30.734323  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:30.736990  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.738049  7657 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:30.676491 14901 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:30.739271 14962 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:30.739648  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:30.740783 14962 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.742445 14937 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:30.742887 14962 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:30.743039 14937 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.743348  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.743605 14937 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:30.744885  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.741256  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.704480  7809 rpcz_store.cc:276] 0709 12:28:28.848882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.849129 (+   247us) service_pool.cc:224] Handling call
0709 12:28:28.849612 (+   483us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:28:28.912203 (+ 62591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.912247 (+    44us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:28:28.933827 (+ 21580us) log.cc:844] Serialized 104 byte log entry
0709 12:28:29.155149 (+221322us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:29.225013 (+ 69864us) log.cc:844] Serialized 14 byte log entry
0709 12:28:29.283689 (+ 58676us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.536395 (+1252706us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b180022edc0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:30.612844 (+ 76449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.674659 (+ 61815us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.703338 (+ 28679us) raft_consensus.cc:1685] finished
0709 12:28:30.703366 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.703638 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118343,"prepare.queue_time_us":130717,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":1493,"raft.queue_time_us":2679755,"raft.run_cpu_time_us":681,"raft.run_wall_time_us":8222,"spinlock_wait_cycles":2480512,"thread_start_us":218918,"threads_started":3,"wal-append.queue_time_us":462454}
W20260709 12:28:30.748147  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74906.1us
I20260709 12:28:30.749271  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:30.749432 14960 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.749603 15054 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:30.750413 15054 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:30.751478 14943 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:30.752128 14943 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:30.752318  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3024) took 1614 ms. Trace:
I20260709 12:28:30.752696  7728 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
W20260709 12:28:30.753248  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.753417 14943 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.758106  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.758355  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1315) took 1792 ms. Trace:
W20260709 12:28:30.759620  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1324) took 1791 ms (client timeout 1892 ms). Trace:
I20260709 12:28:30.760154  7662 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:30.760262  7653 rpcz_store.cc:269] 0709 12:28:28.968609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.033619 (+ 65010us) service_pool.cc:224] Handling call
0709 12:28:30.759210 (+1725591us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54000f3554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.759454 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135631232}
W20260709 12:28:30.760313 15075 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
I20260709 12:28:30.761794  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:30.758380  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.766813  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3022) took 1629 ms. Trace:
I20260709 12:28:30.758899  7655 rpcz_store.cc:276] 0709 12:28:28.965675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.047493 (+ 81818us) service_pool.cc:224] Handling call
0709 12:28:30.051073 (+1003580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.051358 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:30.086778 (+ 35420us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.320671 (+233893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.320798 (+   127us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:30.320970 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.321172 (+   202us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.757980 (+436808us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540028da50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:30.758128 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87216,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":59805,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":390,"spinlock_wait_cycles":2307042560,"thread_start_us":267360,"threads_started":2,"wal-append.queue_time_us":100600}
I20260709 12:28:30.772480 15015 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:30.773080 15015 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.773437 15015 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:30.773850  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 171290us
W20260709 12:28:30.774744  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.868s (SENT)
I20260709 12:28:30.775185  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:30.776481  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:30.775024 14969 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:30.769475  7654 rpcz_store.cc:276] 0709 12:28:29.137190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.337073 (+1199883us) service_pool.cc:224] Handling call
0709 12:28:30.338069 (+   996us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:30.339049 (+   980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.339105 (+    56us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:30.472996 (+133891us) log.cc:844] Serialized 78 byte log entry
0709 12:28:30.556526 (+ 83530us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.610012 (+ 53486us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.765102 (+155090us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.766448 (+  1346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.766531 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.766577 (+    46us) raft_consensus.cc:1685] finished
0709 12:28:30.766589 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.766745 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155158,"prepare.queue_time_us":83525,"raft.queue_time_us":47940,"raft.run_cpu_time_us":552,"raft.run_wall_time_us":21892,"spinlock_wait_cycles":5256960,"thread_start_us":82890,"threads_started":1,"wal-append.queue_time_us":250545}
I20260709 12:28:30.791044  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:30.796092  7657 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:30.794472  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2816) took 1878 ms. Trace:
I20260709 12:28:30.797477  7805 rpcz_store.cc:276] 0709 12:28:28.916413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:28.917421 (+  1008us) service_pool.cc:224] Handling call
0709 12:28:28.917749 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.374739 (+1456990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.778883 (+404144us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:30.794386 (+ 15503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3297277824}
I20260709 12:28:30.752714  7660 rpcz_store.cc:276] 0709 12:28:29.137869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.382227 (+1244358us) service_pool.cc:224] Handling call
0709 12:28:30.383018 (+   791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:30.390151 (+  7133us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:30.435430 (+ 45279us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.484319 (+ 48889us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:30.557187 (+ 72868us) log.cc:844] Serialized 26 byte log entry
0709 12:28:30.557333 (+   146us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.557356 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.557413 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:30.751914 (+194501us) raft_consensus.cc:1685] finished
0709 12:28:30.751947 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.752144 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6349,"prepare.queue_time_us":41862,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":85,"raft.queue_time_us":6691,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":14624,"thread_start_us":40685,"threads_started":1,"wal-append.queue_time_us":121215}
I20260709 12:28:30.807646  7656 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:30.821945 14965 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:30.823622  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:30.824671  7809 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:30.818799  7659 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:30.841421  7805 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:30.842422 14987 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:30.843549 14987 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:30.844796 14987 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.846060 14987 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:30.860071  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.863814  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:30.865069  7805 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:30.866317  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:30.866777  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:30.867743  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:30.869988  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:30.870564  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:30.871389  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96735.5us
I20260709 12:28:30.872208  7803 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:30.872318  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.878s (SENT)
I20260709 12:28:30.874748 14594 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:30.875329 14594 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:30.876549 14594 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.878037 14594 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:30.871434  7810 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:30.886102  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:30.888008  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:30.890182  7807 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:30.901268 14935 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:30.904721  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.908125  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3030) took 1767 ms. Trace:
I20260709 12:28:30.904969  7662 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:30.902367 14935 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:30.908952  7661 rpcz_store.cc:276] 0709 12:28:29.140680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.510837 (+1370157us) service_pool.cc:224] Handling call
0709 12:28:30.554209 (+ 43372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.564430 (+ 10221us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:30.639565 (+ 75135us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.907179 (+267614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.907333 (+   154us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:30.907458 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.907608 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.907943 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46164,"prepare.queue_time_us":75798,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":59,"raft.queue_time_us":24639,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":9708,"thread_start_us":254536,"threads_started":2,"wal-append.queue_time_us":173774}
I20260709 12:28:30.930341  7810 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:30.921060  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:30.936970  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:30.949704  7809 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:30.942724  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3033) took 1801 ms. Trace:
W20260709 12:28:30.965329  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.836388 15054 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:30.975123  7729 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:28:30.983383  7734 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:30.959546  7658 rpcz_store.cc:276] 0709 12:28:29.141565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.594594 (+1453029us) service_pool.cc:224] Handling call
0709 12:28:30.595189 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:30.596018 (+   829us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:30.715550 (+119532us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.942206 (+226656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:30.942290 (+    84us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:30.942351 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.942453 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:30.942637 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47356,"prepare.queue_time_us":168823,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":6483,"thread_start_us":212010,"threads_started":1}
I20260709 12:28:30.999622  7734 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:31.003840  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.008301 14987 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:31.009366 14949 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.642s	user 0.016s	sys 0.009s
I20260709 12:28:31.010561 15013 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:31.011817  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.012786 14935 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:31.013489  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.013962 14791 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.014779 14791 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.015385 14935 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:31.015656  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 393658us
I20260709 12:28:31.017174  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.017706  7733 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:31.018095 14791 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:31.019841 14814 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:31.016055 14935 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.024124  7729 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:31.026084  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:31.027776 15066 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:31.028528 15066 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:31.034428 15087 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.035434 15087 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.038323 15087 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:31.039211  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.046435  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.030112 15066 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.006933  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.053501  7656 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:31.024240 14935 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.075431  7662 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:31.087392 15015 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.058615 14630 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:31.093102  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 291453us
I20260709 12:28:31.093444  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:31.096634  8407 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:31.099946  7808 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:31.101655  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:31.055258 14949 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:28:31.106914  8406 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24. Ops: []
W20260709 12:28:31.106997  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 2.280s (SENT)
I20260709 12:28:31.101603 15015 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.115694  8408 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:31.128823 15080 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:31.105130 14633 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:31.143692  8407 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:31.145752  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:31.148893  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 142189us
I20260709 12:28:31.096508 14965 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:31.155085 15015 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:31.186895 15078 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:31.188128 15078 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.189721  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:31.192315  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.194432 15001 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.178362 14906 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:31.208855  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:31.183493 15015 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:31.169759 14633 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.184199 14582 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.218223  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:31.230826 15081 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:31.235320  7662 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:31.239354 15040 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:31.239583 15032 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:31.240253 15040 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.240785  7777 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.240610 15032 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.242560  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.243449 14791 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:31.244321 14791 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.244773 14633 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.245536 14633 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:31.195209 15001 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.247480 15080 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:31.249769 15001 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.250141 15079 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:31.251248 15078 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:31.242918  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.256501  7658 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.256547  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:31.246910 14791 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.213188 15013 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:31.249773  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:31.263247  7808 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:28:31.231184 14630 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.243077 15040 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.265941  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:31.208711 14906 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.182166 14965 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.219919 14937 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.276098  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:31.277722 14937 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.278285  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.229249 14582 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.279697  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:31.284262  7730 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.289278 14758 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:31.289948 14758 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.291204 14758 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.292711 14758 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:31.295434 14943 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:28:31.305490  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44831.6us
W20260709 12:28:31.307864  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58189.3us
I20260709 12:28:31.309435  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:31.235379 15081 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.312390  7658 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:31.313668  7654 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:31.314317  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.322827  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:31.315276 15001 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:31.325666 15001 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:31.325819  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.325306 15015 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.326207 15001 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.326620 15013 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:31.327867  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:31.328929  7660 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:31.330778  7655 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:31.319676  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.317519 15040 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:31.315362 15066 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.361901  7657 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:31.348294 15013 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:31.362879  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:31.344133 14943 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:31.345494 14594 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:28:31.353199 14582 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.363929 15080 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:31.384172  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69957.8us
W20260709 12:28:31.394459  7626 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.578s (SENT)
I20260709 12:28:31.381305 15013 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.398782  7654 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:28:31.398077 15088 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:31.406011 15088 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.406477 15088 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:31.422847  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.423545  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.577s (SENT)
I20260709 12:28:31.424378  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:31.428238  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.429051  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.434275  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:31.436239  7658 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:31.440887  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:31.441756  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.392537 14943 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.451325  7730 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:31.448683 15080 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:28:31.454689 15079 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:31.455350 15079 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.456772 15079 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.458813 15079 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:31.462639  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 149998us
W20260709 12:28:31.463812  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.472741  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:31.484637  7626 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.758s (SENT)
I20260709 12:28:31.490859 15001 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.492127  7653 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:28:31.494849  7660 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:31.495915  8408 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:31.493055 15087 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:31.566255  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58854.8us
I20260709 12:28:31.576083  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:31.500912 14633 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 12:28:31.576939  7810 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.577571 15082 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:31.578320 15082 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.579558 14791 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:31.580013 15082 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.581027 14791 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:31.581535 14791 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.582113 15082 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:31.584628 15001 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:31.589810  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139842us
I20260709 12:28:31.587399 15081 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.612023  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.592468 15087 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.622355  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:31.627461  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 366702us
I20260709 12:28:31.573088 14906 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:31.621335  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.635032  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:31.642063  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:31.653158 15097 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:28:31.652813 15087 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.647430 14630 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:31.656003 14630 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:31.656561  7805 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:31.657356 14630 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.647460 14633 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:28:31.660420 14633 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:31.660887 14633 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.590837 15013 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.661872 14633 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.663245 14633 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.663578  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.664868  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.582696 14791 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.645112 14987 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:31.668013 14987 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.668978 15087 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.667358 14791 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:31.670225 14987 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:31.671473 14758 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:31.648248  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:31.673161  7807 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:31.673908 14758 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:31.674365 14758 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:31.672123  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:31.688166  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 344830us
I20260709 12:28:31.699813  8408 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:31.714864  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.645989 14594 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:31.728572  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56511.4us
I20260709 12:28:31.721903  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:31.726313  7810 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:31.731544  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.732352  7805 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:31.737175  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.747303  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.746481  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:31.734552 15092 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.527s	user 0.002s	sys 0.005s
I20260709 12:28:31.675490 14758 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.772212 14758 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.772835 14935 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:31.773353 14935 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:31.775746  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.776206 14866 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:31.777073  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.781607  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.795362  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.795567  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:31.796015 15096 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:28:31.796362  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.797089  7812 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:31.797510 15099 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.797683  7806 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:31.709319 14987 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:28:31.798211 15099 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:31.795688  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.801024 15099 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.801694  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.801980  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:31.803020  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.776561  8407 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:28:31.803762  7804 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:31.804692  7810 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:28:31.805770  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.806448  7810 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:31.804802  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.807642  7804 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:31.807844 14943 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:31.748486 15087 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
W20260709 12:28:31.809597  7626 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.020s (SENT)
I20260709 12:28:31.758081 15001 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 12:28:31.820088  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.821687  7661 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:31.823873  8406 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:31.796404  7805 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:31.829224  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.736306 15080 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:28:31.830116  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:31.831955  7734 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:31.829845  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:31.748229 15082 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:31.796973 15081 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 12:28:31.844516  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.847339  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.850993  7807 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:31.852309  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.755429 14594 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:31.864949  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:31.888168  7807 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:31.852559  8406 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:31.878118  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.888470  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 268251us
I20260709 12:28:31.866009  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:31.890498  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 200246us
W20260709 12:28:31.898129  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 486944us
I20260709 12:28:31.837849 15088 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:31.924800  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.927595  7656 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 17. Ops: []
I20260709 12:28:31.930657  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.932588  7810 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.941440  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:31.944844  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:31.946413  8407 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:31.947419 14906 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
W20260709 12:28:31.981901 15079 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.004s
I20260709 12:28:31.990269 14758 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:31.971876 15099 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:31.992533  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69084.4us
W20260709 12:28:31.997794  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.004750  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196085us
I20260709 12:28:31.999665 14758 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.014297  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:32.017740  7803 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:32.044974  8408 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:28:32.046270  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.018339  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:32.019418 15089 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 12:28:32.019773  7811 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:28:32.043295  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:31.997601 15079 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.044114  7810 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:32.066511  8407 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:32.079954 14758 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.081681 14758 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:32.047628 15088 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:32.087960  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 324032us
I20260709 12:28:32.089745  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.090751  7810 raft_consensus.cc:2410] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:32.092315 14630 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:32.094677 14630 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:32.095860 14630 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:32.103087  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84858.4us
W20260709 12:28:32.109092  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 94940.8us
I20260709 12:28:32.113137  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:32.114111  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70981.9us
I20260709 12:28:32.116950  7729 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:32.118214  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:32.089177 15079 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.121245 15079 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:32.093294 15112 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:32.124100  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.124893  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:32.126346  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.127326  8405 raft_consensus.cc:2368] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17.
I20260709 12:28:32.129431  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.130409  7810 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:32.131632  8413 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:32.123449  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.131528  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.135892  7810 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:32.137351  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.138656  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.141856  8407 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:32.142674  7730 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:32.139532  7810 raft_consensus.cc:2376] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18. Re-sending same reply.
I20260709 12:28:32.151005  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.152555  7734 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:32.152319  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.153820  8406 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
W20260709 12:28:32.157225  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53316.6us
I20260709 12:28:32.157249  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.154729  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:32.158686 14906 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:32.160081  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:32.164865  7731 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:32.161767  7810 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:32.163523  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:32.170192  7737 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:32.175776  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:32.181282  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.181722  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:32.182168  8407 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:32.191497 14582 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:32.160327 15089 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:32.200070  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:32.183037  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.202598  7803 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:32.153090 15112 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.213953  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.215382  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:32.215890  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62220.5us
I20260709 12:28:32.223990  7732 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:32.225091  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:32.229066  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.228834  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.230994  7803 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:28:32.232265  7736 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:32.206979  8406 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:32.236263  7730 raft_consensus.cc:2376] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18. Re-sending same reply.
I20260709 12:28:32.239965  7809 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:32.241505  7775 leader_election.cc:400] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17.
I20260709 12:28:32.242059  7775 leader_election.cc:403] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:32.242955 14906 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:32.244927 15117 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:32.245553 15117 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.247144 15117 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.249003 15117 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:32.238088  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:32.229611 15080 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:32.266580  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.268167  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.269810  7733 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:32.224172 15001 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:28:32.281900  8408 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
W20260709 12:28:32.270608  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80808.6us
I20260709 12:28:32.287315  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.288458  7810 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:32.269599 15080 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:32.229492 14935 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.297224 14935 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:32.267060 14906 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 12:28:32.304356  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75527.8us
W20260709 12:28:32.318681  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.324234  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.324801  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:32.294258 15080 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:32.326905  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96309.9us
I20260709 12:28:32.346242  7806 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:32.347703 14791 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:32.348866  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.348973 14791 consensus_queue.cc:1243] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:32.349187 14582 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:32.350061  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.350478  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.351141  7807 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:32.285610 15112 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.356982  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.358714  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.361681  7807 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:28:32.377631  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.378533  7806 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:32.380151  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.381184  7810 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:32.389518 15001 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:32.390487 15001 consensus_queue.cc:1243] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:28:32.391306 15001 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:32.408984  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72530.8us
W20260709 12:28:32.412704  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74747.3us
W20260709 12:28:32.413439  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88756.9us
I20260709 12:28:32.413766 15135 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:32.415369 15082 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:32.409394 15137 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:32.416604  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:32.420331  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 150938us
I20260709 12:28:32.423601  7775 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.430133 15131 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:32.434540 15127 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:28:32.434541 15131 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.435757 15131 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:32.422024 15137 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.437232 15137 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.438529 15137 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:32.444654 15122 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:28:32.446828 15119 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:28:32.449982 15115 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:32.450594 15115 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.451049 15131 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.451730 15115 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.452675 15032 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:32.453220 15115 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:32.454698  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 154370us
I20260709 12:28:32.433873  7806 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:32.433883 15125 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:32.468778  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.455935  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.480628  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1328) took 1246 ms. Trace:
I20260709 12:28:32.481099  7656 rpcz_store.cc:276] 0709 12:28:31.234123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.236001 (+  1878us) service_pool.cc:224] Handling call
0709 12:28:31.236384 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:31.928155 (+691771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:32.480341 (+552186us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540033d6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:32.480553 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589119744}
I20260709 12:28:32.498656 15127 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:32.499311 15127 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.475772 14758 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:32.500458 14758 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.501832 14758 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.504947 15133 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:32.505555 15133 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.464756 15125 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.507612  7806 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:32.510466  7803 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:32.477370 15082 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:32.511592  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55470us
W20260709 12:28:32.510852  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 87289.3us
I20260709 12:28:32.420442  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "575c307765244f48890311159c0ee934"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:28:32.518508  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.519312  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.520581  7806 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:32.523687  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:32.525415  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.514611 14633 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17.
I20260709 12:28:32.512306 15125 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:32.519624 15120 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:32.528056 15082 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.549077 15142 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.549803 15142 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.549067 15082 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.533682 14630 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.556308  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.556908 15132 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:32.557292  7804 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:32.557667 15132 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.558121  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:32.558821  7805 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:32.559635 15120 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.561390  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.572041  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:32.572397  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 147192us
W20260709 12:28:32.574981  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 158456us
I20260709 12:28:32.556080 14630 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.583894  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:32.579849 15081 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.006s	sys 0.006s
I20260709 12:28:32.506940 15133 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.500423 15127 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.596347 15127 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:32.592787 15081 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.006s	sys 0.006s
I20260709 12:28:32.556365 15130 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:32.604696 15130 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.605710  8408 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:32.594911 15136 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:32.562743 14758 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:32.594053 15140 raft_consensus.cc:2764] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:28:32.605146 15130 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:32.591609  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.619801  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.613662  7805 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:32.614542  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:32.621065  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.573649 15080 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.658375 15080 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:32.660969 14630 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:32.625253  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 169403us
I20260709 12:28:32.632826 15142 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:32.558725 15132 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.634258 15131 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:32.560716 15120 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.634640  7804 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:32.609097 15136 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:32.677451 15136 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:32.683795 15099 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:28:32.684536 15099 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.685542 15099 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.686748 15099 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:32.686990  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3023) took 1434 ms. Trace:
I20260709 12:28:32.687563 15080 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:32.688169 15080 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.689304 15080 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.636194 15144 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:32.637305  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.641616  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.642652  8406 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:32.644137 14594 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:32.651463  7553 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:32.718308  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:32.715466  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:32.668097  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47102.7us
I20260709 12:28:32.700799  7808 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:28:32.720863  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2833) took 1453 ms (client timeout 1672 ms). Trace:
I20260709 12:28:32.701027  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:32.721386  7809 rpcz_store.cc:269] 0709 12:28:31.257172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.260575 (+  3403us) service_pool.cc:224] Handling call
0709 12:28:32.710097 (+1449522us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:32.710234 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209479936}
I20260709 12:28:32.687487  7662 rpcz_store.cc:276] 0709 12:28:31.252941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.257468 (+  4527us) service_pool.cc:224] Handling call
0709 12:28:31.268732 (+ 11264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:31.282086 (+ 13354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:31.282120 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:31.307057 (+ 24937us) log.cc:844] Serialized 26 byte log entry
0709 12:28:31.834681 (+527624us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:31.865772 (+ 31091us) log.cc:844] Serialized 14 byte log entry
0709 12:28:32.686212 (+820440us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b18003072a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:32.686444 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:32.686553 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:32.686601 (+    48us) raft_consensus.cc:1685] finished
0709 12:28:32.686711 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:32.686907 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20753,"prepare.queue_time_us":49350,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":75,"spinlock_wait_cycles":19345152,"thread_start_us":485570,"threads_started":1,"wal-append.queue_time_us":613559}
I20260709 12:28:32.703125  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:32.720129  7807 raft_consensus.cc:606] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:32.723531  7807 raft_consensus.cc:614] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:32.721010  7808 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.733117  7657 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:32.734221  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.734884  7656 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
W20260709 12:28:32.736567  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.737381  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.665913  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:32.740022  7728 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:32.744670  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:32.745436  8414 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:28:32.751363  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136913us
I20260709 12:28:32.719097 15088 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:32.758891  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.763926  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:32.766568  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:32.767302  8407 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19. Re-sending same reply.
I20260709 12:28:32.767649  8406 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:28:32.759142  7808 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.773416  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.774327  7735 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:28:32.742317 14594 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.778054  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.780412  7728 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:32.747362 15032 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.753222 14758 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:32.782697  7808 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:32.747936 14935 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:32.790904  7731 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:32.792481  7657 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:32.793249  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.794906  7662 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:32.757829 14949 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:32.801578  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99812.2us
I20260709 12:28:32.804064  7734 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:28:32.804507  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.793948  7659 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:32.725874 15144 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.807753  7653 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:28:32.800734  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:32.804255  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.812094  7655 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:32.812963  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:32.813498  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.814141  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
W20260709 12:28:32.815016  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 149201us
I20260709 12:28:32.775668 15115 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:32.814075  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:32.775441 14987 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:32.847380  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.850080  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.847749  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.856215  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:32.864183  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.865255  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:32.867063  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46663.8us
I20260709 12:28:32.868861  7659 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:32.785229 15112 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:32.810201 14949 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.886296 14949 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:32.806552 15117 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:32.897491  7811 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:28:32.904232  7658 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:32.781934 15088 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:32.912756  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 62768.8us
I20260709 12:28:32.785568 15120 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:28:32.913365  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.914369  7657 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:32.906946  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:32.921007 15145 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:32.847903 14758 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:32.928719  7659 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:32.803732 14935 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:32.907610  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 139830us
I20260709 12:28:32.801340 15032 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:32.819345  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:32.815832 14594 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.934044 15112 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:28:32.934273  7654 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
W20260709 12:28:32.937479  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73377.7us
I20260709 12:28:32.818580 15054 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.101s
I20260709 12:28:32.909935 15139 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:32.909173 15147 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:32.942945  7655 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:32.910154 15115 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:32.905157 14866 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:28:32.959532  7659 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:32.895999 15040 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:28:32.899322 14987 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:32.963177  7809 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:28:32.953056  7655 pending_rounds.cc:85] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 9
I20260709 12:28:32.964900  7655 pending_rounds.cc:107] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:32.965214  7655 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:32.949090  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.969496  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:33.031250  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:33.043323  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 94315.6us
W20260709 12:28:33.045562  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.941211 15131 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:28:33.060938  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.056083  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:32.932510 15104 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 12:28:33.067893 15131 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:33.074306 15131 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.076483 15131 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:28:33.080091  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.069137  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:33.114343  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68849.9us
I20260709 12:28:33.115252  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:33.116500  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:32.953189 14949 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.114s
I20260709 12:28:33.089936 15154 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:33.118212 15154 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.118682 15154 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:33.119166 14949 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:33.120561  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.121114 14949 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.121640 14949 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:33.090140  7811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:33.119521 14866 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:33.123600  7811 raft_consensus.cc:606] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:33.124053  7811 raft_consensus.cc:614] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:33.067638 15082 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.125514 15082 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:33.115275 15127 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.518s	user 0.002s	sys 0.012s
I20260709 12:28:33.064711 15117 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:33.128489  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.098958  7729 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:33.119715 15120 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:33.128168 15082 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.129577  7729 pending_rounds.cc:85] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:33.140722  7729 pending_rounds.cc:107] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:28:33.141124  7729 raft_consensus.cc:2889] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:33.133438 15109 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.136861  7733 raft_consensus.cc:2368] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:33.127089  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.136736 15117 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.147871 15103 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:33.148437 15103 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.148510 15015 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:33.148936 15103 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:33.129737 15125 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.006s
W20260709 12:28:33.153468  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.152032  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:33.170238 14866 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:33.172010 14866 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.173370 15161 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:33.173980 15161 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:33.174584 15161 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:33.193857  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.187419  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:33.044485 15147 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:33.249718  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.137717 15120 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:33.258271  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 202285us
I20260709 12:28:33.268074  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.272276  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.156996 15127 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:33.151477 15160 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.275283  7733 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:33.273102  7736 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:33.290707  7657 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:28:33.265394 15120 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:33.295912 15153 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:28:33.282469 15160 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.296947 15032 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.298235 15160 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:33.299156 15099 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.009s
I20260709 12:28:33.301653  7730 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:28:33.302134  7730 pending_rounds.cc:85] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:28:33.302382  7730 pending_rounds.cc:107] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:28:33.302623  7730 raft_consensus.cc:2889] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:33.101450 14758 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.149230 15015 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.275977 15127 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.307492  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:33.313647  7803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.003s	sys 0.013s
I20260709 12:28:33.187036 15125 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.276162 15066 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:28:33.317662  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:33.315873  7661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.000s	sys 0.017s
W20260709 12:28:33.329032  7661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.000s	sys 0.017s
I20260709 12:28:33.305703 15088 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.156962 15109 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:33.322266  7803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.004s	sys 0.013s
I20260709 12:28:33.324129  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.186692 15115 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:33.317530  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.014s
I20260709 12:28:33.331105  8413 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:28:33.314412 15127 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:33.338217  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.002s	sys 0.014s
I20260709 12:28:33.339699  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:33.336719  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:33.354646  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:28:33.355324 15157 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:33.359926 15178 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 12:28:33.364233 15165 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:28:33.330176 15032 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:33.367808 15156 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.273s	user 0.000s	sys 0.003s
I20260709 12:28:33.415762 15109 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.340976 15112 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.368456 15168 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:33.369962 15125 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.358911  7804 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:33.373539  7811 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:28:33.357754 15121 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 12:28:33.378497  7808 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.001s
I20260709 12:28:33.343390  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.344285  7807 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
W20260709 12:28:33.388444  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 419075us
I20260709 12:28:33.379799  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:33.415917  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.441272 14965 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:33.441769  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2839) took 1733 ms. Trace:
I20260709 12:28:33.441951  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:33.442015 14965 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:33.443491 15113 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:28:33.442248  7808 rpcz_store.cc:276] 0709 12:28:31.708392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.715105 (+  6713us) service_pool.cc:224] Handling call
0709 12:28:31.715461 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:32.700681 (+985220us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54002c7f20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f241c0 00007fc001f25231 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c
0709 12:28:33.441386 (+740705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:33.441420 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:33.441459 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:33.441490 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.441582 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:33.441706 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1490,"raft.queue_time_us":3222,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":641424768}
I20260709 12:28:33.447589  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:33.418632 14594 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.456583 15146 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.429910 15109 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.465435  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:33.471580  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 91938.1us
W20260709 12:28:33.457957 15136 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.780s	user 0.007s	sys 0.006s
I20260709 12:28:33.441781 15125 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.416783 15112 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.415782 15032 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:33.476887  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:33.485178  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 148528us
I20260709 12:28:33.487423  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:33.488075  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:33.461937 14987 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.495754 15181 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:33.484632 14594 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:33.505529  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:33.504629  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:33.515261  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.484964 15160 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:33.494323  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5219201369995972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 12:28:33.503953 15179 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:33.543509  7627 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.578s (SENT)
I20260709 12:28:33.544209  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:33.545447  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:33.432883 15157 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.512444 14594 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.500059 15121 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:28:33.474988 15109 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.509238 15181 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:33.465512 15146 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.490953 15125 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.540186 14987 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:33.540971 15131 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.562083  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.555902 14594 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:33.545272 15177 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:33.577620  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51772.9us
W20260709 12:28:33.566490  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78524.3us
I20260709 12:28:33.486169 15136 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.550980 15160 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:33.570636 15015 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.571043 14949 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.573403  7736 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:33.558786 15181 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:33.577028  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53639us
I20260709 12:28:33.577296 15174 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 12:28:33.489748 14965 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.580330  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:33.570396 15115 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:33.554869 15130 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.855s	user 0.008s	sys 0.006s
I20260709 12:28:33.573071 14987 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.606926  7737 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:33.583379 15179 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:28:33.612182  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68722.6us
W20260709 12:28:33.612787  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3437201369997638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 12:28:33.576674 15131 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.617388  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:33.611012 15130 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.611115 14987 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:33.616019 15115 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:33.650213  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.610491 15015 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:33.639297 15160 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.640110 15136 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.641505 15125 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.642423 14949 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:33.644256 15169 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:28:33.644682 15146 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.645601  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3367) took 1788 ms. Trace:
I20260709 12:28:33.646108 15157 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.678885  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:33.668632  7810 rpcz_store.cc:276] 0709 12:28:31.856995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.382147 (+525152us) service_pool.cc:224] Handling call
0709 12:28:32.711637 (+329490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:32.736925 (+ 25288us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:32.736958 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:32.772321 (+ 35363us) log.cc:844] Serialized 26 byte log entry
0709 12:28:32.871759 (+ 99438us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:32.871811 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:32.871890 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:33.111522 (+239632us) raft_consensus.cc:1685] finished
0709 12:28:33.111551 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:33.645214 (+533663us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54003c5ad0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:33.645521 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6366,"raft.queue_time_us":21620,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4075,"spinlock_wait_cycles":746209536,"thread_start_us":23706,"threads_started":1,"wal-append.queue_time_us":89008}
W20260709 12:28:33.681046 15172 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.143s	user 0.002s	sys 0.000s
I20260709 12:28:33.632468 15188 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:33.683503 15188 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.677765 15130 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.684180 15188 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:33.635474 15141 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:28:33.611588 15179 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:33.689142 15179 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:33.592204 15168 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:28:33.584782 15177 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.667783 15088 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:33.680935 15089 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:33.693084 15089 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:33.694468 15177 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.694195 15089 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.625136  7736 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
W20260709 12:28:33.709635 15182 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 12:28:33.713048 15103 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.006s
I20260709 12:28:33.713595 15103 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.692034 14987 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.665580 15117 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:33.724581  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 144338us
I20260709 12:28:33.727909  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:33.625547 15170 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.091s
W20260709 12:28:33.730202  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.184s (SENT)
W20260709 12:28:33.705101 15161 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.009s
I20260709 12:28:33.704239 15125 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.742393  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:33.744345  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.719625 15088 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:33.722725 15113 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.748648 15196 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:33.756608 15196 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:33.757054 15196 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:33.750124 15125 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.759994 15109 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.760826 15112 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.762147 15125 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.762352 15131 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.745958 15191 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.763916 15191 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.765205 15120 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.767539  7733 raft_consensus.cc:2368] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:33.770339 15196 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.771380 15088 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.729395 15089 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:33.776825 15103 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:33.777486  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:33.777989  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.777632 15113 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.780191 15113 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.780929 15196 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.783056  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:33.783526  8406 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:33.785514 15115 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.731869 15117 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.790427 15117 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.741674 15161 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.793596 15161 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.794458  7553 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 10 to 14, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:33.797518  7553 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:33.799722  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:28:33.831970  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 43943.2us
I20260709 12:28:33.766463 15191 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:33.837121  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 109272us
I20260709 12:28:33.838503  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:33.845032  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:33.845515  7556 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 270) took 41890 ms (client timeout 15000 ms). Trace:
I20260709 12:28:33.832235 15103 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:33.851101  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 261390us
I20260709 12:28:33.841841 15099 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.834934 15130 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:33.847074  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:33.848301  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:33.843564 15196 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:33.867079  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:33.871431  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:33.837107 15161 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
W20260709 12:28:33.864502 15154 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.006s
I20260709 12:28:33.879647  7653 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
W20260709 12:28:33.883384  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 177059us
W20260709 12:28:33.884246  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.194s (SENT) [suppressed 25 similar messages]
I20260709 12:28:33.885178 15179 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.888550  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:33.894490  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3053) took 1104 ms. Trace:
I20260709 12:28:33.895150  7662 rpcz_store.cc:276] 0709 12:28:32.790446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.855365 (+ 64919us) service_pool.cc:224] Handling call
0709 12:28:32.855667 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:32.960378 (+104711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:32.960411 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:32.960458 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:32.960493 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.894087 (+933594us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000f4454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:33.894187 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:33.894394 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240262912}
I20260709 12:28:33.899996  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.901865  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.876825 15130 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.906002 15130 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.909646  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:33.909999  7656 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 13. Current term is 14. Ops: [13.14-13.14]
I20260709 12:28:33.908337  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:33.909202  7811 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:33.915405 15154 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.918231 15154 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.874074 15087 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:33.919181 15087 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:33.919736 15087 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:33.920122  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:33.925541 14594 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.927305  7661 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:33.927522 15127 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.930156 15127 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.930999  7732 raft_consensus.cc:2368] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:33.873927 15099 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.932482  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:33.933507 15099 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.909277  7654 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:28:33.885133  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.229s (SENT)
I20260709 12:28:33.887837  7553 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:33.945943  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:33.903206  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.867718 15120 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:33.955518  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67046.8us
I20260709 12:28:33.960311  7736 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:33.954362  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:33.958784  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:33.845916  7556 rpcz_store.cc:269] 0709 12:27:51.959900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.063474 (+103574us) service_pool.cc:224] Handling call
0709 12:28:33.845259 (+41781785us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:21.678301 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:21.748509 (+ 70208us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:21.748571 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:28:21.748600 (+    29us) tablet.cc:662] Decoding operations
0709 12:28:21.862515 (+113915us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:21.862645 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:21.862690 (+    45us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:28:21.878655 (+ 15965us) tablet.cc:701] Row locks acquired
0709 12:28:21.878731 (+    76us) write_op.cc:260] PREPARE: finished
0709 12:28:21.879001 (+   270us) write_op.cc:270] Start()
0709 12:28:21.879207 (+   206us) write_op.cc:276] Timestamp: P: 1783600101878897 usec, L: 0
0709 12:28:21.895110 (+ 15903us) op_driver.cc:348] REPLICATION: starting
0709 12:28:22.065379 (+170269us) log.cc:844] Serialized 10547 byte log entry
0709 12:28:23.620074 (+1554695us) op_driver.cc:464] REPLICATION: finished
0709 12:28:24.190475 (+570401us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc005958ef0
0709 12:28:25.090518 (+900043us) write_op.cc:301] APPLY: starting
0709 12:28:25.105202 (+ 14684us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:26.659690 (+1554488us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:26.659746 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:28.220124 (+1560378us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:28.361369 (+141245us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 12:28:28.361459 (+    90us) write_op.cc:312] APPLY: finished
0709 12:28:28.362141 (+   682us) log.cc:844] Serialized 236 byte log entry
0709 12:28:28.571491 (+209350us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:28.624254 (+ 52763us) write_op.cc:454] Released schema lock
0709 12:28:28.638921 (+ 14667us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:28.764248 (+125327us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:28.807979 (+ 43731us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46300,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1348645,"apply.run_cpu_time_us":187667,"apply.run_wall_time_us":3679763,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116987,"num_ops":27,"prepare.queue_time_us":601929,"prepare.run_cpu_time_us":28376,"prepare.run_wall_time_us":1009132,"raft.queue_time_us":180923,"raft.run_cpu_time_us":21291,"raft.run_wall_time_us":1077296,"replication_time_us":1721448,"spinlock_wait_cycles":5821184,"thread_start_us":2214919,"threads_started":5,"wal-append.queue_time_us":912792,"wal-append.run_cpu_time_us":93667,"wal-append.run_wall_time_us":2793436}]]}
I20260709 12:28:33.951025 14633 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.961052  7810 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:28:33.964455  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.965029  7662 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:33.967541  7809 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:33.968257  7811 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:33.962610 15196 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:33.971191 15196 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:33.971204 15161 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:33.971805 15196 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:33.955294 15040 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.972487 15040 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:33.975419  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:33.979849  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:33.981196  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:33.974970 15040 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:33.982290 15109 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:33.982983  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:33.983413 15013 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:28:33.970903  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85823us
I20260709 12:28:33.983940 15013 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:33.984279  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2942201369996837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:28:33.984879  7627 connection.cc:256] RPC call timeout handler was delayed by -1.2905871609996211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2622s for a response
W20260709 12:28:33.985858  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0321913799998583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
I20260709 12:28:33.983883  7653 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:33.968948 14633 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.992990 14987 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:33.993283  7809 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:28:33.985011 15013 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:33.995117 15013 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:33.995702  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3398) took 1388 ms. Trace:
I20260709 12:28:33.994853 14633 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.996106  7737 rpcz_store.cc:276] 0709 12:28:32.606935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.607090 (+   155us) service_pool.cc:224] Handling call
0709 12:28:32.608336 (+  1246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:33.607453 (+999117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.995417 (+387964us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400248bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:33.995610 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290164096}
I20260709 12:28:33.997221  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:33.997659  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.998826  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:34.000458  7653 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:28:34.001459  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.001580 14594 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:33.983065 15040 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:34.002679  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.003723  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:34.004181 15204 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:28:34.004714  8406 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:33.994889 15133 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:33.985946  7553 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 12 to 16, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 16 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:34.007368 15109 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:34.007787  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.664s (SENT)
I20260709 12:28:34.008617  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.009651 14594 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:34.010560  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1497) took 1155 ms. Trace:
I20260709 12:28:34.011049  7731 rpcz_store.cc:276] 0709 12:28:32.855704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.855879 (+   175us) service_pool.cc:224] Handling call
0709 12:28:34.010301 (+1154422us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002d7b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:34.010456 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255585664}
I20260709 12:28:34.012544  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.013823  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:33.991591 15088 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.014945 15088 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.002413 15181 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.018045 15181 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:34.015027  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.007905 15109 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.019587 15109 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:34.019610  7731 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:34.020670  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.021243  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.021934  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.022264  7731 raft_consensus.cc:2368] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:34.022643  7733 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:34.024976  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1493) took 1299 ms. Trace:
I20260709 12:28:34.025643  7730 rpcz_store.cc:276] 0709 12:28:32.726012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.736718 (+ 10706us) service_pool.cc:224] Handling call
0709 12:28:32.747252 (+ 10534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:33.303122 (+555870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.024620 (+721498us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b540039b2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:34.024892 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273944320}
W20260709 12:28:34.070140  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62421.2us
I20260709 12:28:34.071870  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.073210  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.075321  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.758s (SENT)
I20260709 12:28:34.076164  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.083298  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.091825  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.095870  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.097317  7736 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:34.074688  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.100679  7730 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:34.106446  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.107963  7811 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 22 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:28:34.114424  7653 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:34.069490  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.018255 15171 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.557s	user 0.003s	sys 0.000s
I20260709 12:28:34.121613  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.125779  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.130811  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:34.133020  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57781.1us
I20260709 12:28:34.134020  7811 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:28:34.134734  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.896s (SENT)
W20260709 12:28:34.029145 15186 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.473s	user 0.001s	sys 0.000s
I20260709 12:28:34.135948  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.137017  7731 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:34.138250  7656 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 12:28:34.139565  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.014041 15188 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.141383 15013 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.142306  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.142046 15013 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.027985 15181 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.144521  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:34.146318  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:34.150830 15188 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:34.151265  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.972s (SENT)
I20260709 12:28:34.165997  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.151959 15195 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:34.153633  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3405) took 1040 ms. Trace:
I20260709 12:28:34.154610  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.155339 15142 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.009047  7553 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 6 to 20. New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:34.042194 15179 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.156863  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.177186  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.057344 15129 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:34.148742  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.139397  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:34.160954  7656 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:34.161651  7736 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:34.162554  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.181787  7811 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:34.163633  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:34.032166 15147 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:34.115407 15088 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.164417  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:34.168740  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.155473 15143 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:28:34.169536  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.157460 14594 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:34.170168 15154 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.041314 15087 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.171913  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.172230 15013 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:34.167114  7728 rpcz_store.cc:276] 0709 12:28:33.113260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:33.113546 (+   286us) service_pool.cc:224] Handling call
0709 12:28:33.143794 (+ 30248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:33.145402 (+  1608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:33.145485 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:33.156592 (+ 11107us) log.cc:844] Serialized 26 byte log entry
0709 12:28:33.395822 (+239230us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:33.395885 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.395987 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:34.028286 (+632299us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:34.153287 (+125001us) raft_consensus.cc:1685] finished
0709 12:28:34.153314 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.153560 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1048,"raft.queue_time_us":39515,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":56,"spinlock_wait_cycles":5900544,"thread_start_us":113475,"threads_started":1,"wal-append.queue_time_us":258851}
I20260709 12:28:34.173326  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:34.244800  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 257099us
I20260709 12:28:34.173904  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.176234  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.245963  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:34.183727  7730 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:34.186691  8408 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:34.246637  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48129.1us
I20260709 12:28:34.248420  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:34.248936  7733 raft_consensus.cc:2410] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:28:34.251175  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:34.262527  7656 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:34.198715  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:34.246726  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.119933 15082 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 12:28:34.195549  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.289144  7730 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:28:34.264992  7737 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:34.268631  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.271389 15172 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:28:34.272311  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.273700 15114 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:34.179085 14906 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:34.274895  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1182) took 2161 ms. Trace:
I20260709 12:28:34.179556 15129 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:34.188566 15089 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 12:28:34.158576 14965 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:28:34.173418 15181 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.284718  7658 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:34.312064  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 160818us
I20260709 12:28:34.256656  7731 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
W20260709 12:28:34.317446  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.020s (SENT)
I20260709 12:28:34.317931  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.300944  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.319069  7730 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:34.320112  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.270910 15032 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:34.179456 14630 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.331969  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.332796  7730 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 14.
I20260709 12:28:34.300549  7803 rpcz_store.cc:276] 0709 12:28:32.113221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.517192 (+403971us) service_pool.cc:224] Handling call
0709 12:28:32.517773 (+   581us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:32.527603 (+  9830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:32.527645 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:32.678148 (+150503us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:32.678896 (+   748us) log.cc:844] Serialized 52 byte log entry
0709 12:28:33.985894 (+1306998us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b180022edc0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1
0709 12:28:33.986082 (+   188us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:33.986122 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.986197 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:34.017461 (+ 31264us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:34.274651 (+257190us) raft_consensus.cc:1685] finished
0709 12:28:34.274675 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.274831 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2234,"raft.queue_time_us":26241,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":55,"spinlock_wait_cycles":85572480,"thread_start_us":633873,"threads_started":1,"wal-append.queue_time_us":657045}
I20260709 12:28:34.302065  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.258054 15125 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:34.188585 15180 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:34.266700 15112 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:34.335908  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.343632 14630 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.343766 15089 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:34.351279 15089 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:34.257290 15137 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:34.282522 15184 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:34.325188  7702 connection.cc:256] RPC call timeout handler was delayed by -1.913374066000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:28:34.365180  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9061740660004034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 12:28:34.365324  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47933.2us
W20260709 12:28:34.365824  7702 connection.cc:256] RPC call timeout handler was delayed by -1.8791740660003597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 12:28:34.366401  7702 connection.cc:256] RPC call timeout handler was delayed by -1.8521740660003161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:28:34.366806  7702 connection.cc:256] RPC call timeout handler was delayed by -1.8125740660002521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 12:28:34.367205  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7765740660001939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 12:28:34.367831  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7594740660001662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:28:34.368287  7702 connection.cc:256] RPC call timeout handler was delayed by -1.42197406600053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 12:28:34.368682  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3661740660004398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
W20260709 12:28:34.369217  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3355740660003903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 12:28:34.265646 15154 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.341703  8408 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:34.341851  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.372191  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.061s (SENT)
I20260709 12:28:34.344892  7730 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:28:34.186621 15142 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.298611 15114 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:34.289516 15109 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.340723  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.364303 14630 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.372704  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.402662  7731 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 14 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:28:34.373723  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3292) took 1050 ms. Trace:
I20260709 12:28:34.404068  7730 rpcz_store.cc:276] 0709 12:28:33.323253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.344143 (+1020890us) service_pool.cc:224] Handling call
0709 12:28:34.344588 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:34.373318 (+ 28730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.373646 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:34.406038  7730 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:34.377058  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:34.369607  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3328740660003859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
W20260709 12:28:34.408082  7702 connection.cc:256] RPC call timeout handler was delayed by -1.3193740660003641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
W20260709 12:28:34.408627  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2554740660002608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 12:28:34.409070  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1726740660001269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2324s for a response
I20260709 12:28:34.410007  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:34.330576 15099 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:34.398514  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.356207 14594 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:34.400662  7658 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:34.330675 14633 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:28:34.421828  8407 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:34.329087 15013 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:28:34.416042  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.333033 14906 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.432734  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:34.426944  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.434958  8406 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:34.418979  7737 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:28:34.480664  8414 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 12:28:34.480737  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48060.2us
I20260709 12:28:34.352304 15089 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.402128 15066 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.440234  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3411) took 1003 ms. Trace:
W20260709 12:28:34.484392  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 118156us
I20260709 12:28:34.496425  7733 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:34.556617  7737 pending_rounds.cc:85] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
W20260709 12:28:34.488140  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78289.7us
I20260709 12:28:34.557062  7737 pending_rounds.cc:107] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:28:34.557402  7737 raft_consensus.cc:2889] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:34.508054  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1496) took 1691 ms. Trace:
I20260709 12:28:34.493319  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:34.413647 15137 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:34.567408  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:34.539207  8405 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:34.488948 15144 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:34.554116  7731 rpcz_store.cc:276] 0709 12:28:33.436897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.403740 (+966843us) service_pool.cc:224] Handling call
0709 12:28:34.439451 (+ 35711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:34.439729 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:34.439771 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:34.439839 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:34.439889 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.439968 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.440133 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:34.562136  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3410) took 1129 ms. Trace:
I20260709 12:28:34.417482 15120 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:28:34.562227  7811 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:34.545984  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.508810 15184 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.501601 15109 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:34.533591 15087 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:34.513192 15153 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:28:34.522549 15196 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.004s
W20260709 12:28:34.483559 15210 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:28:34.490159 15209 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:28:34.574139  7733 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 12:28:34.488124 15206 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:34.497828 15195 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:34.506608 15040 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:34.484635 15180 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:34.574992  7737 rpcz_store.cc:276] 0709 12:28:33.432600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.372582 (+939982us) service_pool.cc:224] Handling call
0709 12:28:34.412092 (+ 39510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:34.557895 (+145803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.562065 (+  4170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:34.563594  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:34.664912  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.665208  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 101749us
I20260709 12:28:34.529706 15154 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.517431 15181 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:34.565183  7627 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.447s (SENT)
I20260709 12:28:34.598054  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:34.538408  7736 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:34.600646  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54804.2us
I20260709 12:28:34.560556  7734 rpcz_store.cc:276] 0709 12:28:32.816638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.817152 (+   514us) service_pool.cc:224] Handling call
0709 12:28:32.858027 (+ 40875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:33.278397 (+420370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:33.278429 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:33.278479 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:33.278515 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.507800 (+1229285us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540037e454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:34.507853 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.507982 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966171520}
I20260709 12:28:34.561075  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:34.686094  7735 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:28:34.606542  8411 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:34.608245  8407 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:28:34.687263  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.522573 14935 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.573019 15162 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.487025 15114 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.419947 15099 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.593152 15113 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.609201  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3409) took 1178 ms. Trace:
I20260709 12:28:34.733328  7728 rpcz_store.cc:276] 0709 12:28:33.431147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.354024 (+922877us) service_pool.cc:224] Handling call
0709 12:28:34.354436 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:34.578174 (+223738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:34.578229 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:34.578299 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:34.578349 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.608922 (+ 30573us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001c43d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:34.608988 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.609136 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513849728}
I20260709 12:28:34.614147  7656 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:28:34.419858 15129 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.597172 15184 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.567425 15066 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:34.647639 15211 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:28:34.682853 15109 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:34.665931  7732 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:34.609314 15188 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:34.689785 15132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.552s	user 0.030s	sys 0.033s
I20260709 12:28:34.713649 15089 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:34.612381 15179 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:34.724234  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.782088  7657 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:34.613708  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.736793 15190 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.171s
I20260709 12:28:34.593720 15137 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.663149 15120 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.593380 15147 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:34.739464  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:34.792091  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3293) took 1462 ms. Trace:
I20260709 12:28:34.792642  7729 rpcz_store.cc:276] 0709 12:28:33.330021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.613291 (+1283270us) service_pool.cc:224] Handling call
0709 12:28:34.792008 (+178717us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:34.660050 15040 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:34.672505 15180 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.746410  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:34.748695  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.677563  7662 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:34.765066 14906 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:34.684870 15142 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:34.489259  7553 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 10 to 17. New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:34.744639 15113 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.748080  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.746986 15206 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:28:34.738816 15162 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.779706 15188 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.744575 15195 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:28:34.772966 15129 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.716053 15143 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:28:34.729609 14965 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 12:28:34.785156  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.785637  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1501) took 1176 ms. Trace:
I20260709 12:28:34.916716  7730 rpcz_store.cc:276] 0709 12:28:33.609495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.429603 (+820108us) service_pool.cc:224] Handling call
0709 12:28:34.430025 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:34.773346 (+343321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:34.773396 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:34.773466 (+    70us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:34.773525 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.785286 (+ 11761us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540037c654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:34.785350 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.785558 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782379264}
I20260709 12:28:34.785902  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.786286  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 221128us
I20260709 12:28:34.774099 15144 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.777590 15109 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:34.780072 15132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.669s	user 0.030s	sys 0.034s
I20260709 12:28:34.928571  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:34.794055  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 454968us
W20260709 12:28:34.853730  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.821225  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:34.809067  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69753.5us
I20260709 12:28:34.646839 15196 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.789734 15114 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:34.919919  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.453s (SENT)
I20260709 12:28:34.924065  7732 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:34.925494  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.947901  7811 raft_consensus.cc:2376] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17. Re-sending same reply.
I20260709 12:28:34.949369  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.950265  7811 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:34.813727 14949 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:34.934476  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:34.915078 15112 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 12:28:34.911624 15188 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:34.911738 14906 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:34.967693  7810 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:28:34.968163  7810 pending_rounds.cc:85] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 15
I20260709 12:28:34.968417  7810 pending_rounds.cc:107] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:28:34.968669  7810 raft_consensus.cc:2889] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:34.969014 14949 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:34.970002  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.970656  7810 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:34.971508  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.971616  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.972548  7804 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:34.973556  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.974472  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.975080  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:34.953423  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.976096  7803 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:34.903959 15180 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:34.792912 15179 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.794703 15120 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:34.957199  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.958201  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.959131  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.992398  7809 raft_consensus.cc:2410] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:28:34.959945  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:34.993144  7729 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:34.961582  7806 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:34.928216 15181 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.964576  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:34.965447 15143 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:34.966024 15132 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.999104  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.999493  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.999904  7655 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:35.001057  7730 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:35.001513  7730 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:35.001760  7730 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:34.944631 15219 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.002103  7730 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:35.002640  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.003520  7656 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:34.979687  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.998963  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:35.004196  7805 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:35.004484  7653 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:34.956247  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:34.976292  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:34.987192  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.987869  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.004428  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.006867  7658 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:34.988844  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.008279  7660 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 14.
I20260709 12:28:34.931705 15109 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:34.990028  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:34.943940 14935 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:35.003387  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:34.952204 15216 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.962734 15099 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.014309  7730 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:35.014153  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.036155  7661 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:35.036580  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.016005  7654 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:35.022754  7658 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:34.978814 15112 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:35.043005  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:34.983458 15184 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:35.027448  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.048122  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1357) took 1178 ms. Trace:
I20260709 12:28:34.995513 14633 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:35.018173  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.049727  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:35.002921 15143 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:35.044598  7809 raft_consensus.cc:2368] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:34.931473 15144 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.978417  7811 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:35.003120 15081 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.058089  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:35.048831  8411 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:35.059036  7806 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:34.970476 14949 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:34.971414 15121 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:28:35.052878  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.054678  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.024870 15190 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:35.062366  7803 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:34.995388 15054 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:28:35.019035 15220 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:35.065034  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.070854  7807 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:35.048835  7659 rpcz_store.cc:276] 0709 12:28:33.869363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:33.872037 (+  2674us) service_pool.cc:224] Handling call
0709 12:28:33.872424 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:33.872881 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:34.115692 (+242811us) log.cc:844] Serialized 14 byte log entry
0709 12:28:34.352775 (+237083us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:34.366550 (+ 13775us) log.cc:844] Serialized 52 byte log entry
0709 12:28:34.366711 (+   161us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:34.415145 (+ 48434us) log.cc:844] Serialized 14 byte log entry
0709 12:28:35.029276 (+614131us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b180037bdc0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:28:35.029487 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:35.029613 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:35.047733 (+ 18120us) raft_consensus.cc:1685] finished
0709 12:28:35.047775 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:35.048031 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70512,"prepare.queue_time_us":407118,"prepare.run_cpu_time_us":197,"prepare.run_wall_time_us":40151,"raft.queue_time_us":84637,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":4935,"spinlock_wait_cycles":5176192,"thread_start_us":669451,"threads_started":3,"wal-append.queue_time_us":577708}
I20260709 12:28:35.027760 14935 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:35.072450  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.004920 15133 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:28:35.033514 15161 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.077101  7811 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:35.079831  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:35.081013  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 483019us
I20260709 12:28:35.081923  7804 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:35.010358 15132 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.086202  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:35.011157 15181 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:35.095759  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82972.6us
I20260709 12:28:35.088688  7809 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:35.063083 15040 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.092857  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.053584 15229 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.093871  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:35.094023  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.096995  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:35.071106 15190 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.104197  7659 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:35.073122 15099 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:35.109196  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.110052  7660 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:35.113000  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.082981 15066 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:35.117698  7660 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:35.019959 15219 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.106613 15190 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:35.112023 15221 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 12:28:35.114344  7732 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:35.106523  7806 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:35.073969 15081 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:35.141345  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47411.5us
I20260709 12:28:35.193580  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:35.173569  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50934.3us
I20260709 12:28:35.121232 15040 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:35.189582  7805 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:35.189862  7810 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:35.122951 15137 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:35.199857  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 279984us
I20260709 12:28:35.095577 15228 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:28:35.213822  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.210945  7803 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:28:35.200882  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.143190 15144 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.217854  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.218595  7654 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:35.121495 14633 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:28:35.179651 15099 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:35.175566 15133 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:35.210140  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:35.228781  7809 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:35.041997 15216 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.105659 15229 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.236729  8408 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:35.236915  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.245535  8406 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:35.191462 15226 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:35.258318  7626 leader_election.cc:400] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:35.258893  7626 leader_election.cc:403] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:28:35.249637  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.622s (SENT)
I20260709 12:28:35.262048  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:35.123174 14935 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.222147 15196 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:35.251576 15112 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:35.252791 15088 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:35.272429  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 78970.5us
W20260709 12:28:35.272852  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 236094us
I20260709 12:28:35.285315  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.112301 15143 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.288237 15143 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:35.269984  7807 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:35.269157  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.265304 15234 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:28:35.290418 15132 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:35.291188 15132 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:35.283416 15196 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:35.276993  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.292621 15196 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:35.267182 15147 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.261914 15087 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:28:35.221743 15220 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:28:35.300859 15087 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:35.302654 14633 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:35.303604 14633 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:35.304128 14633 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.305402 15229 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.306664 15081 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.306533 15099 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.250866 15162 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:35.259725 15133 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:35.318569 15231 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:28:35.319039 15216 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.294153 15196 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.265417 15184 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
W20260709 12:28:35.317513 15223 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:28:35.317003  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80398.4us
I20260709 12:28:35.053872  7553 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 13 to 16, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:35.335592  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:35.375501  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:35.376034 15224 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:28:35.376494  7553 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 8 to 12, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 12 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:35.353463  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.277220 15179 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:35.361486  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:35.384021 15232 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:28:35.284569 15088 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:35.387650 15088 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:35.388033 15133 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.391654  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.369292 15162 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:35.369838  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.395427  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.282449 15112 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.397106  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:35.372689  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:35.404433  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.311389 15120 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:35.401540  7804 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:35.397588 15146 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.407111 15162 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:28:35.402908 15132 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.410667 15112 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:35.410425 15226 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:35.426375 15188 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.437788 15162 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:35.378645  7553 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 13 to 17, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:35.438443 14633 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:35.432068 15146 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.423172 15229 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:35.433713 15219 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.443184  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.447499 15181 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:35.447608  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50606.6us
I20260709 12:28:35.455734  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:35.449538 15089 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.000s
I20260709 12:28:35.450390  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.497345  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:35.467738  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 195291us
W20260709 12:28:35.461925 15129 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.002s
I20260709 12:28:35.468755  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:35.482005  8412 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:35.464340 15162 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:35.499550  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:35.484086  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.510942  7729 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:35.512864  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.509148  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.453280 15190 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.516494 15179 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:35.517032 15190 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:35.517207 15179 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:35.517908  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 116534us
I20260709 12:28:35.517995 15179 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:35.518065  8413 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 12:28:35.520246  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:35.520953  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.633s (SENT)
I20260709 12:28:35.521356 15196 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:35.523424  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:35.524580 15161 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:35.525255 15161 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:35.526262 15161 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.472180 15229 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:35.493156 15226 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:35.554620  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.553s (SENT)
I20260709 12:28:35.492254 15181 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.566093  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.523814 15196 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:35.564915  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.473287 15154 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:28:35.500715 14906 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:35.470293 15109 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.552562 15229 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:35.541643 15219 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:35.549974 15066 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:35.581849  7806 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:35.573019 15214 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.850s	user 0.004s	sys 0.002s
I20260709 12:28:35.565686 15226 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:35.571359 15129 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:35.573019 15244 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:35.610148  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55584.5us
I20260709 12:28:35.608140 15129 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.616338 15129 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:35.614429  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.622031 15193 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:35.622617 15193 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:35.652593  8415 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:35.628715  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:35.657812  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.643841  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1111) took 1508 ms. Trace:
I20260709 12:28:35.660995  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:35.662002  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.618s (SENT)
W20260709 12:28:35.662899  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48410.9us
I20260709 12:28:35.588544 15066 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.590456 15139 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:35.612556 15180 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.000s
I20260709 12:28:35.659957  8407 rpcz_store.cc:276] 0709 12:28:34.135466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.135847 (+   381us) service_pool.cc:224] Handling call
0709 12:28:35.643522 (+1507675us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54005e0e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:35.643715 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549523712}
I20260709 12:28:35.595927 15230 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:28:35.666667  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53375.5us
I20260709 12:28:35.787293  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.788246 15228 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:35.789137 15228 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:35.790282 15120 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:35.791666  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.793095  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.794952 15221 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.795725 15132 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.797158  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.798538  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:35.799283  8408 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:35.799069 15242 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:35.801303  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.798926  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.732100  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.802933  7805 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:35.803171  7657 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:35.715685  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.807050  7661 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:35.596815 15219 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.674150 15139 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:35.703683  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:35.809147  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.810730  8406 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:35.673854  7553 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 12, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 12 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:35.764165  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.734570 15113 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:35.811874 15113 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:35.812060  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48032.4us
I20260709 12:28:35.665736 15193 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:35.812909  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.814180  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.815325  7805 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:35.703922 15088 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.652062 15188 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:35.679157 15114 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.719s	user 0.008s	sys 0.006s
W20260709 12:28:35.830713  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 96988.7us
W20260709 12:28:35.839929  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.767338 15179 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.724166 14965 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:35.751444  7811 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:28:35.715972  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.861395  7658 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:35.596551 15162 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.638466 15109 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.701107 15180 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.733999 14935 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:35.719828  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.735880 15146 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:35.611598 15226 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:35.652439 15196 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.734510 15054 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:35.787897 15230 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:35.794773  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:35.769227 15249 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.261s	user 0.003s	sys 0.000s
I20260709 12:28:35.574627 15181 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.809685  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.673854 15066 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:35.827373  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:35.892189  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 143358us
W20260709 12:28:35.892853  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65571.3us
I20260709 12:28:35.867390 14965 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:35.666821  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:35.903816  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:35.790870 15229 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:35.831525  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:35.832924  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.836942  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:35.834905  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:35.920358  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 85563.1us
I20260709 12:28:35.921394  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:35.666226 15191 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:35.840988  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:35.841807  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:35.842869  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1477) took 1543 ms. Trace:
I20260709 12:28:35.781508 15040 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:35.855430  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:35.826017 15143 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:35.829909 15228 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:35.861593  7804 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:35.829224 15125 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:35.866339  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.938737  7732 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 14.
I20260709 12:28:35.883080  7733 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:28:35.889977  7808 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:35.891139  7807 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:35.893879  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:35.844972 15190 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:28:35.936407 14594 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:35.941011 15099 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:35.941668 15190 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:35.936668 15040 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:35.922497  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89660.2us
I20260709 12:28:35.899923 15132 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:35.944319  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:35.924465  7661 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:35.925595  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84709.5us
I20260709 12:28:35.924405  8415 rpcz_store.cc:276] 0709 12:28:34.295195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.296243 (+  1048us) service_pool.cc:224] Handling call
0709 12:28:35.652145 (+1355902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:35.665510 (+ 13365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:35.837912 (+172402us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54005e0e50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:35.838072 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3081974656}
I20260709 12:28:35.926362 14987 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:35.941064 14594 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:35.942288 15190 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:35.902716  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 235979us
I20260709 12:28:35.950783  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.949409  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:35.923110 15135 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:35.875475 15139 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:35.957944  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:35.966321  7811 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:35.930011 15143 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:35.930014 15109 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:35.963887  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.968108  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:35.943792 15125 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.950459 15113 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:35.954226 14594 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:35.985240  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.987634  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:35.996461  7729 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:28:35.961436 15135 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.938184 15161 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:28:35.976418 15146 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:35.956919 14987 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:35.956292 15132 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:36.007002  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 113225us
I20260709 12:28:35.999482  7732 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:28:36.013216  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75162.9us
I20260709 12:28:35.985924 15139 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:35.978713 15143 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.000880  7735 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:35.979455 15109 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.014946  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:36.000006  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53117.3us
W20260709 12:28:35.998687 15112 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
I20260709 12:28:35.994781 15244 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:35.994506 15125 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.031040  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:36.031044  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3385) took 1103 ms. Trace:
I20260709 12:28:36.036734  7808 rpcz_store.cc:276] 0709 12:28:34.928046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.928225 (+   179us) service_pool.cc:224] Handling call
0709 12:28:36.030842 (+1102617us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b5400011654. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:36.030978 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208224640}
I20260709 12:28:36.007238 15135 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:36.009840 15219 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:36.051471  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45032.7us
I20260709 12:28:36.028698 15109 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.053599  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.031279 15244 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:36.033232 15112 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.067112 14987 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.067735 15244 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:36.069334 15112 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:36.036243 15221 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:36.024524 15230 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:28:36.040124 15146 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:36.077916  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.090s (SENT)
I20260709 12:28:36.079552  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:36.077605  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:36.045017 15184 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.064348 15040 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:36.075307 15120 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:36.126130  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 98152.6us
W20260709 12:28:36.124915  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47041.7us
I20260709 12:28:36.073665 15179 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:36.059285 15032 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:28:36.129537  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.125525 15221 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:36.156471 15260 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 12:28:36.158775 15270 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:28:36.128053 15120 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:36.177747  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48290.5us
I20260709 12:28:36.179062  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:36.184656  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.194s (SENT)
I20260709 12:28:36.185513  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.133739 15180 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:36.064518 15190 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.184026  7700 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:36.100523 15273 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:28:36.082919 15099 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.111s
W20260709 12:28:36.258813  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 181262us
W20260709 12:28:36.266098  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 241972us
W20260709 12:28:36.234728  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 181193us
W20260709 12:28:36.267954  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47899.7us
I20260709 12:28:36.209084  8407 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:28:36.206070  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 389970us
I20260709 12:28:36.282198  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:36.231165  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:36.285851  7660 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:36.194830 15121 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:36.292003 15121 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.292603 15121 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.220557 15272 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:36.294269 15272 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.299902  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.188373 15120 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:36.195143 15257 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
I20260709 12:28:36.195497 15271 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:36.255712 15216 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:36.192896 15219 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:36.244164 15146 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.133318 15132 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.101729 15184 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.220256 15191 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.231602 15221 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:28:36.327963  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.244335 15279 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:36.241485 15113 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:36.339860  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 138850us
I20260709 12:28:36.280356 15161 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:36.343410  7660 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:36.347946  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:36.348994  7812 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:36.350802  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:36.316984 14987 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:36.356961  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89504.4us
W20260709 12:28:36.357340  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57578.8us
I20260709 12:28:36.359750  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:36.357688  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.316063 15273 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:28:36.377489  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:36.339821 15221 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:36.365896  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97591.5us
I20260709 12:28:36.339733 15181 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:36.348910 15099 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:36.397339  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.364527 15032 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:36.401806  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:36.410826  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:36.411320  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.418200  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:36.381063 15221 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:36.419462  7660 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:36.419764  7732 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:36.392860 15133 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:36.364199 15216 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:28:36.351635 15113 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.344702 15191 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.423517  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:36.440585  7656 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:36.420192  7732 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:36.386773 15099 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:36.354322 15161 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.377385 15272 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:36.345355 15279 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.410161 15243 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:36.442400  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.365962 14987 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.419570 15112 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:36.424275 15264 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:28:36.435273 14906 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:36.430155 15145 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:36.426060 15133 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:36.382812 15281 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:28:36.437036 15285 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:36.484834  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60350.6us
I20260709 12:28:36.485164  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:36.459851  7732 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.486088  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.422966 15180 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:28:36.445556 15215 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:36.468715 15275 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.237s	user 0.000s	sys 0.002s
I20260709 12:28:36.487090  7732 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:36.486099  7807 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:36.468747 15112 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.468312 15279 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:36.476781 15145 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.482862 15133 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:36.489813 15285 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:36.416250  7553 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 12, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 12 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:36.504153 15278 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:36.504284 15099 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.512480 15145 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:36.503935 15196 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.546978  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:36.505251 15112 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.552078  7807 raft_consensus.cc:606] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:36.560456  7807 raft_consensus.cc:614] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:36.550753  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 283857us
W20260709 12:28:36.557142 15226 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.665s	user 0.014s	sys 0.001s
I20260709 12:28:36.579460  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:36.584355  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:36.572034 15099 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:36.606954  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100366us
I20260709 12:28:36.605235  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:36.558871 15278 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:36.562387 14906 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:36.598824 15244 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.530s	user 0.015s	sys 0.000s
I20260709 12:28:36.617259  7731 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:36.565498 15184 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:36.564352 15196 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.568620 15226 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:36.736064 15263 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:28:36.738430  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:36.740072  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:36.741439  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:36.630599 15129 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.011s	user 0.017s	sys 0.000s
I20260709 12:28:36.757823 15113 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:36.768885 15284 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 12:28:36.771487 15229 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:36.787331 15290 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:28:36.790118 14987 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:36.791837 15294 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:28:36.794478  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:36.796347  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:36.798026  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:36.810055 15125 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.755s	user 0.002s	sys 0.009s
I20260709 12:28:36.721848  7553 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:36.837061 15139 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.810s	user 0.013s	sys 0.000s
I20260709 12:28:36.845638  7811 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:36.847772  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:36.848724  7731 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:36.849195  7812 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:36.849952  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:36.850446  7809 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:36.850864  7737 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:36.851974  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:36.852886  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:36.604100 15304 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.149s
W20260709 12:28:36.859138  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.869536  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:36.628026 15244 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.745280 15161 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:36.829658  7553 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 5 to 12, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:36.884188  7733 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:36.894289 14965 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:36.871305 15191 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:36.944685  7731 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:36.899305 15195 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:36.899757 15184 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.810771 15125 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:36.847927 15129 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:36.898437 15143 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.000s
I20260709 12:28:36.982110  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:36.989041  7733 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:36.990384  7661 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:37.003211  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:36.985621 15121 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.692s	user 0.014s	sys 0.002s
W20260709 12:28:36.986819 15066 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.084s	user 0.011s	sys 0.004s
I20260709 12:28:37.008265  7808 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:36.987702 15193 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.170s	user 0.006s	sys 0.010s
I20260709 12:28:36.623090 15285 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:37.015460  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.992065 14965 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:36.997879 15143 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.008380 15305 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:37.025336 14630 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.876917 14906 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.041033 15129 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:36.991603 15184 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:37.031431 15267 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.835s	user 0.004s	sys 0.002s
I20260709 12:28:36.986160 15195 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.044018 15271 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:37.064090 15120 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.740s	user 0.014s	sys 0.000s
I20260709 12:28:36.882184 15278 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.067471  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:37.087563 14594 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.080s	user 0.015s	sys 0.002s
I20260709 12:28:37.109236  7736 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:37.207322  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.014s	sys 0.019s
I20260709 12:28:37.019654 15066 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.098958 15285 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.046876 15193 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:37.145817  8377 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.056s	sys 0.043s
W20260709 12:28:37.211432  8377 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.057s	sys 0.043s
I20260709 12:28:37.212399  7737 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:37.018563 15121 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.105685 15271 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:37.098188 15305 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.086558 14630 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.215071 15311 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:37.215978 15311 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.218412 15305 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.112340 15120 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.127528 14594 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.154872 15226 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.155318 15196 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.177554  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.175753 15125 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.221210 15271 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:37.254293 15088 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.210844 15244 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.203938  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:37.207736  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:37.208026  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.015s	sys 0.019s
I20260709 12:28:37.213218  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:37.261353  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57516.2us
W20260709 12:28:37.228648  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 786282us
I20260709 12:28:37.261536  8407 raft_consensus.cc:2368] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:28:37.264218  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:37.230101 15109 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.151s	user 0.005s	sys 0.010s
W20260709 12:28:37.230235 15112 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.003s
W20260709 12:28:37.230338  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.210755  7553 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 11 to 15, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 15 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:37.231194  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:37.228657 15277 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:37.239622  8415 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:37.240460  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3401) took 1445 ms. Trace:
I20260709 12:28:37.243536 15113 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:37.243888 15312 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:28:37.244827 15180 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.245352 15181 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.245980 14594 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.238682 15121 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.243604 15066 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.312924 15040 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.321389 15181 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.322352 15040 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.323185 15120 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:37.324827 15221 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.887s	user 0.011s	sys 0.004s
I20260709 12:28:37.325459 15221 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:37.156662  7732 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.001s
I20260709 12:28:37.328104  7732 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:28:37.328577  7732 pending_rounds.cc:85] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:28:37.328802  7732 pending_rounds.cc:107] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:28:37.329028  7732 raft_consensus.cc:2889] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:37.330462  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.268164  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.332012 15181 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:37.337069 15282 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.947s	user 0.017s	sys 0.014s
I20260709 12:28:37.271509  7733 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:37.273671  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:37.268262  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.344161  7737 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:37.274803  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.281605 14633 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:37.281868 15215 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:37.236923 15193 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:37.283351 15145 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.715s	user 0.014s	sys 0.000s
I20260709 12:28:37.280087 15109 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.265786 15054 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:37.288908  7804 rpcz_store.cc:276] 0709 12:28:35.794587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.796907 (+  2320us) service_pool.cc:224] Handling call
0709 12:28:37.240277 (+1443370us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:37.240397 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146372480}
I20260709 12:28:37.222705 14630 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:37.269892 15135 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.199s	user 0.010s	sys 0.006s
I20260709 12:28:37.346083 15040 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.311992  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.314465 15180 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.293191 15143 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:37.337961 15133 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.815s	user 0.013s	sys 0.001s
W20260709 12:28:37.339877 15099 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.003s
I20260709 12:28:37.309719 15113 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.341727 15195 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.266772  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:37.279304  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.346856  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.389305  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.288714 15112 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.344802  7729 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:37.347851 14633 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.363018  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.363556 15221 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.363976 15243 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:37.361274 14965 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.369165  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.405299  8407 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:37.380079  7736 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:37.384850  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.361353 15285 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:37.386989  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:37.390388  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 123718us
I20260709 12:28:37.390904  7735 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:28:37.371605 15129 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:37.368778 15145 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.425596  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.400480  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.427275  7811 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:37.401646  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.400646  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.428252  7804 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:28:37.407202  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.398867  7732 raft_consensus.cc:2368] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17.
I20260709 12:28:37.414866  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:37.379966 15054 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.421032  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.399917 14633 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:37.365114 15271 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.404081 15243 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:37.427403  7803 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:28:37.427723  7737 raft_consensus.cc:2368] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15.
I20260709 12:28:37.433184  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.434607  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.405999  7809 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:37.428896  7700 leader_election.cc:400] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:28:37.436309  7700 leader_election.cc:403] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:28:37.437242  7733 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:37.398265 14630 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:37.446794 14630 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:37.381238 15135 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.449798  7806 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:37.451067  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.452126  7812 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:37.454726  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.456822  7812 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:37.457120  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:37.456461  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:37.458077  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.458676 15271 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.458853 14594 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:37.438329  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.445955  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.448163 14630 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.455614  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.438351 15143 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:37.445016  7737 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:37.385569 15133 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.459422  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.462872  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.464697  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:37.438820  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.466078  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:37.467381  8407 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:37.467432  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123993us
I20260709 12:28:37.470500  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.426781 15195 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:37.473734  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.473909  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.469008  7735 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:37.475700  7736 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:37.475042  7737 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:37.478466  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.480856  7732 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:28:37.481415  7729 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:37.426556 15285 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.459422 14594 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.488144  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.463249 15143 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.487092  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.476547  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.490068  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.491211  7732 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:37.410530 14965 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.469413 15099 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:37.483995 15283 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.025s	sys 0.031s
I20260709 12:28:37.493032  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.478469 15271 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:37.493067  7735 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:28:37.495201  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.495906  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.477240 15195 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:37.504992  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.484582 15221 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
W20260709 12:28:37.499079  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 234927us
I20260709 12:28:37.510445  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:37.508285  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 238349us
I20260709 12:28:37.493115 14594 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.492053 15143 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:37.516372  8407 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:37.516963  7733 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:37.530437  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:37.507611 15283 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.025s	sys 0.031s
I20260709 12:28:37.509316 15195 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.535774  8413 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:37.503712 15271 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:37.502738 15099 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.542106  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:37.560598  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.560664  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.562232  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.560165  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:37.494861 15243 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.568092  7733 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:37.563505  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.571095  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.546665 15271 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:37.546636 15109 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.577088  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.579375  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:37.579548  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.581633  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:37.581900  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.589567  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:37.590642  7658 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:37.599398  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.600203  7657 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:28:37.571396 15318 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:28:37.616434  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.616515  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.617537  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.617733  7654 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.618165  7655 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:28:37.618907  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:37.617568  7656 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:37.620828  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.622438  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:37.618696  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:37.574002 15180 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.574190 15145 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.627619  7659 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:37.627272  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:37.635828  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.579941 15112 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.606897 15146 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:37.602136 15215 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:37.640195  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.613516 15243 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:37.641153 15322 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:37.613070 15181 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:28:37.648087 15317 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:28:37.659364  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:37.651151  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1377) took 1464 ms. Trace:
I20260709 12:28:37.638716 15054 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.662469  7661 rpcz_store.cc:276] 0709 12:28:36.187372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.191412 (+  4040us) service_pool.cc:224] Handling call
0709 12:28:36.192079 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.990960 (+798881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.650868 (+659908us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b5400398ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:37.651077 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835244288}
W20260709 12:28:37.652122  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:37.653864  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.675030 15162 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:37.651937  7660 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.634234 15040 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:28:37.655568 15145 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:37.669970  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.687469  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:37.691404  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.707988  7661 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:37.708272  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:37.651461 15133 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.692602  7656 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:37.694557 15088 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:37.695514  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.713129  7733 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:37.657609 15327 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.716629  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:37.729938  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:37.732125  7656 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:37.671948 15181 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:37.750144  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:37.751078  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:37.752746  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.657874 15125 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:28:37.688076 15322 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:37.702832 15329 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:37.771374  7657 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:37.764402  7660 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:37.793926  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:37.764446  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:37.776186  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:37.777529  7661 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:37.712484 15285 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:37.778067  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 135853us
I20260709 12:28:37.751158 15088 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.810257  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.810575  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.811408  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:37.811704  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.788681  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3399) took 1994 ms. Trace:
I20260709 12:28:37.751565 15145 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:37.719132 14965 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.808964  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.807598  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:37.812378  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61411.4us
I20260709 12:28:37.815730  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.812880  7810 rpcz_store.cc:276] 0709 12:28:35.793925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.795786 (+  1861us) service_pool.cc:224] Handling call
0709 12:28:35.796177 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:35.945193 (+149016us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:35.975841 (+ 30648us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.059764 (+ 83923us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.232914 (+173150us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b180043fea0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:36.233186 (+   272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:36.233284 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:36.233362 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.788405 (+1555043us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:37.788467 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:37.788612 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":261582,"prepare.queue_time_us":38716,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":179116,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":19718,"spinlock_wait_cycles":418764928,"thread_start_us":83079,"threads_started":1,"wal-append.queue_time_us":158328,"wal-append.run_cpu_time_us":4305,"wal-append.run_wall_time_us":1416731}
I20260709 12:28:37.819866  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.821148  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.822641  7627 leader_election.cc:400] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17.
I20260709 12:28:37.823020  7627 leader_election.cc:403] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:37.817797  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:37.825623  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.827725  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:37.828161  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:37.753527 15327 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:37.840896  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.837945  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.816206 15146 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:37.789968 15329 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.851194  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:37.852351  7810 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:37.853835  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:37.819455 15099 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:37.863360  7810 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:37.864471  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:37.802098 15322 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.831142 15332 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:37.880131  7810 raft_consensus.cc:606] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:37.851516 15146 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.902321  7654 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:37.849851 15333 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.865510 15109 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:37.886314 15328 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:28:37.886188 15336 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:37.865984 15334 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:37.881526 15180 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.890564 15146 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:37.824330 15181 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:37.927924  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99856.4us
I20260709 12:28:37.904367 15133 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:37.929828  7627 leader_election.cc:400] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15.
I20260709 12:28:37.935930  7627 leader_election.cc:403] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:28:37.918442 15320 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:28:37.904820 15099 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:37.917491 15325 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:28:37.916293 15337 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:37.889782 15145 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.964478  7655 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:37.918889 15109 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.974915  8406 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:37.930399 15180 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:37.976531 15087 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:37.978538 15180 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:37.943938 15340 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:37.983731  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2870) took 1252 ms. Trace:
I20260709 12:28:37.984187  7803 rpcz_store.cc:276] 0709 12:28:36.731269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.231346 (+500077us) service_pool.cc:224] Handling call
0709 12:28:37.231773 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.433440 (+201667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.983482 (+550042us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54001b1f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:37.983674 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449178496}
W20260709 12:28:37.971848 15330 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 12:28:37.993947  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:37.981235 14633 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.003s
I20260709 12:28:37.948037 15099 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.925834 15336 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:37.941442 15133 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.017427 15133 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.018918  7731 raft_consensus.cc:2410] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:28:37.957522 15331 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:38.005034  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:38.026899  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97286.1us
I20260709 12:28:38.028467  7728 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:38.028689  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.033035 15311 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.033723 15143 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.035251  7812 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:38.007030 14633 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.037560 14633 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:38.038945 15336 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.042171 15109 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.052627  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:38.052919  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.053540  8407 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:38.052949  7805 raft_consensus.cc:2368] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:28:38.055620  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:38.057076  7728 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:38.005059  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.012290 15180 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:38.019428  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.003521 15088 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:37.992285 15234 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.968525 15332 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17.
I20260709 12:28:37.993870 15341 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:38.068557  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.073542  7654 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:38.066874  7660 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:38.072564  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.076843  8413 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:38.070222  8407 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:28:38.080799  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52182.2us
I20260709 12:28:38.082221  7653 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:38.083213  8412 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 18: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:38.087183  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.068243 15162 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:38.071640 15234 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.089159 15162 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.096347  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.046598 15329 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.097947  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.104851  7655 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:38.105141  7728 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:38.105720  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.044404 15327 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.085925 15336 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:38.092326 15143 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.110322  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.111680 15109 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:38.115597 15162 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:38.118214 15195 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.008s
I20260709 12:28:38.119047 14633 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.119869  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.114957 15334 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.082844 15341 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.124897  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:38.136091  8413 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:38.074893 15180 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.127920  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:38.108737 14594 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.575s	user 0.018s	sys 0.000s
I20260709 12:28:38.132439  7736 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:38.132107  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:38.151301  7728 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:28:38.127650  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.152446  7660 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:28:38.143378 14633 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.142714  7655 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:38.147284  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.029488 15331 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.118117 15234 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:38.164927  7655 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:38.163094  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.134038 15349 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:38.103595 15146 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.149446 15099 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.172271  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.133280 15195 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:38.167047 15271 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.005s
I20260709 12:28:38.153649 15346 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:38.157040 15345 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:38.179730  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.144114 15334 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.168655 15349 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.134409 15109 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.197840 15180 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.177152  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.157397 14594 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.168195 15336 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:38.201807  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.158108 15133 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:28:38.170962 15327 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:38.177042  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:38.203439  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.183485 15271 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.164521 15243 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:28:38.208333 15331 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.210711 15087 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 14. Current term is 15.
I20260709 12:28:38.242162 15195 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.216853 14633 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:38.217825  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.247942 15334 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:38.248411  7655 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:38.249716  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.250581  7655 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:38.250739  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:38.251556  7804 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20.
I20260709 12:28:38.235065 15109 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.221300 15146 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.256043 15333 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:38.255920  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3408) took 2188 ms (client timeout 2613 ms). Trace:
W20260709 12:28:38.258311  7808 rpcz_store.cc:269] 0709 12:28:36.071024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.079205 (+  8181us) service_pool.cc:224] Handling call
0709 12:28:38.255632 (+2176427us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003c4bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.255814 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115304192}
I20260709 12:28:38.259104  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.259948  7808 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:38.261364  7808 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:28:38.262706  7808 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:38.263698  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.222359  8413 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
W20260709 12:28:38.228266  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58122.5us
I20260709 12:28:38.264955 15242 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:28:38.231354  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.265152 15162 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:28:38.232980 15184 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.232810  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.270776 15281 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:38.271921 15281 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:38.270104  7656 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:38.272416 15281 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:38.273754 15281 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.235505 15346 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:38.275496 15346 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:38.275966 15346 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:38.268445  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.240031  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.238003  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1125) took 1031 ms. Trace:
I20260709 12:28:38.243758  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.280652 15215 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:38.280793  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.281265 15215 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:38.281718  7661 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:38.281785  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.282193 15215 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.245723 14594 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.282773  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:38.239111  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2868) took 1614 ms (client timeout 1734 ms). Trace:
I20260709 12:28:38.243494 15341 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:38.283383  7806 rpcz_store.cc:269] 0709 12:28:36.624828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.741183 (+116355us) service_pool.cc:224] Handling call
0709 12:28:38.238887 (+1497704us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54003c50d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.239045 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626661632}
I20260709 12:28:38.235047 15271 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.202680 15349 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:38.285055  7806 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:38.286010  7662 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:28:38.286635  7806 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:38.288341 15341 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:38.288443  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2863) took 1675 ms. Trace:
I20260709 12:28:38.288977  7811 rpcz_store.cc:276] 0709 12:28:36.613153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.846751 (+233598us) service_pool.cc:224] Handling call
0709 12:28:36.847078 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:37.428045 (+580967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.288181 (+860136us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.288378 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333656448}
I20260709 12:28:38.246639 15113 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.271083  7654 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:38.294288 15162 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:38.282963  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.284650  7659 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:38.280022  8408 rpcz_store.cc:276] 0709 12:28:37.206610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.206795 (+   185us) service_pool.cc:224] Handling call
0709 12:28:37.207256 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.223956 (+ 16700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:37.224008 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:37.232139 (+  8131us) log.cc:844] Serialized 26 byte log entry
0709 12:28:37.257388 (+ 25249us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b18003f02e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:37.521556 (+264168us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:37.521600 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.237655 (+716055us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.237732 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:38.237783 (+    51us) raft_consensus.cc:1685] finished
0709 12:28:38.237796 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.237936 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36043,"raft.queue_time_us":13987,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":114,"spinlock_wait_cycles":73076352,"thread_start_us":263949,"threads_started":1,"wal-append.queue_time_us":410635}
I20260709 12:28:38.308153  7653 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:38.308032  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.290977  7627 leader_election.cc:400] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:38.310009  7627 leader_election.cc:403] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:38.324019  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.329914  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1215) took 1003 ms. Trace:
I20260709 12:28:38.328604  7655 raft_consensus.cc:2410] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:38.330441  7812 rpcz_store.cc:276] 0709 12:28:37.326329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.037479 (+711150us) service_pool.cc:224] Handling call
0709 12:28:38.037872 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:38.038088 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:38.167438 (+129350us) log.cc:844] Serialized 14 byte log entry
0709 12:28:38.329451 (+162013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:38.329513 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:38.329562 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.329643 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.329833 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18114,"prepare.queue_time_us":167222,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"thread_start_us":271493,"threads_started":2,"wal-append.queue_time_us":99884}
I20260709 12:28:38.309413 15162 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:38.344214  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:38.349144  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.313634 15341 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:38.396222  7655 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:38.405880 15321 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.757s	user 0.003s	sys 0.000s
I20260709 12:28:38.413872 15346 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.359615 15129 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:38.360019  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3409) took 1117 ms. Trace:
I20260709 12:28:38.360682 15120 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:38.363197  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:38.340621 15054 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.342389 15333 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:38.342849 15271 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.330685 15221 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:28:38.355868  7656 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:38.384629  8411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1490) took 1172 ms (client timeout 1524 ms). Trace:
I20260709 12:28:38.388818 15349 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.345917  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.390436  7804 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:38.391719 15162 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.393277 15228 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:38.357136 14906 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:38.414415 15135 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:38.416604  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3465) took 1435 ms. Trace:
I20260709 12:28:38.417902  7729 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:38.358073 15146 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:38.419611  7661 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [18.11-19.12]
W20260709 12:28:38.422341 15291 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.961s	user 0.006s	sys 0.040s
I20260709 12:28:38.425480 15215 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:38.427248  7657 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 12:28:38.428467 15301 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.425545 15274 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:28:38.429802  7655 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:38.430637  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1123) took 2223 ms. Trace:
I20260709 12:28:38.430807  7654 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:28:38.431947  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.436961  7777 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.435602  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:38.538407  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.535198  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.779964  7662 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:38.780716  7729 pending_rounds.cc:85] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
W20260709 12:28:38.457752  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3410) took 1190 ms (client timeout 1580 ms). Trace:
W20260709 12:28:38.782584  7805 rpcz_store.cc:269] 0709 12:28:37.267485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.267654 (+   169us) service_pool.cc:224] Handling call
0709 12:28:38.444879 (+1177225us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b540042a754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.457652 (+ 12773us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636268288}
I20260709 12:28:38.535338  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.569459  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:38.781608  8415 rpcz_store.cc:276] 0709 12:28:36.207271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.207464 (+   193us) service_pool.cc:224] Handling call
0709 12:28:36.207938 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.288546 (+1080608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.430227 (+1141681us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400691254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.430528 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371962112}
I20260709 12:28:38.540565  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1127) took 1107 ms. Trace:
I20260709 12:28:38.786865  8406 rpcz_store.cc:276] 0709 12:28:37.432925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.445803 (+ 12878us) service_pool.cc:224] Handling call
0709 12:28:37.446181 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:37.975428 (+529247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.540275 (+564847us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.540491 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215450880}
W20260709 12:28:38.706646  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 157652us
I20260709 12:28:38.511015  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50788 (request call id 49) took 1103 ms. Trace:
I20260709 12:28:38.788482  7810 rpcz_store.cc:276] 0709 12:28:37.370139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.864314 (+494175us) service_pool.cc:224] Handling call
0709 12:28:38.473498 (+609184us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54002281a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e9f520 00007fc001f1c652 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.473874 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110387584}
I20260709 12:28:38.488432  8413 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:38.489357  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.488932  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:38.765223  7775 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.009s	sys 0.016s
I20260709 12:28:38.783138  7731 rpcz_store.cc:276] 0709 12:28:36.981621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.981790 (+   169us) service_pool.cc:224] Handling call
0709 12:28:38.416355 (+1434565us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001c5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.416534 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371246464}
W20260709 12:28:38.799058  7775 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.009s	sys 0.017s
W20260709 12:28:38.800889  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 600734us
I20260709 12:28:38.781283  7729 pending_rounds.cc:107] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:38.806684  7729 raft_consensus.cc:2889] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:38.793053  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:38.446386  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 129913us
I20260709 12:28:38.786271  8407 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:28:38.809331  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.811111  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:38.783636  7809 rpcz_store.cc:276] 0709 12:28:37.243003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.287659 (+ 44656us) service_pool.cc:224] Handling call
0709 12:28:37.317087 (+ 29428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:37.435853 (+118766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.359650 (+923797us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b54003737d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:38.359924 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203620608}
I20260709 12:28:38.473258  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1222) took 1059 ms. Trace:
I20260709 12:28:38.815071  7803 rpcz_store.cc:276] 0709 12:28:37.414419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.265858 (+851439us) service_pool.cc:224] Handling call
0709 12:28:38.266236 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:38.266478 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:38.291508 (+ 25030us) log.cc:844] Serialized 14 byte log entry
0709 12:28:38.472706 (+181198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:38.472797 (+    91us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:38.472868 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.472977 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.473169 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25361,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":136,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":27497,"thread_start_us":180702,"threads_started":1}
W20260709 12:28:38.816602  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 379755us
W20260709 12:28:38.782891  8411 rpcz_store.cc:269] 0709 12:28:37.212712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.212905 (+   193us) service_pool.cc:224] Handling call
0709 12:28:38.384291 (+1171386us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.384511 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448933248}
I20260709 12:28:38.817538  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.794726  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:38.514712  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:38.828150  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:38.833420  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:38.833282  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:38.835536  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46306.2us
W20260709 12:28:38.835139  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 489314us
I20260709 12:28:38.841114  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:38.847815  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:38.838668  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:38.861526  7657 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 15: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:38.861644  7654 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:38.833885 15228 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.869374 15311 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:38.870219 15162 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:38.963598 15311 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:38.871912 15120 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:38.981748 15311 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:38.981884 15120 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.834022 15301 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:38.985035 15301 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.985888 15301 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:38.986291 15301 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:38.875703  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 67139.7us
I20260709 12:28:38.877445 15346 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:38.990388 15346 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:38.728473 15329 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.150s
I20260709 12:28:38.933496 15242 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.878945  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:38.880834 14594 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:38.991930 15242 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.993691 15120 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.994469 15120 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:38.994694 15364 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:38.995729 15364 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:38.996362 15364 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:38.875737  7660 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:38.890018  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1217) took 1563 ms. Trace:
I20260709 12:28:38.866760 15146 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:38.901299 15285 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:38.904951 15281 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:38.815173 15269 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.908413 15125 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:38.921130  7662 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:38.927606 14906 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:39.058255  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.064288  7812 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:38.826196 15129 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:38.928577 15361 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:38.852465 15112 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:38.867551 15135 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:38.965746 15365 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:38.871153 15221 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:38.967859  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 129288us
I20260709 12:28:39.077636  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:38.975386 15228 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:38.979779  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:38.950850 15196 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:28:38.997948  7804 rpcz_store.cc:276] 0709 12:28:37.326828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.257404 (+930576us) service_pool.cc:224] Handling call
0709 12:28:38.390148 (+132744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:38.784324 (+394176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.889699 (+105375us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003737d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:38.889912 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304195328}
I20260709 12:28:38.880173 15184 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
W20260709 12:28:38.999608  7701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.007s	sys 0.006s
I20260709 12:28:39.096272 15221 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.065258 15120 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.011960  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:39.069898 15361 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:39.013547  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:39.016637  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.112619  7736 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:39.019129  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:39.114122  7735 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:39.029898  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.031158  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3336) took 1652 ms. Trace:
I20260709 12:28:39.032378  7805 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:39.032440  7655 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:39.043689  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.016952 15285 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:39.090859 15360 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.055966 15269 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:38.987207 15301 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:39.060012  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0332557780002389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:28:38.843192 15271 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.067396  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.068751  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:39.025224 15273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.599s	user 0.032s	sys 0.064s
I20260709 12:28:38.843303 15054 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.156818  7731 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:39.147441  8380 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.680s (SENT) [suppressed 24 similar messages]
I20260709 12:28:39.160071  7810 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:39.161952  8411 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:39.167138  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.168175  7805 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:39.137154 15221 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:39.103951  7701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.007s	sys 0.006s
I20260709 12:28:39.108232  7737 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:39.012318  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.063823 14906 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:39.076754 15311 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.055307 15364 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.095253 14987 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:39.093995 15125 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:39.253830  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:39.117316  7729 rpcz_store.cc:276] 0709 12:28:37.378448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.492216 (+113768us) service_pool.cc:224] Handling call
0709 12:28:38.417485 (+925269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:38.849069 (+431584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.024860 (+175791us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54001c5cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:39.031038 (+  6178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127142144}
I20260709 12:28:39.255167  8415 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:28:38.999903  7626 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.004s	sys 0.008s
W20260709 12:28:39.257212  7626 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.005s	sys 0.008s
I20260709 12:28:39.114885 15120 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:39.258023  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 900098us
I20260709 12:28:39.136932 15162 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.135406  7734 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:39.277472  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 874183us
I20260709 12:28:39.095872 15112 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:39.100581 15135 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:39.217793  7732 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:39.086491  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.198305  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.280050  8379 leader_election.cc:400] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20.
W20260709 12:28:39.154963 15273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.746s	user 0.032s	sys 0.065s
W20260709 12:28:39.161888 15099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.055s	sys 0.094s
I20260709 12:28:39.225129  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:39.220429 15301 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.229583  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:39.237035  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:39.238195  7653 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:39.238121  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:39.239862  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:39.241465  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:39.134295  7730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.004s	sys 0.008s
W20260709 12:28:39.310330  7730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.004s	sys 0.008s
I20260709 12:28:39.310417  7657 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:39.263489  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 446009us
I20260709 12:28:39.265267  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.267279  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.268607  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:39.255028  7655 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:39.129364 15184 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:28:39.275465  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.338699  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.340044  7736 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:39.341876  7810 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:39.300896 15099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.058s	sys 0.101s
W20260709 12:28:39.351424  7659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.007s	sys 0.013s
W20260709 12:28:39.355726  7659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.007s	sys 0.013s
I20260709 12:28:39.280548  8379 leader_election.cc:403] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:28:39.226979  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.226172 15346 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
W20260709 12:28:39.360842  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81342.6us
I20260709 12:28:39.315590  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.199935 15040 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.302160 15242 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.136225 15360 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:39.363758  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48304.8us
I20260709 12:28:39.337162 15364 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:39.368760  7732 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:39.312777 15273 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:39.306953 14987 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:39.330542  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.376333  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.387401  7736 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:39.196836 15146 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.391105  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.393354  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:39.398427  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:39.402444  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.361768 15274 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.406646  7809 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:28:39.403074  7737 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:39.397724  7803 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:39.361610 15341 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:39.370074  7810 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:39.412801  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:39.387933 15099 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:39.418212  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52993us
I20260709 12:28:39.375383 15333 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:39.422258  7731 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:39.325963 15162 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.290885 15361 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.395462  7736 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:28:39.436846  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.412917 15145 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:39.376367 15040 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:39.458644  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.013s
W20260709 12:28:39.461041  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.005s	sys 0.013s
I20260709 12:28:39.433646 15099 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.465194  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.465310 15099 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.470242 15215 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:39.470839 15215 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.471210 15215 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:39.475382 15143 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:39.475940 15371 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:39.477759 15145 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:39.404336 14987 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.487836 15184 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:39.488456 15184 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:39.353489 14906 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.489519 15054 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.455915  7736 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.501608  7736 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:39.502276  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:39.461515  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 597916us
I20260709 12:28:39.397794 15273 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.389544 15135 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.396358 15125 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.508864  7803 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:39.412051 15112 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.493480 15361 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.505069  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.495398 15311 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.533301 15120 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.533646 15349 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.535000 15244 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:39.539433  8405 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:39.539757  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:39.545785  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3459) took 2691 ms. Trace:
I20260709 12:28:39.546469  7730 rpcz_store.cc:276] 0709 12:28:36.854925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.859439 (+  4514us) service_pool.cc:224] Handling call
0709 12:28:36.859812 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:36.860220 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.860253 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:36.892796 (+ 32543us) log.cc:844] Serialized 52 byte log entry
0709 12:28:38.551997 (+1659201us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b1800345fa0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:39.310896 (+758899us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:39.310938 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.311005 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:39.545306 (+234301us) raft_consensus.cc:1685] finished
0709 12:28:39.545340 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:39.545689 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":17622,"spinlock_wait_cycles":17455360,"thread_start_us":758669,"threads_started":1,"wal-append.queue_time_us":2555111}
I20260709 12:28:39.529049  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.516225 15193 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:39.564456  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:39.570595  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:39.489797 15184 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.600749 15184 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:39.539472 14594 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:39.576717 15360 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.588820 15273 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.589515 15271 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.604104 14594 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:39.508944 15054 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
W20260709 12:28:39.609511  8412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.023s	sys 0.037s
W20260709 12:28:39.610685  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81868.6us
W20260709 12:28:39.628010  8412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.023s	sys 0.038s
I20260709 12:28:39.633493  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.638742  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.562827 15244 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:28:39.653900  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:39.672881  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.003s	sys 0.022s
W20260709 12:28:39.673815  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.004s	sys 0.022s
W20260709 12:28:39.674474  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 879781us
I20260709 12:28:39.677130  7736 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 18. (term mismatch)
I20260709 12:28:39.718390  7809 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:39.700856  7812 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:39.657421 14594 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.639240 15360 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:39.720489 15120 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:39.677397 15365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.025s
W20260709 12:28:39.721352 15365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.000s	sys 0.026s
I20260709 12:28:39.721021 15360 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.677708  7736 pending_rounds.cc:85] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 17
I20260709 12:28:39.660159 15368 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:39.722790  7736 pending_rounds.cc:107] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 20.18
I20260709 12:28:39.723294  7736 raft_consensus.cc:2889] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:39.701699  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:39.724888  7810 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.685657  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:39.729558  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44214.1us
I20260709 12:28:39.730216 15365 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:39.720992 14594 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.737457  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3060) took 1343 ms. Trace:
I20260709 12:28:39.738101  7659 rpcz_store.cc:276] 0709 12:28:38.394406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.397236 (+  2830us) service_pool.cc:224] Handling call
0709 12:28:38.397584 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:38.397814 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:38.397839 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:38.398344 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:28:39.356683 (+958339us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:39.356751 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.356868 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:39.737145 (+380277us) raft_consensus.cc:1685] finished
0709 12:28:39.737179 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:39.737376 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":957799,"threads_started":1,"wal-append.queue_time_us":1234382}
I20260709 12:28:39.742751  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:39.743595  7737 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:39.744976 15285 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.745752  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:39.746297 15285 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:39.747018 15120 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:39.747545 15120 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:39.748101 15120 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:39.749226 15298 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:28:39.750588 15329 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:39.751108 15329 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:39.752017 14987 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.752140 15329 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.752903 14987 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:39.753471 15329 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:39.756804 15269 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:39.771816  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.004s	sys 0.023s
W20260709 12:28:39.772202  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.004s	sys 0.023s
I20260709 12:28:39.773779  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:39.774495  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.779975  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:39.639411 15273 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:39.782217  7812 raft_consensus.cc:2368] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:39.786443  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:39.787299  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:39.791095  7658 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.023s	sys 0.009s
W20260709 12:28:39.791757  7658 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.023s	sys 0.009s
I20260709 12:28:39.799698  7810 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:39.771576 15228 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:39.819186  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44773.4us
I20260709 12:28:39.820664  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.822644  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:39.806058  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.785822  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.804036  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:39.826251  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.781620 15349 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:39.849107  7803 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:39.758671 15120 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.771960 15162 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.852370  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3094) took 2249 ms. Trace:
I20260709 12:28:39.848459 15228 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.855118  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:39.766535 15244 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:28:39.818452 15193 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:39.801285 15346 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:39.856698  7658 rpcz_store.cc:276] 0709 12:28:37.598532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.598673 (+   141us) service_pool.cc:224] Handling call
0709 12:28:37.690280 (+ 91607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:38.255192 (+564912us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b18002a9a60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f1abfb 00007fc001f2564b
0709 12:28:38.264508 (+  9316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:38.264537 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:38.271742 (+  7205us) log.cc:844] Serialized 26 byte log entry
0709 12:28:38.755666 (+483924us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:39.792465 (+1036799us) log.cc:844] Serialized 14 byte log entry
0709 12:28:39.792900 (+   435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.793017 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:39.793084 (+    67us) raft_consensus.cc:1685] finished
0709 12:28:39.847011 (+ 53927us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:39.847514 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104156,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":54613,"spinlock_wait_cycles":3588992,"thread_start_us":1541638,"threads_started":3,"wal-append.queue_time_us":700208}
I20260709 12:28:39.844655 15362 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:39.845093 14987 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:39.848163 15143 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:39.904349  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:39.905155  8413 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.912500 15228 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.913275 15365 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:39.914330  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.865275 15193 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:39.924150 15193 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.924970 15360 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.925634 15193 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:39.926607 15221 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:39.927254 15221 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:39.913409 15228 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:28:39.932067 15040 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:39.932953 15040 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:39.928299 14987 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:39.934254 15129 consensus_meta.cc:287] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.001s
I20260709 12:28:39.933677 15040 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.928424 15221 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.933792 14987 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.938411 15221 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:39.940142 14987 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:39.940934 14987 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:39.946393  8413 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:39.800876 15273 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:39.864696 15162 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:39.934751 15129 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:39.859615 15341 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 12:28:39.860212 15215 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:39.955720  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.020s	sys 0.036s
W20260709 12:28:39.965791  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.020s	sys 0.036s
W20260709 12:28:39.956955 15301 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.002s
I20260709 12:28:39.971833  7807 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
W20260709 12:28:39.972282  8380 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.017s	sys 0.025s
W20260709 12:28:39.972749  8380 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.017s	sys 0.025s
W20260709 12:28:39.966229  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 542385us
W20260709 12:28:39.966709  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.003s	sys 0.050s
W20260709 12:28:39.966104 15364 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.008s
W20260709 12:28:39.975862  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 510746us
W20260709 12:28:39.978070  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:39.973088  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 825620us
W20260709 12:28:40.000924  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.995553  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:39.982518  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:39.977811 15242 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.025s
W20260709 12:28:39.977540 15333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.011s	sys 0.021s
I20260709 12:28:39.973839 15368 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:40.008630  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 182466us
I20260709 12:28:39.988652 15040 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:40.014732 15143 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:40.015168  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.022859  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:39.985369 15274 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.004s
I20260709 12:28:40.023308  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:40.025555  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:40.006413 15281 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.948s	user 0.016s	sys 0.000s
I20260709 12:28:40.037325  7737 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:28:40.007730  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:40.013155 15242 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.008s	sys 0.025s
I20260709 12:28:39.980293 15372 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:40.034650  8411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.005s	sys 0.008s
I20260709 12:28:40.038551  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:39.979977 15371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.010s	sys 0.024s
W20260709 12:28:40.049240  8411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.005s	sys 0.009s
W20260709 12:28:40.055995  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48353.2us
W20260709 12:28:40.063748  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.016567 15333 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:40.022008 15135 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.060976  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:40.060150  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 426768us
I20260709 12:28:40.151541  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:40.141094  7626 leader_election.cc:400] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
W20260709 12:28:40.144619  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 340696us
I20260709 12:28:40.154789  7626 leader_election.cc:403] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:28:40.153802  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:40.066920  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:40.163720  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:40.164041  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 103126us
W20260709 12:28:40.164994  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 98155.9us
W20260709 12:28:40.169778  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 107101us
I20260709 12:28:40.170634  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:40.028434 15379 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:40.046442 15341 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:40.173452  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.036557 15040 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:40.173465  7807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:40.179694  7733 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:28:40.192389  7733 pending_rounds.cc:85] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 14
I20260709 12:28:40.192685  7733 pending_rounds.cc:107] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:28:40.192922  7733 raft_consensus.cc:2889] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:40.182403  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.194027  8413 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:28:40.068311 15112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.002s	sys 0.016s
I20260709 12:28:40.189932  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:40.057610 15244 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:40.061777 15371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.011s	sys 0.024s
I20260709 12:28:40.059340 15274 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20.
I20260709 12:28:40.060151 15372 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:40.197413  8412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.014s	sys 0.026s
W20260709 12:28:40.218753  8412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.017s	sys 0.028s
W20260709 12:28:40.198035  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 202591us
W20260709 12:28:40.209687  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 360439us
I20260709 12:28:40.216979  7807 raft_consensus.cc:606] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:40.152817 15196 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:40.219910  7807 raft_consensus.cc:614] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:40.220039  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:40.061998 15099 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.220716  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:40.057561 15346 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.136s
I20260709 12:28:40.180564 15277 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.178433 15373 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:40.194073 14594 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:40.061143 15125 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:40.231949  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:40.153185 15269 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:28:40.144771 15329 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
W20260709 12:28:40.247247  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 323271us
I20260709 12:28:40.244166  8415 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:40.207598 15112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.006s	sys 0.021s
I20260709 12:28:40.192094 15386 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:40.286937  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:40.271607  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.224323 15281 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:40.169142 15135 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:40.329888  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:40.327891  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 163105us
I20260709 12:28:40.242231  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:40.228749 15244 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:40.208266 15146 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:40.246521 15120 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:40.223256 15372 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:40.355649  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:40.246548 15285 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:40.240677 14594 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:40.365630  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 175844us
I20260709 12:28:40.247524 15349 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:40.363278  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 121164us
I20260709 12:28:40.368361  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:40.298290 15054 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:40.225122 15341 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:40.318287 15399 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:40.338531 15215 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:40.229067 15099 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.299340 15196 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:40.354486 15273 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:40.358485 15146 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.385325  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:40.371660 14594 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:40.229180 15040 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:40.353375 15396 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:40.410049  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:40.369920 15368 raft_consensus.cc:2764] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:28:40.372390 15401 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.381234 15281 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:40.378381 15341 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.387002 15054 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:40.272410 15333 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.337656 15373 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 12:28:40.439837 15145 raft_consensus.cc:993] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:28:40.436163 15298 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:40.442528 15389 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:28:40.424218 15196 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.423977 15146 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:40.460657 15381 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.432s	user 0.000s	sys 0.002s
W20260709 12:28:40.505859  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 280186us
I20260709 12:28:40.394649 15361 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:28:40.497009 15349 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.406988 15346 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.507584 15346 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.497912  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:40.509815  7729 raft_consensus.cc:2368] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:40.452229  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.434913 15040 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:40.444530 15411 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:40.455600  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.453514  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.455541  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:40.456007  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.456619  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.457351  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:40.456811  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.461277  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:40.461552  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.490667  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.454864  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.460373  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.452428 15329 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.506031 15196 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:40.511698  8413 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:40.522194 15329 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.524760 15196 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.525066  7660 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:40.526007  7730 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:40.526883  7735 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:40.525575 15196 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.528615  7654 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:40.530476  7734 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:40.531112  7658 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 14.
I20260709 12:28:40.531874  7732 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:40.516418  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.534914 15040 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.497531  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.428380 15273 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:40.539614  7736 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:40.506095  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1504) took 1701 ms. Trace:
I20260709 12:28:40.541209  7728 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:40.497176  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.542423  7659 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:40.540676  8406 rpcz_store.cc:276] 0709 12:28:38.804368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.849400 (+ 45032us) service_pool.cc:224] Handling call
0709 12:28:38.953364 (+103964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:39.221201 (+267837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:39.221265 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:39.221991 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 12:28:39.622614 (+400623us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:39.622669 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.923627 (+300958us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:39.923703 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:40.505764 (+582061us) raft_consensus.cc:1685] finished
0709 12:28:40.505788 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:40.506023 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":157,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":31440,"spinlock_wait_cycles":613712128,"thread_start_us":309548,"threads_started":1,"wal-append.queue_time_us":696603}
I20260709 12:28:40.513162  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.503996 15298 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.546363 15298 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:40.546437 15273 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:40.547426 15273 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:40.511729 15311 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:40.548789 15311 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:40.549996 15311 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.514839  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1544) took 1254 ms. Trace:
W20260709 12:28:40.457700 15221 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.001s
I20260709 12:28:40.499640  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:40.453279 15398 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.439206 15341 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:40.552011  7733 rpcz_store.cc:276] 0709 12:28:39.260407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.290158 (+ 29751us) service_pool.cc:224] Handling call
0709 12:28:39.290700 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:40.193402 (+902702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:40.514413 (+321011us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:40.514737 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043147008}
W20260709 12:28:40.556315  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.453441 15396 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:40.493403 15281 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:40.562238 15281 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.564711  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:40.518603 15129 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:40.570981 15364 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.571951 15364 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.574332  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.574113 15329 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.575973  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.576522  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.577517 14906 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:40.578302  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.579044  8406 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:40.580013 15346 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.581143  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.581688  8406 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:40.582047 15228 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.582743 15228 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.583015 15368 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.584857  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:40.585454 15228 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.586668  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.587255  7659 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:28:40.589023  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.583262  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.583911 15368 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.589164  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:40.590418  8413 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:40.588477 15196 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.574818 15364 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.595249 15368 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:40.594873  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.589566  8414 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:28:40.600584  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.437502 15403 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:28:40.610688  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77033.7us
I20260709 12:28:40.592592 15311 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:40.614418  7654 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:40.621305  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:40.631276  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.634176  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.635576  7733 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:40.605247 15373 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:40.605710 15388 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:40.659036  7733 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:28:40.659914  7654 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:40.661218  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:40.609923 15403 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:40.661350  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 161790us
I20260709 12:28:40.667896  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:40.669720  7803 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:40.670109  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:40.670717  7805 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:40.671489  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.672984  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.673116  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.674590  7803 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:28:40.675763  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.676056  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.677523  7811 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:28:40.678795  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.673759  7808 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:40.682252  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.684859  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.686059  7808 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:40.643572 15221 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:40.702390  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:40.690904  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:40.703253  7656 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:40.691545  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:40.702435  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.699163  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.703195  7810 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:40.709199  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.705852  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:40.710227  7661 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:40.712716  7806 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:40.671586 15196 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.715700  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:40.718062  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.723556  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.664880 15402 raft_consensus.cc:1081] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:28:40.729552  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.678243 15215 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:40.679905 15403 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:40.730652  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.730554  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:40.732962  7807 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:28:40.713752  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.699806  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:40.747929  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.003s
W20260709 12:28:40.762667  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.007s	sys 0.003s
I20260709 12:28:40.648962 15364 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:28:40.692193 14987 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:40.698243 15410 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.640123 15377 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 12:28:40.778769  7627 leader_election.cc:400] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
W20260709 12:28:40.787907  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 90928.3us
I20260709 12:28:40.776674  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:40.728859 15395 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
W20260709 12:28:40.791612  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 131441us
I20260709 12:28:40.784584  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:40.758486 15215 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.758760 15228 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:40.833455  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 437691us
I20260709 12:28:40.834461  7805 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:40.836364  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.879300  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:40.834815  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:40.787981  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:41.236254  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.055s	user 0.003s	sys 0.006s
I20260709 12:28:41.237003  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:41.239086  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.709235 15054 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.803104 15120 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:41.282898 15120 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:40.790436 15135 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:41.004307 15184 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.403s	user 0.010s	sys 0.006s
I20260709 12:28:40.758592 15196 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.289134  8415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:28:41.290400  8415 raft_consensus.cc:606] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:40.836109 15215 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:40.559083 15396 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.785950 15221 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 12:28:40.925393 15228 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:41.297055 15228 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.884279 15285 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:41.299808 15285 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:40.771953 15346 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.318197 15386 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:40.883617 15361 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 12:28:40.769367 15362 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:28:40.811877 15368 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 12:28:41.306303 15386 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:40.782950 15374 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 12:28:41.308043 14987 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.309283 14987 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:41.033074 15390 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:28:41.316069 15384 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:41.316833 15384 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:41.317750  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1340) took 2182 ms. Trace:
I20260709 12:28:40.240710 15277 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:41.113070 15391 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.474s	user 0.005s	sys 0.000s
I20260709 12:28:40.196213 15379 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:41.074793 15393 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:28:41.019299 15397 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
W20260709 12:28:40.988868 15399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.021s	sys 0.065s
W20260709 12:28:41.008806 15406 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:28:41.075999 15414 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 12:28:41.108105 15418 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:28:41.109203 15420 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 12:28:41.034735 15421 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:28:41.290980  8415 raft_consensus.cc:614] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:41.246915  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.000s	sys 0.013s
W20260709 12:28:41.249055  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 464377us
W20260709 12:28:41.252213  8380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.048s	sys 0.068s
W20260709 12:28:41.273715  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.004s	sys 0.006s
I20260709 12:28:40.553861 15398 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:41.343731  8380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.049s	sys 0.069s
W20260709 12:28:41.344419  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 712178us
I20260709 12:28:41.344980  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.346755  7655 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:28:41.348044 15399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.021s	sys 0.070s
I20260709 12:28:41.348649 15399 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:41.350615  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 541190us
I20260709 12:28:41.352393  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.355737 15432 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.357090 15432 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.359638 15402 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.359908  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1548) took 1926 ms. Trace:
I20260709 12:28:41.346972 15398 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.160321 15403 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.680740 15388 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.360314  7731 rpcz_store.cc:276] 0709 12:28:39.454695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.456100 (+  1405us) service_pool.cc:224] Handling call
0709 12:28:39.456600 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:39.461077 (+  4477us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:39.970950 (+509873us) log.cc:844] Serialized 14 byte log entry
0709 12:28:40.209666 (+238716us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:40.210314 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:28:40.210549 (+   235us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:40.210607 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:40.210727 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:41.359333 (+1148606us) raft_consensus.cc:1685] finished
0709 12:28:41.359377 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:41.359687 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3948,"prepare.queue_time_us":983110,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":22903,"raft.run_cpu_time_us":688,"raft.run_wall_time_us":8155,"thread_start_us":736285,"threads_started":2,"wal-append.queue_time_us":746035}
I20260709 12:28:41.364512  7662 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:41.374543 15196 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.378175  8413 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:28:41.280016  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 445193us
W20260709 12:28:41.385602  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.000s	sys 0.014s
I20260709 12:28:40.533265 15411 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.302175 15346 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.350876 15399 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:41.124814 15341 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.005s
I20260709 12:28:41.389783 15411 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.390892 15341 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:40.806607 15410 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.318349  8408 rpcz_store.cc:276] 0709 12:28:39.134925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.141321 (+  6396us) service_pool.cc:224] Handling call
0709 12:28:39.547454 (+406133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:39.548040 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:39.548087 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:39.548921 (+   834us) log.cc:844] Serialized 26 byte log entry
0709 12:28:40.010854 (+461933us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:40.010939 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:40.011042 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:41.315377 (+1304335us) raft_consensus.cc:1685] finished
0709 12:28:41.315420 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:41.317148 (+  1728us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540059ecd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:41.317480 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57673,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"spinlock_wait_cycles":932685184,"thread_start_us":461348,"threads_started":1,"wal-append.queue_time_us":1099563}
I20260709 12:28:41.397522 15410 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:41.398931  7659 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:28:40.435576 15401 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.402258 15401 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.402436  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:41.409768 14987 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.360628 15402 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.326846 15379 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.411880  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:41.360818 15277 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:41.332173  7808 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
W20260709 12:28:41.416436  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 628400us
I20260709 12:28:41.417629 15431 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:41.333276  7653 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:41.420432 15228 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.421612  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:41.383278  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:41.280884 15129 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:41.485571 15129 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:41.486661 15129 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:41.470222  8415 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:28:41.283809 15120 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:41.445133  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58494.5us
I20260709 12:28:41.485584  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:41.449518  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.450430  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:41.406965 15392 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.797s	user 0.002s	sys 0.002s
I20260709 12:28:41.463618  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1502) took 2660 ms. Trace:
I20260709 12:28:41.464697  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:41.466548 15402 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:41.467829  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85035.8us
I20260709 12:28:41.468789 15361 consensus_queue.cc:1243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:41.469206 15277 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:41.459506 15386 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:28:41.318565 15384 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.532301 15384 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:41.459329 15431 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.533872 15431 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:41.472180  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 127574us
I20260709 12:28:41.536187  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:41.538125  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46921.4us
I20260709 12:28:41.538466  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.475550 15437 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:41.539855  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.540201 15437 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:41.540822 15437 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:41.541318 15437 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:41.453572 15341 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.478549 15271 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:41.543488 15438 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.544360 15438 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.544956  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:41.481724 15435 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:41.483179 15401 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:41.546245 15435 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:41.470141 15362 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:41.547477 15362 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:41.492157  8408 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:41.549185 15362 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.470337  7661 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:41.549908 15435 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.302219 15099 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.552731 15435 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:41.558151 15277 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:41.557986 15402 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:41.532490 15193 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.606s	user 0.015s	sys 0.012s
I20260709 12:28:41.477495  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:41.547356 15228 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.389884 15346 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.547101 15401 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:41.559944 15277 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.520869  8412 rpcz_store.cc:276] 0709 12:28:38.802941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.846038 (+ 43097us) service_pool.cc:224] Handling call
0709 12:28:38.846526 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:39.628721 (+782195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:39.628774 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:39.639835 (+ 11061us) log.cc:844] Serialized 26 byte log entry
0709 12:28:40.219320 (+579485us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:40.219384 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:40.219464 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:41.455645 (+1236181us) raft_consensus.cc:1685] finished
0709 12:28:41.463070 (+  7425us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:41.463420 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37100,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":1360638,"threads_started":2,"wal-append.queue_time_us":1038099}
I20260709 12:28:41.585446  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:41.588132  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:41.581842  7804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:41.590822 15322 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.591825 15386 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:41.593721  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:41.594568 15398 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.575959 15362 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:41.571884 15341 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.562456 15438 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.601615  8415 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:41.604732 15398 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.605247 15329 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.610397  7804 raft_consensus.cc:606] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:41.616365  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 151890us
I20260709 12:28:41.616523 15193 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:41.680627 15322 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.618485 15196 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:41.682238 15196 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:41.682821 15196 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:41.683383 15403 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.669093 15228 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:41.684422 15273 consensus_meta.cc:287] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.136s	user 0.017s	sys 0.011s
I20260709 12:28:41.684958 15273 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.686290 15228 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:41.687556  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:41.628780 15146 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.124s	user 0.015s	sys 0.007s
W20260709 12:28:41.630298  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 92053.1us
I20260709 12:28:41.597136 15434 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.633231  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:41.633566  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:41.633749  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:41.634397 15423 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:41.643816  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 291717us
I20260709 12:28:41.647550 15371 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:41.698747  7804 raft_consensus.cc:614] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:41.648041  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:41.651554 15410 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:28:41.576864 15271 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.100s
I20260709 12:28:41.687453 15341 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:41.704772  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 71467.2us
I20260709 12:28:41.707695  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:41.690301 15386 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:41.662966 15244 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.298s	user 0.017s	sys 0.006s
I20260709 12:28:41.582525 15401 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.699350 15371 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:41.679165 14906 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.101s	user 0.010s	sys 0.014s
W20260709 12:28:41.679764 15298 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.132s	user 0.011s	sys 0.012s
I20260709 12:28:41.679581  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:41.577797 15379 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.654647 15129 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.677095  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:41.725543  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:41.682296 15388 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.730459 15129 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:41.732380  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:41.733528 15364 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.735332 15242 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:41.735931 15329 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.739342 15403 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:41.742590  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:41.743755  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:41.743867  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96035.2us
I20260709 12:28:41.724054 15371 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:41.746781 15400 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.751987  7811 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:41.721103 15401 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:41.747995  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:41.700949 15146 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.751436 15273 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:41.790333  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77120.5us
I20260709 12:28:41.758493 15298 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.790316 15040 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:41.793440 15372 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.406s	user 0.008s	sys 0.017s
I20260709 12:28:41.804303  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:41.779732 15364 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.809981  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:41.812813 15379 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:41.816047  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 83737.6us
I20260709 12:28:41.781184 15322 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.816890 15411 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:41.825057 15333 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:41.828965 15120 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:41.831080 15215 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.005s
W20260709 12:28:41.831430 15311 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.191s	user 0.008s	sys 0.018s
I20260709 12:28:41.814244 15424 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:41.791589 15386 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.843421 15040 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.771806 15242 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:41.760977 15244 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16.
I20260709 12:28:41.776618 15403 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:41.707836 15423 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.863044 15411 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.859548  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:41.843497 15271 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.847780  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3438) took 1152 ms. Trace:
I20260709 12:28:41.816090 15447 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:41.875528  7810 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:41.882072  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 202656us
W20260709 12:28:41.854574  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 146992us
W20260709 12:28:41.883982  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:41.904870  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.876768 15400 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.910180 15400 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.911486 15398 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.912004 15271 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.915788  7731 raft_consensus.cc:2368] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20.
I20260709 12:28:41.883199 15403 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:41.869212 15215 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.887392 15242 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.726351 15434 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.876434 15040 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:41.890998 15221 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:41.891927  8405 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:41.896855 14594 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.428s	user 0.018s	sys 0.009s
I20260709 12:28:41.896365 15431 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.867285 15388 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.880684  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:41.898324  7808 rpcz_store.cc:276] 0709 12:28:40.695430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.697668 (+  2238us) service_pool.cc:224] Handling call
0709 12:28:41.847464 (+1149796us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54003eb2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:41.847660 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308076416}
I20260709 12:28:41.907404  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:41.871766 15298 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.886003 15120 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.914685 15271 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:41.934947  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.898142 15411 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:41.914573 15379 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.945779  8380 leader_election.cc:400] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20.
I20260709 12:28:41.957019 15431 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.946713 15388 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.947497  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:41.958889  8414 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:41.959443  7803 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:41.948129  7731 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:41.948660  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.961414 15388 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.962666  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:41.963840  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3121) took 1303 ms. Trace:
I20260709 12:28:41.964282  7655 rpcz_store.cc:276] 0709 12:28:40.660137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.662007 (+  1870us) service_pool.cc:224] Handling call
0709 12:28:40.665012 (+  3005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:41.347352 (+682340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.963456 (+616104us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540002ec54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:41.963738 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565868928}
I20260709 12:28:41.965103  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:41.965633 15274 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:41.966288 15184 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:41.966086 14594 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.967015 15184 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.967406 15146 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:41.967543 15184 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:41.968835 14594 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:41.969753 14594 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:41.970145 14594 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.970531 14594 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:41.960932  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2891) took 1287 ms. Trace:
I20260709 12:28:41.971315 15273 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:41.972437  8415 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:28:41.972922 15273 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:41.971701  7803 rpcz_store.cc:276] 0709 12:28:40.673777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.675513 (+  1736us) service_pool.cc:224] Handling call
0709 12:28:41.960661 (+1285148us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001324d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:41.960847 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2906399104}
I20260709 12:28:41.976114 15329 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.977883  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:41.967510 15271 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:41.980589 15271 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:41.980815 15329 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:41.981328 15329 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.982234 15329 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:41.982144 15271 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.979387  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:41.983567  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115463us
I20260709 12:28:41.984383 14906 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:41.988060  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:41.987978  8405 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:41.984939 15221 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:41.863811 15372 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:41.950212 15422 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:42.015489  8408 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:42.021293  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:41.968832 15398 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:41.955005 15364 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:41.954522 15437 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:42.028716  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 122182us
W20260709 12:28:42.024468  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 175962us
I20260709 12:28:42.013263 15242 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:42.011677 15146 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.012712 14906 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:42.043802  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.032717 15398 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:42.062340  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97320.3us
W20260709 12:28:42.069545  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81615us
I20260709 12:28:42.033004 15434 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.049633 14906 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:42.092100 15446 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.162s	user 0.001s	sys 0.001s
W20260709 12:28:42.095885  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.099684  7809 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:28:42.101939 15221 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.103400  7660 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
I20260709 12:28:42.121326 15437 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:42.074177 15449 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:28:42.071089 15398 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:42.084473 15384 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:42.085158 15271 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:42.156239  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77012us
I20260709 12:28:42.082921 15341 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.180408 15374 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.181198 15374 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.084237 15146 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.203831  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3088) took 1422 ms. Trace:
I20260709 12:28:42.204464  7662 rpcz_store.cc:276] 0709 12:28:40.781007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:41.366660 (+585653us) service_pool.cc:224] Handling call
0709 12:28:41.368226 (+  1566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:41.595821 (+227595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:41.595876 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:41.635651 (+ 39775us) log.cc:844] Serialized 26 byte log entry
0709 12:28:41.710750 (+ 75099us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:41.710806 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.912228 (+201422us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b540017b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:41.912295 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:42.203406 (+291111us) raft_consensus.cc:1685] finished
0709 12:28:42.203431 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:42.203736 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68695,"raft.queue_time_us":76796,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":2082,"spinlock_wait_cycles":150716288,"thread_start_us":74494,"threads_started":1,"wal-append.queue_time_us":169469}
I20260709 12:28:42.206956 15422 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.196635 15434 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:42.207753 15422 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.208410 15434 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.210558 15434 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.210388 15422 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.212581 15400 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:42.214174 15322 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:42.164256 15437 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:42.175345 15215 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.235312  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:42.194743 15384 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:42.212213 15184 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:42.235055 15196 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.003s
I20260709 12:28:42.252249  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:42.219977 15346 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.255944 15346 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.258826 15346 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.245704  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:42.234190 15422 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.251871 15424 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:42.192883 15274 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:28:42.263412 15437 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.259759 15196 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.299130 15379 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:42.299438 15437 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.300046 15379 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.300475  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.299525 15196 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.301384  7732 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:42.303928  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.305353  7733 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:28:42.259007 15401 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.310137 15384 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.274482 15432 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.219615 15431 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.239477 15447 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:42.313274 15447 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.308786 15401 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.316212 15146 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:42.316659 15447 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.234187 15273 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.319407 15379 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.263200 15403 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.323022 15403 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.284003  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3120) took 1624 ms. Trace:
I20260709 12:28:42.323901  7660 rpcz_store.cc:276] 0709 12:28:40.659815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.660947 (+  1132us) service_pool.cc:224] Handling call
0709 12:28:42.283583 (+1622636us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540028d054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:42.283853 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2842361216}
I20260709 12:28:42.332780 15184 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.215118 15322 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.341452 15322 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:42.346343  7733 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:42.356104  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 138592us
I20260709 12:28:42.357735  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:42.358884 15424 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:42.359475 15424 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:42.360194 15424 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:42.286484  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89093.1us
I20260709 12:28:42.288278  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:42.295601 15377 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:42.368669 15377 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.369212 15377 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:42.295169 15422 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.373795 15422 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.376214  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:42.296358 15454 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:28:42.262368 15455 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.163s	user 0.000s	sys 0.003s
I20260709 12:28:42.378996  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.313199 15431 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.381088  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.382282  7737 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:28:42.383001 15431 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.281414 15372 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.393239  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.395035  7737 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:42.396234  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.395253  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.397357  7735 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:42.230356 15434 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:42.399744 15434 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.402935 15434 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.413097  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.425865 15285 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.441146 15285 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:42.441480  7728 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:28:42.288848 15374 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.440013  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3433) took 1940 ms. Trace:
I20260709 12:28:42.440331  7732 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:42.442999 14594 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:42.443780  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 86410us
I20260709 12:28:42.446130  7809 rpcz_store.cc:276] 0709 12:28:40.499294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.499435 (+   141us) service_pool.cc:224] Handling call
0709 12:28:40.499819 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:42.100413 (+1600594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:42.439525 (+339112us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54003737d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:42.439904 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3581044096}
I20260709 12:28:42.447666  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:42.455034  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.463271 14594 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.465189  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:42.467610  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:42.468021  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.467698  7728 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:28:42.468772  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.468933  7728 pending_rounds.cc:85] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
W20260709 12:28:42.451108 15040 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.005s
I20260709 12:28:42.505071  7735 raft_consensus.cc:2410] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:42.481304  7728 pending_rounds.cc:107] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:28:42.514034  7728 raft_consensus.cc:2889] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:42.514225  7660 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:42.515559  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:28:42.515764  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.516767  7806 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:42.517129  7658 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:42.517584  7655 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:42.518641  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.519826  7658 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:42.521134  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:42.515959  7810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2894) took 1811 ms (client timeout 1710 ms). Trace:
I20260709 12:28:42.516992  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.523141  7660 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:42.518656  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:42.508793  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65675.3us
I20260709 12:28:42.519579  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.524374  7660 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:42.525835  7655 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:42.530243  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.530861  7809 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:42.523893  7810 rpcz_store.cc:269] 0709 12:28:40.704846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.708424 (+  3578us) service_pool.cc:224] Handling call
0709 12:28:42.515636 (+1807212us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400372dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:42.515864 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610620288}
I20260709 12:28:42.512120  7732 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:42.539680  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.539080  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:42.545503  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.546535  7661 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:42.551867  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.552663  7655 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:42.553395 15424 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.554296  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.555015  7808 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:42.555405  7806 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:42.556437  7658 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:42.556870 15394 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:42.557503  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3080) took 2093 ms (client timeout 2027 ms). Trace:
I20260709 12:28:42.511168 15040 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:42.557968  7653 rpcz_store.cc:269] 0709 12:28:40.449075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.451900 (+  2825us) service_pool.cc:224] Handling call
0709 12:28:42.530629 (+2078729us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540033e0d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:42.542739 (+ 12110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890612992}
W20260709 12:28:42.538084 15329 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.003s
I20260709 12:28:42.560777 15329 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.561276 15135 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:42.562105 15135 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.560061  7659 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:42.563694  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.564558  7728 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:42.565290 15135 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.545189 15399 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.566401 15399 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.566844  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:42.569998 15399 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.571482  7662 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:42.572053 15399 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:42.573256 15371 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.731s	user 0.015s	sys 0.000s
I20260709 12:28:42.572963 15399 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.574034 15371 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.575803  8415 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:42.577761 15371 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.581182  7655 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:42.588716  7730 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:42.590386  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.591786  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:42.588641  7660 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.594468  7810 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:42.579051 15371 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:42.602150  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:42.605362  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.619017  7730 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:42.609357 15424 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.610307  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:42.610975 15457 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.471s	user 0.000s	sys 0.004s
I20260709 12:28:42.579618 15040 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.615403  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.622730  7736 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:42.629153  7660 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:42.632474  7655 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:42.638463  7732 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:42.639006  7653 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:42.639917  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.640296  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.639958  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.643150  7729 raft_consensus.cc:2410] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:42.643898  7659 raft_consensus.cc:1240] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:42.640658  7728 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:42.631114 15411 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.677s	user 0.005s	sys 0.010s
I20260709 12:28:42.623721 15277 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:42.649379  7655 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:28:42.627489 15371 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:42.635020 15135 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:42.664788  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.654601 15277 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:42.677728  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.682964  7728 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:28:42.657943 15329 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.688496  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:42.688822  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:42.691550  7653 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:42.667497 15135 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.711691  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.659533 15411 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.725286  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:42.730899  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.731882  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.697335 15277 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:42.742020  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:42.711267 15135 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:42.746497  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:42.735317 15386 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.027s	sys 0.039s
I20260709 12:28:42.754323  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.755404  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:42.760424  7656 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:42.760784  8408 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:42.761771  7803 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:42.764046  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.764348  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:42.765103  8414 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:42.735621 15399 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:42.767761  7658 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:42.735069 15242 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:42.767818  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 197157us
I20260709 12:28:42.768422  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:42.760116 15386 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.027s	sys 0.039s
I20260709 12:28:42.776046  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.776626  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:42.784518  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.779855  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:42.793139  7732 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:42.795687  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.807051  7729 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:42.795235 15372 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:42.812887 15372 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:42.813453 15372 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:42.822942  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.828702  7659 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:42.791977 15371 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.834556  7655 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:42.778487 15215 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:42.845060  7730 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:42.845613  7730 pending_rounds.cc:85] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:28:42.845880  7730 pending_rounds.cc:107] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:42.846207  7730 raft_consensus.cc:2889] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:42.850953  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.871304  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.872181  7732 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:42.874019  8408 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:42.882421  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.788488 15242 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:42.896736  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.899456  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.900450  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.910413  7811 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:28:42.899528  7657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3081) took 2433 ms (client timeout 1710 ms). Trace:
W20260709 12:28:42.917279  7657 rpcz_store.cc:269] 0709 12:28:40.449407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.453156 (+  3749us) service_pool.cc:224] Handling call
0709 12:28:42.852234 (+2399078us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b540033d1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:42.882373 (+ 30139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5137252736}
I20260709 12:28:42.923872  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:42.915251  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:42.929003  7660 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:28:42.932371  7809 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:28:42.933262  7732 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:42.936014  7730 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:42.901340 15040 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:42.879679 15215 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:42.953534  7656 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:42.928069 15399 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:42.963352  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:42.971531  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.945705 15322 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:42.980798  7660 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:42.980654  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.985253  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:42.990566 15466 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:28:42.995178  7729 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:43.000768  7728 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:43.003135  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.003846  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:43.004678  7729 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:42.986785 15411 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:43.006443 15471 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:28:43.012213  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.004009  7807 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:28:43.014021  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 232840us
I20260709 12:28:43.023330  8408 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:42.997242 15322 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:43.039650  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.037870  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.043875  7729 raft_consensus.cc:2410] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:28:43.048157  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:42.985950 15040 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.065557  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.066455  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.067199  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.070883  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.071548  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:43.072185  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.072575  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.074903  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.080844  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:42.993088 15399 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.053015  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:43.088590  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.095041  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.096094  7731 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:28:43.097164  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:43.097986  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.100020  8405 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:43.100589 15371 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.101435  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.101306 15371 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.102181  8405 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.102510  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.103631  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:43.105073  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57731.3us
I20260709 12:28:43.106040  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.107465 14594 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:43.108163 14594 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.108525 14594 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.108803  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.110538  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.098819  8407 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:43.115674 15396 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:28:43.116948 15459 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.899s	user 0.000s	sys 0.004s
I20260709 12:28:43.118175  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.118839  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.119103  7734 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:43.120303 15423 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.120493  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:43.121030 15423 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:43.141826  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.101722  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.131646  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3124) took 2470 ms. Trace:
I20260709 12:28:43.143775 15329 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:43.143652  7654 rpcz_store.cc:276] 0709 12:28:40.661262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.662686 (+  1424us) service_pool.cc:224] Handling call
0709 12:28:42.556229 (+1893543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:42.556460 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:42.556495 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:42.556538 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:42.556571 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:42.556676 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:43.131302 (+574626us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540033e0d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:43.131433 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4310853504}
W20260709 12:28:43.088827 15377 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.719s	user 0.016s	sys 0.000s
I20260709 12:28:43.121404 15271 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:28:43.145668 15271 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:43.146111 15271 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:43.144405 14906 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.992s	user 0.010s	sys 0.006s
I20260709 12:28:43.147146 15271 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.147593 14906 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.148885 15472 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
I20260709 12:28:43.145323 15377 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.151335 15377 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.153072  7733 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:43.135819  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.130795  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.124833  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1554) took 1375 ms. Trace:
I20260709 12:28:43.155174  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:43.160703 15470 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:28:43.160863  7733 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:43.155581  7735 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:43.089162  7804 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:43.098045  7731 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:43.157022  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.163435  7805 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:43.156461  7810 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:43.158703  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.166129  7728 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:42.688117  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:43.139544  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.172102  7734 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:43.159771  7736 rpcz_store.cc:276] 0709 12:28:41.749891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:41.790169 (+ 40278us) service_pool.cc:224] Handling call
0709 12:28:41.848951 (+ 58782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:42.631995 (+783044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:43.124491 (+492496us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54001450d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:43.124758 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765102976}
I20260709 12:28:43.176326  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.107542  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:43.177136  7804 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:28:43.177695  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 70293.2us
I20260709 12:28:43.163820  7805 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:43.180069  7730 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:43.185530  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:43.092223 15398 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.916s	user 0.015s	sys 0.000s
I20260709 12:28:43.177498  7737 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:43.200126  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.155725 15322 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.130728 15242 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.204802  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:43.208809  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:43.187094 15285 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.004s
I20260709 12:28:43.210462  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.219298  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.211825  7729 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 16 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:28:43.219770  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.207748  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.221638  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.221817 15371 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.224766  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.226383  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.129520 15040 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.227782  7732 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:28:43.228334 15135 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.229131  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.229743 14906 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.230613  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:43.231563  7806 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:43.218093  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.233006 15372 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:43.233896  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44829.6us
I20260709 12:28:43.224778  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.244158  7732 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:43.222666  7736 raft_consensus.cc:2410] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:28:43.127334 15193 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 12:28:43.211930  7728 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:43.246315  8411 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:28:43.249171  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:43.249590  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.217864 14594 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.252760  7736 raft_consensus.cc:2410] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:28:43.253042  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.245286  7805 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:43.254335  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.255129  7803 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:43.241504 15394 raft_consensus.cc:2764] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:28:43.258833  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:43.259886  7729 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:43.200462 15398 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.265985  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.270108  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:43.271616  7737 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:43.268549  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:43.208735 15476 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 12:28:43.253507  7700 connection.cc:256] RPC call timeout handler was delayed by -1.0258268080001471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
I20260709 12:28:43.278491 14594 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:43.279328  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2715754460000426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 12:28:43.279753  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1302754459998141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
W20260709 12:28:43.280041  7702 connection.cc:256] RPC call timeout handler was delayed by -1.082575445999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 12:28:43.280298  7702 connection.cc:256] RPC call timeout handler was delayed by -1.082575445999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 12:28:43.280434  7736 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:28:43.285358 15399 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.286332 15399 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:43.286788 15399 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:43.288676  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.288942  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.290066  8405 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:43.291844 15372 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.292342 15423 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:43.292754 15423 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:43.292901 14594 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:43.293475 14594 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.293838 14594 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.294873 15398 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.295890 15215 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.295729 15398 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:43.296521 15398 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:43.297288  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.298303  7809 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:28:43.289152  7811 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:43.305115  8408 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:43.300590  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.303718  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.304301  7805 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:28:43.258534 15135 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.306085  7808 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:43.300382  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:43.313898  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 455686us
W20260709 12:28:43.316212  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 135063us
I20260709 12:28:43.318881  7736 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:43.315505  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:43.302954 15469 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.409s	user 0.001s	sys 0.003s
I20260709 12:28:43.313310 14906 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:43.323681  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:43.320700  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:43.320742  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.326047  8415 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:43.303546 15403 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:43.255779 15040 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:28:43.285544 15242 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.332203  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:43.357290  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:43.365758  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:43.353892  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70991.8us
I20260709 12:28:43.267190 15402 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:28:43.359187  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:43.374267  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:43.357921  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60717.2us
I20260709 12:28:43.374462  7811 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:43.383370 14594 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.383001  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.352948 15432 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:43.387147  7659 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:43.403980  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:43.406021  7736 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:43.406510  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:43.407694 15399 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.409180 15135 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:43.409911 15242 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.356343 15403 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.411587 15403 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:43.412490 15242 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.413506 15472 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:43.414011 15472 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.414407 15472 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:43.415951  7653 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:43.414764  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.354032 15394 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.063s
W20260709 12:28:43.417526 15277 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.014s
I20260709 12:28:43.404338  7808 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:43.421012  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:43.429714 15472 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.434208  7736 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:43.418035 15277 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.436393  7811 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:43.394789 15396 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:43.418779 15400 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:28:43.441852 14594 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.353199 15193 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:28:43.441175  7732 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:43.434314  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 226740us
W20260709 12:28:43.448515  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 341356us
I20260709 12:28:43.453295  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.456668  7653 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [21.18-21.18]
I20260709 12:28:43.459736  7732 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:43.455054  7737 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:28:43.431618 15040 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.463941  7805 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:43.437250 15242 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:43.465529  7807 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:43.433004 15135 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:43.435777 14906 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:43.468904  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.451632  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:43.470899 14594 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:43.444376 15396 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:43.527459  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74315.2us
I20260709 12:28:43.436348 15372 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:28:43.538162  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.539566  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.540328  7653 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:43.541900 15277 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.542048 15472 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:43.549396  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97848.4us
I20260709 12:28:43.523612 15285 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.536932 15396 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:43.526541 15242 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.534335 14906 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.548692 14594 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:43.560094  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.574519 14594 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.575279 14594 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:43.575203  7658 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:43.579277  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 158470us
I20260709 12:28:43.580742  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:43.605371  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 295940us
W20260709 12:28:43.607841  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 211735us
I20260709 12:28:43.641695  7733 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:43.530709 15040 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.649542  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:43.652127  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.653508  7658 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:43.573674 15242 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.564659 15486 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:43.579012 14906 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:43.660665  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.557924 15285 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.647692 15377 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:43.679708  7733 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 12:28:43.685535  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 225045us
I20260709 12:28:43.669463  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:43.635335 15424 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:28:43.696305  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.697305 14594 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.663070 15488 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:43.702076  7658 raft_consensus.cc:2410] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:43.659927 15491 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:43.665321 15486 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:43.656170 15403 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.672291 15377 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:43.720368  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 166338us
I20260709 12:28:43.721715  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.722640 14594 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.724701  7733 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:28:43.703213  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 165166us
I20260709 12:28:43.728271  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:43.673733 15432 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:43.739178  7810 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.563s	user 0.017s	sys 0.002s
I20260709 12:28:43.741627  7810 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:43.706461 15488 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:43.739781  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.713457 15491 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:43.749176 15486 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.750696 15271 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:43.752041 15507 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.752830 15507 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.754703 15403 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:43.755839 15498 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:43.757809 15507 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.758579 15507 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:43.759150 15507 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.761919 15271 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.761461 15488 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.768114  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.768622 15488 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:43.768885  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:28:43.763080  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:43.763568  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:43.771889  7660 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:43.764242  7653 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:43.764561  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.765097  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:43.765727 15489 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:43.774425 15489 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.774926 15489 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:43.765566  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.779989  7662 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:43.780061 15271 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.782752 15271 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.785969 15271 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:43.766876  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.766018 15507 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.744189 15322 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:43.737512 15487 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:43.729951 15377 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.698874 15135 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.797792 15491 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:43.798089  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.786679 15396 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:43.803542  7734 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.011s
I20260709 12:28:43.803988 14594 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:43.820292 14594 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:43.809345  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:43.745277 15424 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:43.814613  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47947.1us
W20260709 12:28:43.803908  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75772.2us
I20260709 12:28:43.797885 15040 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.822590  7658 raft_consensus.cc:2393] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:43.823102 14594 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:43.817750  7734 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:43.812826 15322 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:43.831612  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 265273us
W20260709 12:28:43.853794  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.854354  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:43.852329  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:43.911890 15502 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 12:28:43.946775 15475 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:43.947376 15475 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:43.948969  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3511) took 1409 ms. Trace:
I20260709 12:28:43.948566 15475 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.949467  7735 rpcz_store.cc:276] 0709 12:28:42.539658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.539869 (+   211us) service_pool.cc:224] Handling call
0709 12:28:43.155348 (+615479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:43.175842 (+ 20494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:43.948717 (+772875us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400072bd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:43.948900 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392505088}
I20260709 12:28:43.950069 15475 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:28:43.930799  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.935829 15215 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:43.952869 15215 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:43.953831 15215 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.916713 15489 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.958235  7655 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:43.930523 15285 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.823674 15508 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:43.959369 15285 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:43.921375 15242 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.959964 15285 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.962265 15285 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:43.964895 15242 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.957887 15372 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:43.880254  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:43.968465  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88308.5us
I20260709 12:28:43.954986 15215 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:43.969635 15411 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:43.904976 14906 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:43.970089 15516 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:43.970310 15411 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:43.970698 15516 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.971257 15516 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:43.970975 15384 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:43.972204 14906 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.972534 15502 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:28:43.927295  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.973304 15502 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:43.974653 15502 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.975898 15411 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.977298  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1240) took 1271 ms. Trace:
I20260709 12:28:43.977483 15411 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.977741  7806 rpcz_store.cc:276] 0709 12:28:42.706026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.706284 (+   258us) service_pool.cc:224] Handling call
0709 12:28:43.977115 (+1270831us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400295754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:43.977242 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147404672}
I20260709 12:28:43.978571  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:43.983052  8415 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:43.984557 15490 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:43.957888 15489 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.986047 15476 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:43.814023 15487 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:43.986474  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1514) took 2416 ms. Trace:
I20260709 12:28:43.912506 15396 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.986523 15476 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:43.856516 15424 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.989099 15476 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.935676  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:43.987254  8407 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:43.991010 15424 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:43.852941 15271 consensus_queue.cc:1243] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:28:43.990885  8413 rpcz_store.cc:276] 0709 12:28:41.570210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:41.575011 (+  4801us) service_pool.cc:224] Handling call
0709 12:28:41.599863 (+ 24852us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:41.691399 (+ 91536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:41.691444 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:41.708951 (+ 17507us) log.cc:844] Serialized 52 byte log entry
0709 12:28:42.024586 (+315635us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:42.024653 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:42.024760 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:43.116148 (+1091388us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:43.984442 (+868294us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b24002f53b8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffecbb9c4 00007fbffecb9acb 00007fbffee74dc9 00007fbffee74a55 00007fbffee73fb0 00007fc001aea488 00007fc001aeabe8 00007fc001ac2398 00007fc001abd76a 00007fc001abcbb6
0709 12:28:43.986202 (+  1760us) raft_consensus.cc:1685] finished
0709 12:28:43.986227 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:43.986404 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18840,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"spinlock_wait_cycles":174212608,"thread_start_us":274012,"threads_started":1,"wal-append.queue_time_us":426873}
I20260709 12:28:43.994836 15516 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:43.997102  8414 raft_consensus.cc:2368] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22.
W20260709 12:28:43.905036  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 156332us
I20260709 12:28:43.987008 15487 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:43.871848 15432 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.005227 15377 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:44.007735  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 182677us
I20260709 12:28:43.900007 14594 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.008603  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:44.015460  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57009us
W20260709 12:28:43.888063  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 218698us
I20260709 12:28:44.009866 15489 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:44.009932 15500 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:44.009472  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 155211us
I20260709 12:28:44.021049  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:43.818138 15274 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:28:43.874333  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76343.2us
I20260709 12:28:43.914149 15400 raft_consensus.cc:3055] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:44.023784  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.042567  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:44.041589  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.049079  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.050818  7654 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.051764  7653 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:44.052464 14594 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:44.054001 15271 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:44.054950  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.055265 15377 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.056205 15377 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.037366 15274 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:44.057227  7700 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.058154 15513 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.058291 15274 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.059386 15513 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:44.040032 15424 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:28:44.061605 15487 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.062194  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.855159 15507 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:44.063467 15488 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:44.064306  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:44.067190  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.070032  7730 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:28:44.070112  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.073313  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:43.855341 15322 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.068995  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:44.075536  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:44.076717  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.083065  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.079775 15513 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:28:44.086171  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.087682  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.088363  7661 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:44.086345  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:44.092909  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.036900 15489 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.094705  7809 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:44.037163 15521 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.100394  7736 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:44.100282  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.095829  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.100186  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.097640  8411 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:28:44.101703  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.084942  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.042222 15400 raft_consensus.cc:740] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.075171 15515 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:28:44.105348  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.105857  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.106125  7660 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:44.106576  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:44.099905  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.107632  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:44.108387  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.101395  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.104336  7807 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.109340  7805 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:44.109822  7662 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.108551  8413 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.107697  7657 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:44.115279  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.115459  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.115672  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.080806 15402 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:44.074931 15329 raft_consensus.cc:2764] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:28:44.074604 15396 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:44.076679 15277 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:44.141083  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3519) took 1306 ms. Trace:
W20260709 12:28:44.178696  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93886.4us
I20260709 12:28:44.209517  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.200459  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:44.141744  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.130167  7656 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.092123 15507 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.215745  7657 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.158346  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:44.218770  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:44.220461  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.221318  7806 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.096499 15520 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:44.139410  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:44.198479  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 141427us
I20260709 12:28:44.209151  7728 rpcz_store.cc:276] 0709 12:28:42.834734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.251545 (+416811us) service_pool.cc:224] Handling call
0709 12:28:43.274106 (+ 22561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:43.304551 (+ 30445us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:43.606581 (+302030us) log.cc:844] Serialized 14 byte log entry
0709 12:28:43.899841 (+293260us) log.cc:844] Serialized 14 byte log entry
0709 12:28:44.140411 (+240570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:44.140505 (+    94us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:44.140576 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.140813 (+   237us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:44.141000 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":396927,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":5383,"raft.queue_time_us":137199,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"thread_start_us":761544,"threads_started":3,"wal-append.queue_time_us":278187}
I20260709 12:28:44.208853 15513 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:44.225482 15513 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:44.225664  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:44.215940  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.226440 15456 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 12:28:44.227324  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:44.229319 15524 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:44.230937  7728 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:44.232996  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:44.238109  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:44.225996 15513 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:44.245177  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.248180  7730 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:44.224280 15400 consensus_queue.cc:260] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.250582  7736 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:44.213317  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.104887 15489 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.267266  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.270524  7812 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.265933  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.272725  7656 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.274170  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:44.276602  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63421.5us
I20260709 12:28:44.276983  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.121250 15512 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:44.278023  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68085.5us
I20260709 12:28:44.277964  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.218132 15396 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.292023  7809 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:28:44.158628 15486 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:44.298971 15486 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:44.300066 15486 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.301684 15486 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.291862 14594 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:44.292243 15520 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.303611 14594 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:44.304709 14594 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.210750 15402 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.292737  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.307032 15402 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.308144  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.309859  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:44.310978 15277 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.311587  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:44.312721  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.312582 15432 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:44.315752 15515 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:44.316179 15487 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.308352 15135 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:44.298398  7736 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:44.126955 15508 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.095388 15509 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.191s
I20260709 12:28:44.308111  8413 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:44.297216  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:44.322546  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44649.8us
I20260709 12:28:44.328161  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.331983 15372 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:44.332607  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.333596 15285 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:44.316737 15515 consensus_queue.cc:1243] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:44.211915 15377 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:28:44.339485  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.158360 15496 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:28:44.197504 15499 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.136s
I20260709 12:28:44.181178 15491 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:44.350075  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82965.4us
I20260709 12:28:44.347451  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.351076  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.352422  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.294550 14906 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:44.269758 15215 consensus_queue.cc:1059] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:28:44.319082 15396 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.266162 15271 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:28:44.354341  7662 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:44.398152  7806 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:44.376670  7809 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:44.338086 15135 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:28:44.115935 15521 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.339007 15242 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:44.353204 15516 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.386890 15491 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.403709 15377 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:44.415706 15377 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:44.416252 15377 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:44.417208 15377 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.405203 15277 raft_consensus.cc:2764] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:28:44.345319 15507 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.420607 15507 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:44.422695  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1149) took 1216 ms. Trace:
I20260709 12:28:44.423313 15242 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.423213  8407 rpcz_store.cc:276] 0709 12:28:43.206256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.206523 (+   267us) service_pool.cc:224] Handling call
0709 12:28:43.986930 (+780407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:43.990479 (+  3549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.422388 (+431909us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54005e0e50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:44.422589 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765747072}
I20260709 12:28:44.423898 15242 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.356778 15402 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.430419  7661 raft_consensus.cc:2410] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:28:44.433071 15521 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:44.432586 15322 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.436918  7807 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:28:44.444485  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 317069us
I20260709 12:28:44.449183  7659 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:44.430116 15500 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:44.455731  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.459743  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.460192 15193 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:44.461099  7657 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:44.461732 15040 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:44.462703  7812 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:44.444767 15402 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:44.463936 15489 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.466622  7661 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.447824 15322 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:44.466933  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:44.483744  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 267877us
I20260709 12:28:44.487201  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:44.492632  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.496220  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.473088 15040 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.468890 15193 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:44.515479  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 163140us
I20260709 12:28:44.502550 15491 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:44.535852 15400 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:44.540076 15040 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.541993 15040 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:44.544251  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 209036us
I20260709 12:28:44.545357  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.544896  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.547616 15277 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.517863 15193 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.524197 15513 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.509143 15488 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.548528 15521 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:44.528107 15399 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:44.532238 15285 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.472720 15500 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.545217 15525 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:44.549333  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.580279  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:44.595201  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47805.7us
I20260709 12:28:44.583027  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:44.597379  7730 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:44.597569  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.582635  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:44.505756  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:28:44.606098  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 100505us
W20260709 12:28:44.605997  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 258651us
I20260709 12:28:44.610661  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.615674  7736 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:28:44.616950  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.554952 15472 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:28:44.639829  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.640866  7734 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:44.567611 15193 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.552489 15489 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.645376  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.553021 15513 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.671595  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.617553 14906 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:44.673048 14906 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.673967 15491 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.618789 15488 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:44.675392 15488 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.593741 15242 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.622365 15285 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:28:44.655988  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.620981 15400 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:44.663316 15507 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:44.669692  7730 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:44.670449 15541 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.598695 15537 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:44.644088 15472 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:44.580729 15396 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.622193 15521 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.648154  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.703341  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.657171  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.703651  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.676260 15488 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.676424  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.706171  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:44.695631  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.696735  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.702069  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.675477 15538 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:44.707607  7661 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:44.707607  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:44.708120 15538 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.703907  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.708647 15538 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.709734  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.710438  8415 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:28:44.708259  7657 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:28:44.711074  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.713739  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.715891  7661 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:44.717090  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:44.701443  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:44.720572  7733 raft_consensus.cc:1240] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:44.722075  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:44.657874 15500 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.723708  7658 raft_consensus.cc:2368] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22.
I20260709 12:28:44.725451 15396 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.726553 15540 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.726951 15377 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:44.727952  7654 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:44.729089 15472 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.691602 15507 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:44.735409 15541 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:44.736093  7659 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.738592  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.741923  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.746955  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.749316  7654 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.739980 15507 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:44.746479  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:44.753189  7659 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:44.678097 15529 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 12:28:44.757419 15285 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:28:44.757889  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 127345us
I20260709 12:28:44.758170 15285 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:44.758864  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.759456 15544 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.759846 15513 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:44.761945  7660 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [19.13-20.14]
I20260709 12:28:44.740190 15521 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:44.767669  7654 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:44.760355  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:44.768463  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.772958  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.740505 15377 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.771572  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:44.775825  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.776366  7659 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:44.774500 15399 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:28:44.779676 15500 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:44.740469 15541 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:44.780634 15541 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:44.760017 15544 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.767298 15521 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.774503 15529 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:44.784266 15529 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.785504 15544 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:44.774668 15513 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:44.746939 15491 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:44.789930  7659 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:44.790650  7654 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:28:44.794701  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.802143  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.806602  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:44.809005  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.779203 15377 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:44.813619  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.793764 15500 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:44.818872  7777 leader_election.cc:400] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22.
I20260709 12:28:44.804097 15399 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:44.819723  7777 leader_election.cc:403] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:28:44.804098 15242 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.814488 15372 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:44.810011 15432 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.812032 15512 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:44.825807 15529 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.824602 15518 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.819828 15532 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:44.824810 15537 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:44.788416 15285 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.817960 15322 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:28:44.839951 15507 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.842639 15507 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:44.844058  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.827572 15488 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:44.855124  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.856945 15400 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:28:44.861792 15277 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:44.863109 15277 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:44.863721 15518 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:44.864089 15040 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:44.864229 15518 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.864718 15040 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.865981 15518 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.866189 15040 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.868028 15040 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:44.869131 15518 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:44.869938  7658 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 12:28:44.871872  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.822507 15500 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:44.868005 15400 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:44.875993 15400 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.876164 15500 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.873565 15488 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.872709 15537 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22.
I20260709 12:28:44.877197 15400 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.880208 15400 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.868412 15193 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.880900 15193 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.882848  7659 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:44.883348 15193 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.846061 15456 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:28:44.908214 15538 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:44.923683  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 181856us
I20260709 12:28:44.899937 15372 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:44.917404 15193 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:44.940667 15538 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:44.933815 15496 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:28:44.933621 15532 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:44.945142  7778 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.511s (SENT) [suppressed 14 similar messages]
I20260709 12:28:44.945482 15532 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.928089 15507 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:44.938589 15541 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:44.928522  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:44.962744  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.966740  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.974632  7810 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:44.977007  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:44.978111  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.979522  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.976460  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3386) took 2144 ms. Trace:
I20260709 12:28:44.960325 15538 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.987974  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:44.989754  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.965870 15521 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:44.960446 15526 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:44.998145  7734 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:45.000689  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.001626  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:44.969066 15329 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:44.950929 15193 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:44.981050  7731 rpcz_store.cc:276] 0709 12:28:42.832427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.251647 (+419220us) service_pool.cc:224] Handling call
0709 12:28:43.264331 (+ 12684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:43.281596 (+ 17265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:43.281630 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:43.288086 (+  6456us) log.cc:844] Serialized 26 byte log entry
0709 12:28:43.464169 (+176083us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:43.724571 (+260402us) log.cc:844] Serialized 14 byte log entry
0709 12:28:44.891314 (+1166743us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b18003905e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:44.966251 (+ 74937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.966355 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:44.966459 (+   104us) raft_consensus.cc:1685] finished
0709 12:28:44.976196 (+  9737us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:44.976385 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16811,"prepare.queue_time_us":239049,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":111671,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":255,"spinlock_wait_cycles":12289280,"thread_start_us":452996,"threads_started":3,"wal-append.queue_time_us":350040,"wal-append.run_cpu_time_us":3253,"wal-append.run_wall_time_us":1152439}
W20260709 12:28:45.010399  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.019474  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.021960  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:44.983285 15512 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:44.964432 15507 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.024335  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.031731  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:45.005491 15399 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:45.054323  7662 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:45.054360  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:45.006290 15538 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:45.060007  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 312658us
I20260709 12:28:45.028657 15513 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:45.020479 15521 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:45.034188 15285 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:45.065366  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.066551  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:45.034967 15557 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:45.067960  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:45.110450  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.110337  7657 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:45.075788  8414 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:28:45.064050 15532 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.107731  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:45.108371  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:45.061566 15565 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.109251  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.110865  8407 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
W20260709 12:28:45.076297 15561 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:28:45.116010  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.119553  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.117367  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:45.125510  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45674.6us
I20260709 12:28:45.125684  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:45.124946  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.037657 15372 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.131332  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.131927  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.132329  7731 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:45.133488  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.133548  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.134443  7657 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:45.135963  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.134270  7731 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.137042  7736 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:45.138130  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.139433  7735 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:45.140727  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.141453  7736 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:45.142647  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3521) took 2210 ms. Trace:
I20260709 12:28:45.143185  7729 rpcz_store.cc:276] 0709 12:28:42.932160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.267664 (+335504us) service_pool.cc:224] Handling call
0709 12:28:43.281932 (+ 14268us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:43.291688 (+  9756us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:43.523510 (+231822us) log.cc:844] Serialized 14 byte log entry
0709 12:28:44.822602 (+1299092us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b1800228580. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:28:44.822817 (+   215us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:44.824730 (+  1913us) log.cc:844] Serialized 52 byte log entry
0709 12:28:44.957877 (+133147us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:44.957937 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.958040 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:45.142259 (+184219us) raft_consensus.cc:1685] finished
0709 12:28:45.142374 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:45.142563 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9140,"prepare.queue_time_us":362027,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2585,"raft.queue_time_us":10917,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":4219,"spinlock_wait_cycles":8767872,"thread_start_us":457301,"threads_started":3,"wal-append.queue_time_us":499377,"wal-append.run_cpu_time_us":1322,"wal-append.run_wall_time_us":1001350}
I20260709 12:28:45.105172 15242 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:45.145875 15242 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.147979  7731 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:45.152227  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.153700  7737 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:45.106248 15525 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:45.164391 15560 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 12:28:45.165627 15559 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:28:45.120976 15285 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.119550 15540 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:45.172576  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.175729  7736 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:45.171728 15521 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.194636 15285 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.195716 15285 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:45.196573 15285 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:45.196839  7656 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:45.197139 15285 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:45.125512 15565 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.198346  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:45.198096 15285 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.199672 15565 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.200591 15377 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.204268  7661 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [19.13-20.14]
I20260709 12:28:45.119870 15557 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.210374 15557 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.211089 15525 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.106580 15394 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
W20260709 12:28:45.227166  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 282051us
I20260709 12:28:45.203037 15377 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.234807  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:45.223146  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 105867us
I20260709 12:28:45.231212  7736 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 12:28:45.236845  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.236694  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:45.238404  7731 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:45.241346  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 114261us
I20260709 12:28:45.242560  7735 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:45.244199 15538 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.227864 15498 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
W20260709 12:28:45.245587  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.246738 15193 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.246986 15525 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.247929 15502 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:45.234418 15394 consensus_queue.cc:1243] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:28:45.229961 15541 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.252945  7659 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:45.253763  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.258206  7737 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:45.260229  7729 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:45.261545  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.263070  7728 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:45.255863  7658 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:28:45.265381  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:45.265262 15277 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.269501 15277 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.272258 15277 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:45.281718  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:45.297312  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.297648  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:45.298249 15581 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:45.262301 15396 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.306351  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:45.307389 15506 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:45.307844 15506 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:45.308256 15506 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:45.308511 15424 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:45.309247  7728 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:45.309094 15424 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.313097  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.313882  7728 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:45.313177 15424 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.240139 15496 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:28:45.293967 15501 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:45.292085 15277 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:45.331032  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:45.342350  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.342890  7653 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:45.326691 15507 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.263603 15572 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:28:45.305301 15558 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:28:45.331276 15501 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:45.371115 15570 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:28:45.319698 15396 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.381620 15396 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:45.387820 15577 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 12:28:45.358982 15514 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.373361 15501 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:45.364708 15040 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:45.375844 15285 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:45.409051 15400 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:45.390684 15514 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.465721 15583 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:45.466421 15583 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:45.467767 15514 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.470150  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.471035  7728 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:45.405404 15285 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:45.472456 15285 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.474126 15285 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.476804  7653 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
W20260709 12:28:45.586570  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.590165  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:45.592082  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:45.593182  8414 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:45.614806  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:45.477628 15400 consensus_queue.cc:1243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:45.626366  8407 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 12:28:45.651818  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.657622  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:45.522284 15514 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:45.662166  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 320038us
I20260709 12:28:45.585260 15396 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.477037 15506 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:45.677161  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 333871us
I20260709 12:28:45.676542  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:45.626638 15524 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:28:45.663532 15277 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:45.708835  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:45.769804  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.664261 15584 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
W20260709 12:28:45.793849  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 96084.7us
I20260709 12:28:45.806968 15277 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.681722 15514 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.804085 15432 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:45.811134  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 78847.2us
I20260709 12:28:45.687537 15396 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.826351 15598 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:45.831843 15524 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:28:45.804628 15566 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:28:45.839169 15598 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.840975 15524 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.830978 15600 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.843588 15600 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:45.833977  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.848155  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3539) took 1278 ms. Trace:
I20260709 12:28:45.848563  8414 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:28:45.829598 15506 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.647392 15583 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.848973  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.852510  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.848678  7733 rpcz_store.cc:276] 0709 12:28:44.569407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.580592 (+ 11185us) service_pool.cc:224] Handling call
0709 12:28:44.580969 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:44.721072 (+140103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:45.847879 (+1126807us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002486d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:45.848089 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320369024}
I20260709 12:28:45.853468  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:45.871325  7662 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:45.873692 15588 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:45.876226  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76137.1us
I20260709 12:28:45.877641  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:45.878934  7733 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:45.848435 15432 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.880538 15432 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.881023 15537 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:45.834714 15372 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:45.849341 15502 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.005s
I20260709 12:28:45.855084 15524 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.884017 15597 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.884630 15597 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.887138 15456 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:45.883661 15372 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:45.852196 15598 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.834841 15215 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:45.878764  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:45.892894  7653 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [19.13-20.14]
I20260709 12:28:45.903101  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.904170  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:45.903905  7660 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:45.912284 15588 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:45.912526 15242 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:45.887207 15597 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:45.914558 15537 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:45.915544 15524 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:45.917665 15475 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:45.895641 15274 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:45.849997  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:45.919448  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:45.924021  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.907596  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:45.929527  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:28:45.920953  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:45.934954 15502 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:45.935972 15502 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:45.937353 15502 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.900761 15558 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.895861 15583 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:45.945677  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 95759.9us
I20260709 12:28:45.948922  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:45.950346  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.951273  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.950486  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:45.952248  7807 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:45.960008  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.893929 15372 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:45.961359  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.892426 15514 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.900981 15600 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:45.899574 15215 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:45.969588  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.924994 15476 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:45.971067  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.978021  7812 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:45.954907 15242 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:45.973255  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:45.969121  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:45.981092  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:45.984480  7807 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:45.887844 15456 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.955850 15592 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.956159 15585 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:45.988584  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.989228  7806 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:45.939839 15558 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:45.963970 15518 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:45.988065  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.996696  7807 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:28:45.997033  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.003743  7661 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:45.996472 15518 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.033751  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.013832 15591 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:46.037006  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.052891  7807 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:46.047737 15399 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.001753 15372 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:28:46.054042  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:46.052350 15489 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:46.059753  7775 leader_election.cc:400] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22.
I20260709 12:28:46.060639 15322 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:46.118742  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 59209.7us
I20260709 12:28:46.069694 15432 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.073412 15456 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.076453 15396 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:46.077800 15501 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.006s
I20260709 12:28:46.079968  7809 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:28:46.034685 15518 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:46.126267  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1243) took 2820 ms (client timeout 1762 ms). Trace:
I20260709 12:28:46.131527  8407 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:28:46.131562  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.137903  7661 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:46.134310  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:45.996873 15592 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.140349  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:46.115091 15399 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:46.135555  7804 rpcz_store.cc:269] 0709 12:28:43.298933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.309176 (+ 10243us) service_pool.cc:224] Handling call
0709 12:28:46.099510 (+2790334us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54001324d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.118997 (+ 19487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190513536}
I20260709 12:28:46.145254  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.150620  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.153434  7809 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:46.118220 15489 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:46.157399  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.158280  7809 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:46.122449 15322 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.122638 15575 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.159186  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.161350  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.161403  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.163168  8412 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:46.165653  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.167768  7809 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:28:46.168907  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.172245  7805 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:46.149189 15508 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.178054  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.154879 15587 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:46.183471  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 202456us
I20260709 12:28:46.184723  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.186975  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.188318  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.192088  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.119167 15215 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.178566 14906 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.141829 15501 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.199882  7803 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:46.184031 15572 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:46.219262 15572 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.220280 15472 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:46.219980 15572 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.222429  7811 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:46.190798 15591 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:46.185616 15587 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:46.192378 15242 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:46.235702  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.175655 15575 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.236678  7811 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:46.236768 15242 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.238214 15322 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.184034 15508 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:46.240106 15285 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.765s	user 0.015s	sys 0.000s
W20260709 12:28:46.201782 15537 consensus_queue.cc:1175] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 18 status { last_received { term: 17 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:46.235785 15587 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.196988 15489 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.199927 14906 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.215626 15423 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.216483 15581 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:46.218052 15329 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.240274 15572 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.215042  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:46.247136 15529 consensus_queue.cc:1175] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" responder_term: 20 status { last_received { term: 20 index: 17 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:46.247294  8409 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:46.252071  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.253347  8409 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:46.247609  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.248209 15581 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.248936  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.256948  8407 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:46.248526 15329 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:46.241951  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:46.258522  7661 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:46.246069 15423 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:46.201838 15215 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:46.218299 15608 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:46.248936  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.263245  7810 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:46.257280 15581 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.270169 15581 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.243639 15489 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:46.256033  7804 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:46.276120  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91482.6us
I20260709 12:28:46.277657 15607 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:46.279559 15040 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:46.280145 15040 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.281263 15040 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.281812 15371 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:46.282344 15371 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.282826 15371 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.293267  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.301188  7812 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:28:46.302284  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.306052  7812 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:28:46.307413  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.300738 15322 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:46.305603 15271 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:46.333843 15575 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.336588 15424 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:46.304566 15605 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:28:46.339468  8406 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:28:46.317765 15040 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.340251 15567 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.333902  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.347685  7812 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:46.289904 15584 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:28:46.337226 15424 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:46.303998 15496 consensus_queue.cc:1059] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:28:46.346570  7807 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:28:46.312424 15432 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.355544  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.361032 15424 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.341075 15271 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.395874 15271 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.383731  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.351257  7661 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:46.382362  8408 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
W20260709 12:28:46.367316  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 180424us
I20260709 12:28:46.348506 15508 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.401916  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:46.371135 15584 consensus_queue.cc:1243] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:46.379634 15496 raft_consensus.cc:3055] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:46.399591 15613 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.358899 14906 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.405956  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.414368  7728 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:46.414485  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.403921 15490 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.415468  7804 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:46.391521 15592 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.415788  7810 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:46.418001  7812 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:46.419749  7734 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:28:46.423734  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58757.8us
I20260709 12:28:46.427129  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.428107  7810 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:46.428927  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.434038  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.425580 15588 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.435297  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.441694  7803 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:46.435568 15507 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:46.436496 15558 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:46.437450 15274 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.507s	user 0.013s	sys 0.002s
I20260709 12:28:46.410723 15496 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.440817  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.443118  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.443186  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.413113 15277 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:46.444077  7807 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:46.354004 15567 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.447530  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.447818  7812 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:46.456513  7810 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:46.416498 15424 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:46.441766  7728 raft_consensus.cc:2376] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21. Re-sending same reply.
I20260709 12:28:46.451954  7733 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:46.446753  7737 pending_rounds.cc:85] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 13
I20260709 12:28:46.459631  7737 pending_rounds.cc:107] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:28:46.460006  7737 raft_consensus.cc:2889] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:46.452888  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.421514 15584 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.453946  8415 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:28:46.448035  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.455683  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.460636  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.463157  7729 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:46.462416  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.464108  7804 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:46.438987 15399 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.470202  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.473922 15518 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.435652 15588 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:46.478581  8406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1156) took 1761 ms (client timeout 1703 ms). Trace:
I20260709 12:28:46.423435 15490 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.480778  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:46.481576  8406 rpcz_store.cc:269] 0709 12:28:44.710331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.716778 (+  6447us) service_pool.cc:224] Handling call
0709 12:28:46.471751 (+1754973us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.471844 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3729323904}
I20260709 12:28:46.479689  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.463037 15277 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.484051  7729 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:46.484338  7810 raft_consensus.cc:2376] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19. Re-sending same reply.
I20260709 12:28:46.465910 15193 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.467947 15472 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:46.485183  7656 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:46.422654 15613 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.455209 15507 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.469254 15584 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:46.533349  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.542493  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.542785  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.543469  7733 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:46.484813  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.544507  7807 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:28:46.486249  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.528044  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.545703  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.546591  7810 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:46.485574  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:46.551041  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.551855  7811 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:46.555058 15432 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:28:46.555425 15277 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:46.556632 15621 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:46.559505  7804 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:28:46.550758 15472 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.563970 15472 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.564158  7657 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:46.497738 15583 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.557s	user 0.015s	sys 0.002s
I20260709 12:28:46.563293  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3113) took 1512 ms. Trace:
I20260709 12:28:46.571650  7654 rpcz_store.cc:276] 0709 12:28:45.050824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.051603 (+   779us) service_pool.cc:224] Handling call
0709 12:28:45.052082 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:45.052426 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:45.052469 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:46.242693 (+1190224us) spinlock_profiling.cc:243] Waited 9.64 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:46.243200 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:28:46.416396 (+173196us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:46.416518 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.416640 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:46.562777 (+146137us) raft_consensus.cc:1685] finished
0709 12:28:46.562817 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.563169 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22176128,"thread_start_us":155089,"threads_started":1,"wal-append.queue_time_us":219728}
I20260709 12:28:46.575557 15530 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.576540 15530 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:46.582796 15622 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:28:46.583750 15518 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.588587 15498 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:46.589870 15399 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.591430  7809 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:28:46.598819  7805 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:46.612423  7734 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:46.578943 15530 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.617097  7809 pending_rounds.cc:85] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 13
I20260709 12:28:46.511189 15513 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.620311  7809 pending_rounds.cc:107] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:28:46.620707  7809 raft_consensus.cc:2889] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:46.515878 15604 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:46.625103 15513 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.630012 15513 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.631536  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.632561  7803 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:46.633507  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.552639 15193 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.635843 15613 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.638412 15585 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:46.638934 15585 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.640413 15507 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.641289  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.645503  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.646333  7729 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:46.643734 15193 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.643386 15271 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.617107 15498 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:46.654574  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.656669  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.654819  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.657974  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.659309  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.659965  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:46.588655 15329 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.671849 15193 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:46.672446 15193 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.672876 15193 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:46.673954 15329 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.677392 15587 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.681676 15498 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.683094 15498 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.661918 15371 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.692965  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:46.695932  8406 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:46.671929  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.697319  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.699606  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.700337  7729 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.703441  7735 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:46.701884  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:46.708994  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 589263us
I20260709 12:28:46.709761 14594 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.710316 14594 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.712355 15491 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:46.713096 15491 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:46.712615  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.715407 15530 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.715741 15399 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.716096  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.715557 15491 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.716383 15532 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.716097 15530 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.717191 15532 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.717728  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.719250 15530 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.719612 15489 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:46.720428 15489 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:46.716439 15242 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:46.721279 15567 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.721400  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.721450 15242 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.722431 15242 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:46.703569 15507 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:28:46.723667  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.724526 15530 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:46.725358 15530 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:46.725615 15588 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.726912 15530 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:46.512567 15496 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.705888  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.728423  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.730264  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.703856  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:46.715471  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:46.741983  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1162) took 1152 ms. Trace:
I20260709 12:28:46.742465  8408 rpcz_store.cc:276] 0709 12:28:45.590028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.616222 (+ 26194us) service_pool.cc:224] Handling call
0709 12:28:46.741753 (+1125531us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540046cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.741903 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663756160}
I20260709 12:28:46.743151  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.742477 14594 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.748068  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1161) took 1158 ms. Trace:
W20260709 12:28:46.750139  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46358.5us
I20260709 12:28:46.748588  8415 rpcz_store.cc:276] 0709 12:28:45.589710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.589845 (+   135us) service_pool.cc:224] Handling call
0709 12:28:46.747856 (+1158011us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.747993 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984061824}
I20260709 12:28:46.756848  8413 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:28:46.757232  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.758155  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.759845  7654 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:46.762827  7656 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:46.765758  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.768498  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.769447  7731 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:46.700006 15526 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:46.770884  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.771795  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.781175  7736 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 15.
I20260709 12:28:46.693815 15371 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.773598  7735 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:28:46.779596  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.758711 15329 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:46.759851 15594 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:46.787310  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.784417 14594 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:46.788339  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:46.788373  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.740319 15491 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:46.790089  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.788225  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.791424 15575 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:46.792415 15500 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:46.783910 15615 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:46.793825  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.784162 15396 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:46.843748 15396 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.833696  7729 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:46.757616 15394 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.795297  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:46.795977 15520 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:46.788888  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.800213  7804 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:46.753230 15399 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.804431 15472 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:46.804908  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84564.7us
I20260709 12:28:46.807377 15532 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.760236 15507 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:46.774039  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.811297  7737 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:28:46.815336  7654 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:46.783119  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.810304 15329 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:46.818257  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.817540 14594 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:46.820441  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:46.825846  7700 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.801s (SENT)
I20260709 12:28:46.817533  7653 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:46.703078 15537 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.087s
I20260709 12:28:46.772732 15513 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:46.837257  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.839948  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.847441  7735 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:46.881757  7804 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:28:46.848814  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53762.6us
I20260709 12:28:46.882344 15501 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.847838 15394 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.885509 15394 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.847067 15396 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:46.886607 15623 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:46.819801 15594 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:46.887319 15614 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.449s	user 0.003s	sys 0.006s
I20260709 12:28:46.887544  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:46.888456  8408 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:46.887636 15594 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:46.890457 15496 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:46.850379 15520 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:46.892839 15520 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.893239 15520 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:46.853677 15399 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.856539 15472 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:46.896154  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:46.897004  7809 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
W20260709 12:28:46.855448  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:46.899513  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.899348  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.857106  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:46.901600  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44595.6us
I20260709 12:28:46.902590  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.903301  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.903688  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.904074  8409 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:46.904412 15500 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:46.857954  7810 raft_consensus.cc:2376] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
I20260709 12:28:46.905756 15491 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:46.859727 14594 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:46.906396 15491 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:46.906862 14594 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.907119 15584 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.860080  7732 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.908273  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.908967  8405 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:46.857681 15507 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:46.910305 15507 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.910455  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.860256  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:46.905102 15500 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:46.921197  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.921432 15513 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:46.921792  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1436) took 1806 ms. Trace:
I20260709 12:28:46.921977 15513 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:46.921242  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61111.9us
I20260709 12:28:46.923595  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.856559  7731 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 12:28:46.911639  8413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1154) took 2208 ms (client timeout 1801 ms). Trace:
I20260709 12:28:46.862630 15490 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:28:46.862933 15606 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.866377  8415 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:28:46.872006 15613 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:46.860977  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.913757  8411 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:28:46.874085 15587 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:46.932209  7662 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:46.859154 15329 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:46.932629 15604 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:46.933202 15604 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:46.933580  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.925910  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.923118  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.935675  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.935951  7654 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:46.935962  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1158) took 2223 ms. Trace:
I20260709 12:28:46.930928 15606 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.873284 15271 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:46.937286  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:46.871802 15585 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.914812  7661 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:46.938167 15585 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.938812 15606 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:46.940505  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.921820 15500 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.918481  7653 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:46.920274  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.936838  8411 rpcz_store.cc:276] 0709 12:28:44.712396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.712729 (+   333us) service_pool.cc:224] Handling call
0709 12:28:46.913322 (+2200593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.932049 (+ 18727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.935743 (+  3694us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54000c76d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.935898 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5060116352}
I20260709 12:28:46.943743  7658 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:28:46.944666  7653 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:46.946619  7658 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:46.924471  7659 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:46.924844  8413 rpcz_store.cc:269] 0709 12:28:44.703582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.707369 (+  3787us) service_pool.cc:224] Handling call
0709 12:28:46.911351 (+2203982us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54000c76d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.911517 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4596976000}
I20260709 12:28:46.861982 15575 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:46.934954  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:46.950021  7662 raft_consensus.cc:3060] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.922273  7657 rpcz_store.cc:276] 0709 12:28:45.115841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.199384 (+ 83543us) service_pool.cc:224] Handling call
0709 12:28:46.563937 (+1364553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.564663 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.921565 (+356902us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540028df50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:46.921718 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137431296}
I20260709 12:28:46.928148  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.928994  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.952011  8405 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:28:46.907451 14594 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:46.895325 15615 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:46.929944 15399 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:46.930392  7656 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:46.931483  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:46.962109  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:46.962451  8408 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:46.933117  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.875423  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.936048  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:46.963006  7807 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:46.851075  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 62305.2us
I20260709 12:28:46.964057  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:46.965095  7658 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.926004  7655 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:46.948890 15575 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:46.922520 15513 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:46.905063  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:46.938942  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.953123  7700 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:46.953704  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:46.955032  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.913197  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:46.957238  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:46.959317  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49033.4us
I20260709 12:28:46.935699  7737 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:46.966476  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.968420  7732 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:46.978012  7656 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:28:47.045636  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.046449  7653 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:47.046785  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:46.986307 14594 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.045636  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.049714  7661 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:28:47.050225  7656 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 13
I20260709 12:28:47.050884  7656 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:28:47.051460  7656 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:47.051965  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 115273us
I20260709 12:28:47.052951  7659 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
W20260709 12:28:46.986713  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 111376us
W20260709 12:28:47.053458  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 100499us
I20260709 12:28:47.054278  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:47.049518  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:46.986064  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:47.040930  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:47.051753  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 118858us
I20260709 12:28:47.058391  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.058630  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.063038  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:28:47.048967  7810 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:47.054461  7735 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:47.045728 15285 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:28:47.047739 15432 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:47.076308  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:46.960917 15613 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:47.063796  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:47.081911  7656 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:47.052789  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:47.079305 14594 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.083771  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.083694 14594 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:46.985764 15271 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:47.087787  7810 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:28:47.082386  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:46.922457 15242 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.086297  7812 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:47.082082  7653 raft_consensus.cc:2410] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:28:47.057921 15396 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.064121 15532 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:47.063651 15530 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:47.095124  7662 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:46.986346  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:47.065212 15489 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:47.076447 15501 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:47.052042 15526 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:28:46.965358 15615 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.096060  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:47.070751 15507 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.111811 14594 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.084420 15605 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:46.907487 15491 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.081677 15613 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.088009 15271 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.135840 14594 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.094048 15040 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:47.110450 15606 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:47.084101 15581 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:28:47.110900 15530 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.143081 15399 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:47.151180 14594 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.091821 15537 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:28:47.092026 15588 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:28:47.152235 15630 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:47.160390 15575 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.167702  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.168557  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.169116  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.288363  7809 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:47.175380  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:47.141758 15587 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:47.290064 15587 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.290601 15587 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:47.142093  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:47.123308  7737 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:47.110360 15285 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
W20260709 12:28:47.296875  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.142040 15501 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:47.298630 15501 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.106638 15396 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.302211 15396 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.124092 15532 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:47.193969  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.286059  8412 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:47.304162 15532 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.149879 15613 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:47.288226  7806 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:28:47.287426  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 297771us
I20260709 12:28:47.153952 15040 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:47.127744 15500 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:28:47.154093 15271 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:47.341480 15594 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.320907  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.322362 14594 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.170395 15605 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:47.284262 15584 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.343559 15584 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.343724 15605 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.346051 15584 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.172782 15606 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.346854  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.347703 15606 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:47.194566 15530 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:47.196995 15193 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.353717 15193 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.355309 15507 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.332283 15607 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.368016 15607 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.370675 15607 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.272279 15597 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:28:47.310427 15630 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.377127 15615 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.378325 15630 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.379462 15432 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.343079  8405 raft_consensus.cc:2368] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
W20260709 12:28:47.384609  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.327106  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:47.287196 15399 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.177443 15526 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:47.272454 15592 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.137s
I20260709 12:28:47.337685  8412 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:28:47.319919 15384 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:47.309779  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 167763us
I20260709 12:28:47.342223 15242 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.328996 15040 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:47.386096  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59111.3us
I20260709 12:28:47.334501  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.416795  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:47.419641 14594 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:47.502468  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 445082us
I20260709 12:28:47.380326 15432 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.501221  7810 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:47.411672 15584 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:47.392032 15399 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:47.516288 14594 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.501960  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:47.392880 15193 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:47.523120  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 106448us
I20260709 12:28:47.531955  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:47.532927  7658 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.002s
I20260709 12:28:47.538249  7658 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
W20260709 12:28:47.522375  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 469626us
I20260709 12:28:47.538347  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:47.326993 15329 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.503509 15498 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:28:47.547401  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.589291  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.592291 14594 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.594712 14594 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:47.428491 15384 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:47.540472 15604 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.001s
I20260709 12:28:47.535969 15490 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:47.597294  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.599575 14594 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.533303 15584 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.547428 15472 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:47.540092 15193 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:47.611640  7656 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 12:28:47.554431 15489 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:47.612651  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 108399us
W20260709 12:28:47.613538  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.017s	sys 0.040s
W20260709 12:28:47.624517  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.018s	sys 0.040s
I20260709 12:28:47.620765  8411 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:47.568717 15498 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:47.528287 15597 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:28:47.617790  7656 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:28:47.624648 14594 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.625540  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:47.619334 15513 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.004s
I20260709 12:28:47.633370 15513 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.616855 15584 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:47.618450 15193 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:47.647514 15432 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.650188 15329 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.650640 15384 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.651544 15594 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.642680 15394 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:47.675750 15394 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:47.676837  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.592334  7551 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.625547 15490 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:47.660239  7553 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 306) took 52648 ms (client timeout 15000 ms). Trace:
I20260709 12:28:47.614588 15271 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.635615 15640 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:47.683858 15583 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:47.687145  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.011s	sys 0.036s
W20260709 12:28:47.687623  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.011s	sys 0.036s
W20260709 12:28:47.688211  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 701894us
I20260709 12:28:47.691483  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.693945  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:47.601081 15604 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.697350  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:47.671479 15496 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.780s	user 0.012s	sys 0.006s
I20260709 12:28:47.629938 15530 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:47.654896 15520 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.761s	user 0.017s	sys 0.000s
I20260709 12:28:47.664543  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:47.692476  7656 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
I20260709 12:28:47.728688  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:47.730171  7804 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:47.706861 15585 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.768s	user 0.007s	sys 0.007s
I20260709 12:28:47.695312 15581 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.730651 15520 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.731192 15500 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:47.731455 15581 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:47.732563  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68953.2us
I20260709 12:28:47.733368 15520 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.733870 15581 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.734377  7654 raft_consensus.cc:2368] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:28:47.736645 15271 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.706009 15592 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:47.737943  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.738961  7660 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 22 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:28:47.684103 15640 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.684918 15607 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.677109 15394 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.712652  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:47.715848  7661 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:28:47.718225  8408 raft_consensus.cc:2368] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23.
I20260709 12:28:47.726157  7807 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:47.728675  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:47.733608 15500 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:47.684486 15583 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:47.708657 15490 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.740944  7662 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:47.742976  7804 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:47.744413  7808 raft_consensus.cc:2410] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:28:47.744261  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.745002  8413 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:47.717118 15641 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:28:47.750662 15604 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.767721  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:47.803381  7653 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:47.770201  7658 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:47.738543  7551 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 18, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 18 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:47.807372  7775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.018s
I20260709 12:28:47.777419  7655 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:47.807575  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2945) took 1181 ms. Trace:
I20260709 12:28:47.747879 15489 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.808111  7808 rpcz_store.cc:276] 0709 12:28:46.626670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.640975 (+ 14305us) service_pool.cc:224] Handling call
0709 12:28:47.804399 (+1163424us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001b1f54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:47.807502 (+  3103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2517147008}
W20260709 12:28:47.807842  7775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.011s	sys 0.018s
I20260709 12:28:47.814350  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:47.774969  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:47.654881 15587 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:47.809206  7700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.012s	sys 0.019s
I20260709 12:28:47.816306  7658 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:47.817032  7700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.012s	sys 0.019s
I20260709 12:28:47.771538  7803 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
W20260709 12:28:47.817561  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 762880us
I20260709 12:28:47.768244 15513 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:47.765431 15530 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.757316 15606 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.852785 15242 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:47.853619 15242 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.860528  7736 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:47.861739  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:47.863770  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:47.860441  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 303462us
I20260709 12:28:47.870388  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:47.793691 15607 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.925789  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:47.926997  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.831375 15520 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.849555 15040 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:47.943667 15520 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.945225 15587 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.946044 15520 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.946274 15040 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:47.930101  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:47.942221 15193 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.966569  8409 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:47.931289  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65804.1us
I20260709 12:28:47.932231 15432 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.932809  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3500) took 1162 ms. Trace:
I20260709 12:28:47.933807 15242 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.940790 15607 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:47.928928  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.947572  7732 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:47.953498  8405 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:47.969475  7803 rpcz_store.cc:276] 0709 12:28:46.770505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.771010 (+   505us) service_pool.cc:224] Handling call
0709 12:28:47.932568 (+1161558us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003eb7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:47.932723 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116248960}
I20260709 12:28:47.977406  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:47.979596  8415 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:47.979674  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:47.973683  8412 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:47.992148  7732 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:28:47.755126 15498 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.994937  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:47.999095  8379 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.005915 15642 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:47.972553 15605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.008s	sys 0.018s
I20260709 12:28:47.955320 15606 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.017081  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:48.012847 15193 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:48.012619 15605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.009s	sys 0.018s
W20260709 12:28:48.035881  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 410424us
I20260709 12:28:47.925073 15583 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.014204 15513 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:48.019320 15193 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:48.024534 15613 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.003s
I20260709 12:28:48.041849  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:48.042527  8409 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:48.048080  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:48.056206  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:48.068894  7662 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:48.028874 15641 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:48.038941  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 500682us
I20260709 12:28:48.073558  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:47.982554 15432 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.031953 15498 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:48.076846  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.077638  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.033713 15271 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:48.032150 15650 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:48.087543  7803 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:48.087489  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:48.104338  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:48.105038  7807 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:48.105952  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:48.106540  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:48.087642  7736 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 15.
I20260709 12:28:48.088718  7653 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:48.088881  7737 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:48.090231  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:48.090265  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:48.090847  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:48.111948  7811 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:48.091938  7804 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:48.038496 15520 raft_consensus.cc:2764] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:28:48.110313  7777 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:48.118556  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.120795  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.121887  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:48.077744 15193 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:48.139693  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:48.125361  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:48.142027  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:48.142287  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:48.143334  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.144059  7730 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.085750 15597 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.084549 15583 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:48.097271 15277 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.102501 15650 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.083739 15513 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:48.015615 15642 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:47.946521  7551 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 13, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 13 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.101104 15500 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.172174  7777 leader_election.cc:400] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:28:48.107991 15587 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.114296 15537 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:48.160553 15532 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.003s	sys 0.016s
W20260709 12:28:48.186137  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.019s	sys 0.036s
W20260709 12:28:48.212404  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.020s	sys 0.036s
W20260709 12:28:48.171690 15472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.032s
I20260709 12:28:48.191841  7777 leader_election.cc:403] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:28:48.079164 15613 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.163105 15592 raft_consensus.cc:2804] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:48.161320 15581 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:48.086971 15641 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:48.201478  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 507199us
I20260709 12:28:48.203145  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:48.100234 15644 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:28:48.295307  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.207634  7811 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:48.295852 14906 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.296221  7731 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.168753 15606 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:48.173247 15285 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.296561 14906 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.297173 15285 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.209460 15587 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.299346 14906 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:48.216732  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44745.2us
I20260709 12:28:48.184113 15652 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:48.300359 15587 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.300647 15652 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:48.301188 15652 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.221381  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.303041  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:48.223093  7730 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:48.223452  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.304893  7735 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:48.224632  7733 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.183205 15639 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:48.306867 15639 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.309468 15639 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:48.201203 15501 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.040s	sys 0.058s
W20260709 12:28:48.310194 15501 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.040s	sys 0.058s
I20260709 12:28:48.150056  7737 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:28:48.310989  8407 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:28:48.184368 15650 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:48.314677  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:48.164984 15585 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:28:48.315665  7731 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:48.190116 15588 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:48.316569  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:48.316941 15588 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:48.317555 15588 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:48.251169  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 250822us
I20260709 12:28:48.319968 15520 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:48.320761 15520 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:48.321009  8379 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.322054 15520 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.161547 15645 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:28:48.194962 15638 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:48.212337 15532 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.017s
I20260709 12:28:48.174018 15597 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.278939  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:48.304275 15648 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:28:48.128741 15647 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.110s
W20260709 12:28:48.216094 15472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.033s
I20260709 12:28:48.228008 15040 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.244940 15646 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.217342 15607 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.176465 15277 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.277046  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.246917 15637 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:48.209067 15500 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.347407  7728 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.371133  7804 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:48.325568  7551 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:48.355217 15472 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:48.375191  7551 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.377650  7551 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 10 to 12, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:48.366055 15676 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:48.379773 15676 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.380193 15676 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:47.682519  7553 rpcz_store.cc:269] 0709 12:27:54.993289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.993737 (+   448us) service_pool.cc:224] Handling call
0709 12:28:38.252122 (+43258385us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400234a28. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00588c161 00007fc0059163af 00007fc005917b11 00007fc007e17fa1 00007fc007e187ff 00007fc007cd7a1b 00007fc007ddf90e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc
0709 12:28:47.631525 (+9379403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:38.905551 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:38.906579 (+  1028us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:38.906660 (+    81us) write_op.cc:435] Acquired schema lock
0709 12:28:38.906717 (+    57us) tablet.cc:662] Decoding operations
0709 12:28:39.130612 (+223895us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:39.130778 (+   166us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:39.130821 (+    43us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:28:39.549487 (+418666us) tablet.cc:701] Row locks acquired
0709 12:28:39.549554 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:28:39.549859 (+   305us) write_op.cc:270] Start()
0709 12:28:39.550121 (+   262us) write_op.cc:276] Timestamp: P: 1783600119549760 usec, L: 0
0709 12:28:39.550163 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:28:39.551245 (+  1082us) log.cc:844] Serialized 5884 byte log entry
0709 12:28:41.853736 (+2302491us) op_driver.cc:464] REPLICATION: finished
0709 12:28:42.064062 (+210326us) write_op.cc:301] APPLY: starting
0709 12:28:42.064238 (+   176us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:42.507742 (+443504us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:42.519423 (+ 11681us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:42.561555 (+ 42132us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:42.561935 (+   380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 12:28:42.562025 (+    90us) write_op.cc:312] APPLY: finished
0709 12:28:42.562580 (+   555us) log.cc:844] Serialized 139 byte log entry
0709 12:28:42.564060 (+  1480us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:42.564533 (+   473us) write_op.cc:454] Released schema lock
0709 12:28:42.564615 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:42.571956 (+  7341us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:42.571995 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1104242,"reported_tablets":49,"spinlock_wait_cycles":67843712,"child_traces":[["op",{"apply.queue_time_us":193697,"apply.run_cpu_time_us":31081,"apply.run_wall_time_us":507883,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66827,"num_ops":15,"prepare.queue_time_us":576648,"prepare.run_cpu_time_us":21585,"prepare.run_wall_time_us":1125198,"raft.queue_time_us":333529,"raft.run_cpu_time_us":40683,"raft.run_wall_time_us":333062,"replication_time_us":2219879,"thread_start_us":1046409,"threads_started":4,"wal-append.queue_time_us":855712,"wal-append.run_cpu_time_us":30360,"wal-append.run_wall_time_us":3142546}]]}
I20260709 12:28:48.360682 15040 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.323199  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.213977 15537 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.339565  7733 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:48.397014  7805 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:28:48.415587  7805 pending_rounds.cc:85] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 9
I20260709 12:28:48.415903  7805 pending_rounds.cc:107] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:28:48.416256  7805 raft_consensus.cc:2889] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:48.417299  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 94174.4us
I20260709 12:28:48.419102  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:48.324790 15642 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.340961 15271 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:48.357290 15285 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.389331 14906 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:48.472561  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 218427us
I20260709 12:28:48.354588 15675 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
W20260709 12:28:48.480126  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 163514us
W20260709 12:28:48.491328  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72283.2us
I20260709 12:28:48.363701 15606 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.496111  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.479378  7808 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:48.365873 15607 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.423038 15501 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:48.343319 15644 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 12:28:48.502661  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.498083  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:48.505445  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.367753 15637 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.467739 15496 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:48.364830 15646 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.510636  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.516804  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.375206 15472 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.297173 15513 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:48.523819  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 223431us
I20260709 12:28:48.528327  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.534055  8407 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:28:48.513417 15285 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.500905 15667 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:48.536378  7700 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.703s (SENT)
I20260709 12:28:48.543560  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.509815 15271 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:48.545020  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47020.3us
I20260709 12:28:48.515170 15496 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:48.515290 15432 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.511180 15638 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:48.550835  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.551620  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.573758  7733 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:48.574208  7735 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:48.552438  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.574904  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.575783  7733 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 15: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.576272  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.577289  7732 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:48.512332 15655 raft_consensus.cc:2764] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:28:48.495445 14906 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.526116 15489 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:48.572808  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.582243  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:48.558035  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.576624  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.598335  7733 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.540863 15594 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:28:48.570894  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60317.6us
I20260709 12:28:48.547873 15271 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.525612 15592 raft_consensus.cc:697] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:48.591586  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.591665  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.592482  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.592651  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:48.570037 15584 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.889s	user 0.012s	sys 0.002s
I20260709 12:28:48.539772 15641 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.538785 15642 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.600345  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:48.598415  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.585243  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.602160  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.618624  7733 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:48.569473 15575 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.604389  7730 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:48.624223  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:48.570017 15639 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:48.605924  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.557807 15285 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.603313 15242 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:48.571707 15675 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:48.610850 15660 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:28:48.580871 15490 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:48.586933 15491 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:48.669036  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.621215  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.625753  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.606281  7736 raft_consensus.cc:2393] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.609711 15271 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:48.606550 15432 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:48.647679 15661 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:28:48.679064  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.617727 15584 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.634538  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.609597  7732 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:48.693725  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.637048  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.617517 15424 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:28:48.640017 15662 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:28:48.699805 15606 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.700587 15606 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:48.701483 15606 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:48.652176 15242 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.702030 15606 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.704033 15646 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.704689 15677 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:48.678992 15500 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.705135 15242 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.705380 15677 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:48.705633 15500 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.706334 15500 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.641104 14906 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.641656 15277 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.707266 15680 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:48.629043 15649 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 12:28:48.707969 15680 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:48.610715 15489 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.622812 15641 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:48.708837 15500 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.709501 15500 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.709411 15641 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.710089 15500 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.710750 15489 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.711926 15641 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:48.712574 15678 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:28:48.696425 15537 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:48.675858  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50193.2us
I20260709 12:28:48.714238  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.714463 15588 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.715658 15537 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:48.716007 15637 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.718055 15588 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.709013 15680 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:48.719918 15654 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:28:48.624531 15642 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.676767  7737 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:48.630860 15575 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.688920  7736 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:48.634181  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:48.653339 15651 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:48.748647  7736 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:48.748883  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.643267 15657 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:48.698927  7734 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:48.693876 15592 consensus_queue.cc:237] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.655520 15597 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.670471  7729 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.753161  7737 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:48.721074 15424 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.619578 15683 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:48.721940  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2948) took 1003 ms. Trace:
I20260709 12:28:48.668553 15607 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.677211  7733 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:48.670941 15491 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:48.768234  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.749898  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:48.750299  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 116323us
I20260709 12:28:48.769918  7733 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:48.630487 15496 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.680974 15490 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:48.779987  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.649586  7730 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:48.752038 15651 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:48.786058  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:48.757174  7735 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:28:48.761719  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.761510  7810 rpcz_store.cc:276] 0709 12:28:47.718795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.718967 (+   172us) service_pool.cc:224] Handling call
0709 12:28:47.719494 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:47.720088 (+   594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:47.720132 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:47.720938 (+   806us) log.cc:844] Serialized 26 byte log entry
0709 12:28:48.091203 (+370265us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:48.091274 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:48.091344 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:48.721596 (+630252us) raft_consensus.cc:1685] finished
0709 12:28:48.721631 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:48.721854 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":213,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":127,"thread_start_us":369714,"threads_started":1,"wal-append.queue_time_us":550766}
I20260709 12:28:48.654351 15675 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.689858 15501 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 12:28:48.835265  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 211082us
W20260709 12:28:48.775007 15658 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:28:48.838353  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:48.838788  7653 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 12
I20260709 12:28:48.839143  7653 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:28:48.839439  7653 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:48.770610  7736 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:48.688364 15040 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:48.771011  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.845288  7733 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:48.771250  7700 leader_election.cc:400] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
I20260709 12:28:48.767503  7732 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:48.772732 15285 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.773034  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:48.849769  7656 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:48.618618  7551 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 14, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 14 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.851125  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.774017  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:48.778316  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:48.779539  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 262850us
I20260709 12:28:48.765604 15693 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:48.854547  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:48.762192 15399 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.192s	user 0.015s	sys 0.001s
I20260709 12:28:48.756189  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.781713  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.784358  7654 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:48.647758 15532 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 12:28:48.863781  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.668555 15638 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:48.762423 15424 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:48.782496  8415 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:48.866272  7736 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:48.755061 15657 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.780541 15490 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:48.832204  8407 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:48.787454  7737 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:48.834273  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:48.839984  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:48.761214 15683 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.782534 15644 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.848295  7700 leader_election.cc:403] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:48.846735  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.773717 15491 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:48.750918 15646 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.851436  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:48.828819 15651 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:48.855525  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.858059  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 2949) took 1136 ms. Trace:
I20260709 12:28:48.859792 15399 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:48.860620 15498 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.772s	user 0.007s	sys 0.006s
I20260709 12:28:48.863047  7734 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.865007 15684 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:48.706631 15677 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:48.848313 15640 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:48.861996  7551 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 7 to 16, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 16 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:48.925802 15689 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:28:48.925637  7551 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:48.927510  7551 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:28:48.929375  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 328827us
I20260709 12:28:48.929806  7551 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 4 to 18, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 18 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.887277  8408 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:48.900499  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
W20260709 12:28:48.905920  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 134854us
W20260709 12:28:48.909188  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53746us
I20260709 12:28:48.909209  7806 rpcz_store.cc:276] 0709 12:28:47.722147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.722331 (+   184us) service_pool.cc:224] Handling call
0709 12:28:47.722756 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:47.723878 (+  1122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:47.723931 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:47.724839 (+   908us) log.cc:844] Serialized 52 byte log entry
0709 12:28:48.032973 (+308134us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:48.033038 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:48.033141 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:48.857781 (+824640us) raft_consensus.cc:1685] finished
0709 12:28:48.857803 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:48.857982 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":450,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":307519,"threads_started":1,"wal-append.queue_time_us":519862}
I20260709 12:28:48.939075  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.939864  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:48.940927  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:48.943838  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:48.944900  7735 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:48.880993 15657 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:48.913899 15285 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
I20260709 12:28:48.920519 15688 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:48.921271 15669 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:48.922226 15399 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.922943 15675 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.923092 15667 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:48.923578 15500 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:48.914481 15646 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:48.924317 15676 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.000s
W20260709 12:28:48.886705  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46807us
I20260709 12:28:48.935097  7734 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:48.887467 15683 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:48.933917  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:49.031023  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 196732us
I20260709 12:28:48.906111 15644 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:49.029013  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 89222.7us
W20260709 12:28:49.080446  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 146625us
I20260709 12:28:49.088557  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.095885  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.099905  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.099684  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.102226  7657 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:49.104174  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.105465  7655 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:49.107306  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.108799 15696 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:49.035845 15641 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:49.092224 15500 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.111423  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.031013 15606 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:49.065567 15652 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.003s
I20260709 12:28:49.036586 15496 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:49.077857 15584 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.124006  7656 raft_consensus.cc:1240] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:49.071949 15575 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.051084 15040 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.076221 15242 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:49.120746  7662 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:49.128634  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.157562  7655 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:49.116053  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.063701 15669 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:49.064311 15639 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:49.109934 15681 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.079799 15676 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.093282 15592 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:49.163589 15639 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.172559 15639 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:49.109802 15696 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:49.175796  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.176658  7656 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:49.152053  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.110096 15285 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:28:49.147629 15606 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:49.181366 15686 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.256s	user 0.001s	sys 0.001s
I20260709 12:28:49.123754 15641 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.185590 15641 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.127406 15500 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.158751  7661 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:49.054700 15684 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.091s
I20260709 12:28:49.155936  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.145284 14906 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:49.168658  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90467us
W20260709 12:28:49.171429 15700 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
I20260709 12:28:49.176116 15696 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.165237 15575 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.043963 15688 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.165441  7551 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 16, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:28:49.197140  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45190.8us
I20260709 12:28:49.202346  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:49.118841 15699 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:28:49.211699  7656 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:49.211783  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.142776 15652 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:49.208719 15650 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.897s	user 0.007s	sys 0.007s
W20260709 12:28:49.217676  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 126513us
I20260709 12:28:49.208540 15592 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:49.231141  7777 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.162050 15040 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.234727  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.220793 15606 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:49.242887  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:49.248705 15677 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:49.249241 15677 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.249547 15699 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:49.249686 15677 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.250146 15699 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.250669 15699 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:49.235998 15588 consensus_queue.cc:1059] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:28:49.252326 15588 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:49.251734  7551 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 18, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 18 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:49.252920 15588 raft_consensus.cc:3055] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:49.253441 15588 raft_consensus.cc:740] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:49.253687  7551 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:49.256292 15688 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.257079 15646 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:49.236907 15501 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:28:49.261194 15490 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.236444 15592 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.263306 15592 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:49.245690 15669 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.268617 15676 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.237547 15607 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:49.246771 15687 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.269798 15607 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.270336 15607 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.270020 15687 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.272576 15677 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.274621 15652 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.224207 15667 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:49.192087 15681 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.278345  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.280031  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.275112 15677 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:49.280736  7734 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.281455  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.212217 15575 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.242236 15597 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:49.221228 15641 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:28:49.272684 15687 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.247536 15606 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:49.252965 15040 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:49.299891 15271 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 12:28:49.236202 15650 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.313350  7734 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:49.300238 15597 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.275323 15710 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:28:49.246443 15491 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:49.317919  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 75101.4us
I20260709 12:28:49.308373 15606 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:49.324643  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:49.319972 15597 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.333690  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.333971  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.334112  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.334697  7658 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:49.334973  7655 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:49.330296  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95635.2us
I20260709 12:28:49.334925  7660 raft_consensus.cc:2410] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:28:49.337952  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.339102  7659 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:49.340711  7659 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:49.340812  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.341711  7655 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.341176  7659 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:49.343024  7659 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.343865  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.344133  7659 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:49.346099  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.347568  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.349270  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.350443  7655 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:49.353322  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.354933  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.311942 15424 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.356158  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.359488  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.316568 15271 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.361743  7662 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:49.362433  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:49.327898 15644 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.351841 15684 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:49.399904 15650 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.376238  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3521) took 1284 ms. Trace:
I20260709 12:28:49.374985 15642 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.361303 15652 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:49.380404 15575 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.382009 15491 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.383637  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:49.384069 15424 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:49.386322  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.387171  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.387964  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.389482 15588 consensus_queue.cc:260] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.389984  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:49.391244  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:49.396823  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.397338 14906 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.398149  7730 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:28:49.328858 15667 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.400091 15684 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:49.366535  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.461495 15683 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:49.572681 15701 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 12:28:49.400413 15500 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.573670  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3523) took 1481 ms. Trace:
I20260709 12:28:49.574290 15683 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:49.574117  7805 rpcz_store.cc:276] 0709 12:28:48.091983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.135095 (+ 43112us) service_pool.cc:224] Handling call
0709 12:28:48.135514 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.416979 (+281465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.573431 (+1156452us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b540042a754. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:49.573602 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590814080}
I20260709 12:28:49.358523 15638 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:49.500433 15656 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:49.576490 15656 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:49.577157 15651 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.001s
I20260709 12:28:49.577557 15656 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:49.578681 15657 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 12:28:49.578930 15656 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:49.579140  8411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1394) took 1607 ms (client timeout 2066 ms). Trace:
W20260709 12:28:49.579717  8411 rpcz_store.cc:269] 0709 12:28:47.971842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.972054 (+   212us) service_pool.cc:224] Handling call
0709 12:28:49.578918 (+1606864us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:49.579072 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969733888}
I20260709 12:28:49.581164  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.581910  8411 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:49.578207 15490 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:49.493887  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67498us
I20260709 12:28:49.506876  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.593186  7657 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:49.402323  7662 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:49.411279  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.595429  7655 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:49.588665  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:49.590278  8415 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:49.505463  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.477576  8412 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:49.592079  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.309365  7551 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 11 to 17, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 17 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.605643  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:49.606045  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.606786  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:49.607605  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 241194us
I20260709 12:28:49.608259  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.363634 15676 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:49.497632 15607 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.410846  7659 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:28:49.610337  7735 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:49.611038 14594 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:49.513216  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.607698  8406 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:49.617002  7659 pending_rounds.cc:85] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 10
I20260709 12:28:49.614085  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.614302  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:49.400555  7804 rpcz_store.cc:276] 0709 12:28:48.091520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.113176 (+ 21656us) service_pool.cc:224] Handling call
0709 12:28:48.113459 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.372739 (+259280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.375900 (+1003161us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003c5ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:49.376151 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591719168}
W20260709 12:28:49.617084  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104030us
I20260709 12:28:49.612489  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:49.618507  7702 connection.cc:256] RPC call timeout handler was delayed by -1.273955112999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 12:28:49.618855  8411 raft_consensus.cc:2368] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18.
I20260709 12:28:49.619792  7659 pending_rounds.cc:107] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:28:49.620303  7659 raft_consensus.cc:2889] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:49.620326  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.621930  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:49.484500 15703 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.307s	user 0.003s	sys 0.000s
W20260709 12:28:49.460192 15711 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:28:49.628422  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.635486  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.635876  7700 leader_election.cc:400] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18.
I20260709 12:28:49.636425  7700 leader_election.cc:403] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:28:49.636618  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.636940  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.637637 15644 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.638216  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.640100 15710 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 17. Current term is 18.
I20260709 12:28:49.634418  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:49.619617  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.641821  8405 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:28:49.638228 15644 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.631484  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.633781  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.639595  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.644558 15644 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.644722 15710 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:49.645803  8406 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:49.646803  7658 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:49.647111 15680 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:49.647877 15638 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:49.634366  7626 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:49.648226  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:49.648619 15520 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:49.480065 15613 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:49.656564 15399 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:49.657506 15684 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.658234  7624 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:49.658533 15642 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:49.415920 15688 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:49.661445  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.737937 15688 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:49.661795  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.738663 15688 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:49.664047 15687 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:49.739578  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3435) took 1184 ms. Trace:
I20260709 12:28:49.740054  7732 rpcz_store.cc:276] 0709 12:28:48.557813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.724787 (+166974us) service_pool.cc:224] Handling call
0709 12:28:48.725127 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.849149 (+124022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.739354 (+890205us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:49.739521 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95919360}
I20260709 12:28:49.741420 15687 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.741976 15687 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:49.592017 14906 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:49.648572 15710 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:49.479553 15271 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.614701 15496 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:49.660277 15605 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.727033  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:49.728941  7624 leader_election.cc:403] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:28:49.730089  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.731137 14594 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:49.694469 15242 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:28:49.732259 15530 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:28:49.726886 15384 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:28:49.733749  7657 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:49.660544 15677 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:49.695654 15399 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.752377  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.753247  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.659066 15581 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.151s
I20260709 12:28:49.504907 15681 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:49.824998  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.579160 15657 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.691670  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.771021  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.632370 15491 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:49.661116 15040 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:49.802282 15520 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:49.784010  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 132859us
W20260709 12:28:49.842749  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 230306us
I20260709 12:28:49.843487  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.844244  7733 raft_consensus.cc:2376] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20. Re-sending same reply.
I20260709 12:28:49.844949  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:49.845391  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.787011  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.792420  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.497632 15699 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.502892 15592 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.785388  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:49.780468 14594 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.577665 15651 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.694011 15513 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:49.692179 15683 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.777551  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:49.917411  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 140026us
I20260709 12:28:49.825287 15646 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:49.397276 15639 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.687736 15584 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:49.753486  7735 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:49.800661 15501 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
W20260709 12:28:49.813529  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 155454us
I20260709 12:28:49.816771  7657 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:28:49.817698 15677 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:49.818545 15399 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:49.819147 15242 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:49.811205  7728 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:49.632510 15575 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.152s
I20260709 12:28:49.829372  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.829900  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.825626  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3593) took 1081 ms. Trace:
I20260709 12:28:49.659595 15606 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.983538  7734 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:49.728149 15644 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:28:49.834268  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.834535  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.782647  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:50.018136  7804 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:50.018261  7729 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:49.837055  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.849772  7811 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 12:28:50.019883  7811 pending_rounds.cc:85] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 16
I20260709 12:28:50.020166  7811 pending_rounds.cc:107] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 12:28:50.020399  7811 raft_consensus.cc:2889] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:50.021765  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.923733 15646 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.908687  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.915357  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:49.916271  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:50.030339  7730 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:50.030550  7809 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:49.921178  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:49.802685  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:49.832201 15500 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:49.825191 15271 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:49.783838 15496 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.824580 15675 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:49.925778 14594 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:50.069002 14594 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:49.841130 15656 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:49.842406 15040 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.017302 15575 consensus_queue.cc:1243] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:49.932044  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:49.805100 15605 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:49.936280  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.941759  8414 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:28:49.991798  7731 rpcz_store.cc:276] 0709 12:28:48.744735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.869051 (+124316us) service_pool.cc:224] Handling call
0709 12:28:48.884045 (+ 14994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:48.886509 (+  2464us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:48.886543 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:48.904297 (+ 17754us) log.cc:844] Serialized 26 byte log entry
0709 12:28:49.138382 (+234085us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:49.138438 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.139813 (+  1375us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:49.483791 (+343978us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:49.810052 (+326261us) raft_consensus.cc:1685] finished
0709 12:28:49.810087 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.825547 (+ 15460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95745,"raft.queue_time_us":20750,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":30370,"spinlock_wait_cycles":12367616,"thread_start_us":202555,"threads_started":1,"wal-append.queue_time_us":268918}
I20260709 12:28:49.916824 15513 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.017207  7732 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:49.862178  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:50.101697  7803 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:49.948117  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:50.024829  7810 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:50.046264  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.098s (SENT) [suppressed 23 similar messages]
I20260709 12:28:50.061506  7804 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:49.919653 15698 consensus_queue.cc:1059] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:28:49.909916 15520 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:49.949393 15710 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:49.942394 15242 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:49.833002 15491 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:50.077723  7733 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:49.985340 15690 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.137s
I20260709 12:28:50.087074  7551 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 8 to 15, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 15 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:28:50.046264 15646 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:50.106874  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.145576  7804 raft_consensus.cc:2410] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:28:50.108376  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.113977  7809 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:50.116539  7803 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:28:50.059024 15681 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:28:50.117131 15731 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:50.101694  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:49.926843 15683 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:50.120275  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 172257us
I20260709 12:28:49.942009 15584 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.099783 15040 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:50.139750  7805 raft_consensus.cc:2376] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22. Re-sending same reply.
W20260709 12:28:50.140184  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 231304us
W20260709 12:28:50.098446 15704 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.932s	user 0.008s	sys 0.028s
I20260709 12:28:50.074941 15271 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.116720 15732 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:50.147604  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:50.218282  7809 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:50.149441  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:50.219599  7810 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:50.153759  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.220825  7806 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:50.155345  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.155541  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.118621 15513 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:50.162444  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:50.100201 15675 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:50.228042  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 608510us
W20260709 12:28:50.229647  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57790.2us
I20260709 12:28:50.145427 15607 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:50.136950 15604 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:50.238780  7811 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:50.240125  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.266530  7803 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:50.241441  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.267462  7804 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:50.268806  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.270533  7804 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:50.244467  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.244098  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.272739  7806 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:50.258281  7808 raft_consensus.cc:2368] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
W20260709 12:28:50.254163 15667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.021s	sys 0.030s
I20260709 12:28:50.253962 15731 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:28:50.229358 15530 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:28:50.293530  7803 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:50.293717  7810 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:28:50.301571  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1171) took 1516 ms. Trace:
I20260709 12:28:50.254374 15491 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.165596  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:50.493539  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 447290us
I20260709 12:28:50.408787  7808 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:50.650931  8413 rpcz_store.cc:276] 0709 12:28:48.784744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.784935 (+   191us) service_pool.cc:224] Handling call
0709 12:28:48.837053 (+ 52118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:48.852338 (+ 15285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:48.852373 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:48.858197 (+  5824us) log.cc:844] Serialized 26 byte log entry
0709 12:28:49.052055 (+193858us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:49.052106 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.052201 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:50.097587 (+1045386us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:50.301156 (+203569us) raft_consensus.cc:1685] finished
0709 12:28:50.301187 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:50.301487 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14747,"raft.queue_time_us":17656,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15333,"spinlock_wait_cycles":40231680,"thread_start_us":187692,"threads_started":1,"wal-append.queue_time_us":301941}
W20260709 12:28:50.561888  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 709099us
I20260709 12:28:50.661196  7804 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:50.661242  7810 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:50.662135  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:50.593592  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 359582us
W20260709 12:28:50.618696  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.663017  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:50.440078  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 518915us
I20260709 12:28:50.662379  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:50.448767  7659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.002s	sys 0.007s
I20260709 12:28:50.665391  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:50.318754 15657 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:50.665686  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:50.368172 15687 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.626s	user 0.016s	sys 0.000s
W20260709 12:28:50.310004 15667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.021s	sys 0.031s
I20260709 12:28:50.366253 15675 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.496032 15639 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.674252 15736 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:50.683159 15736 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.683645 15736 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:50.684993 15040 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.687901  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:50.668088 15597 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.305s	user 0.015s	sys 0.000s
I20260709 12:28:50.370994 15699 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:50.615569 15743 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:28:50.607735 15604 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:50.670728  7810 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
W20260709 12:28:50.500684 15680 consensus_meta.cc:287] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.811s	user 0.012s	sys 0.000s
W20260709 12:28:50.647212 15745 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:28:50.671463  7811 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:28:50.672362  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:50.363008 15688 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.000s
W20260709 12:28:50.665783  7659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.003s	sys 0.007s
I20260709 12:28:50.287338 15489 raft_consensus.cc:3055] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:28:50.437780 15498 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.003s	sys 0.005s
I20260709 12:28:50.656196 15605 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.664211  7777 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:50.689242  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:50.648864 15677 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:50.468194 15520 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.504760 15496 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.690794  7808 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:50.700284 15680 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:50.667270  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 501712us
I20260709 12:28:50.485404 15735 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:50.692929  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.668023 15733 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:50.665063 15606 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:50.678172 15581 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:50.692240 15597 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.495962  7551 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 18, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 18 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.687415 15271 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:50.727794 15683 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:50.678598 15687 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.715878  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1465) took 1939 ms. Trace:
I20260709 12:28:50.728533  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1470) took 1347 ms. Trace:
W20260709 12:28:50.718364 14906 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.974s	user 0.010s	sys 0.002s
W20260709 12:28:50.693419 15718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.009s	sys 0.023s
W20260709 12:28:50.737418  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73297.4us
W20260709 12:28:50.738525  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49461.3us
I20260709 12:28:50.288234 15242 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.652138 15732 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.744187  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:50.752733  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.764426 15530 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:50.753564 15675 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:28:50.729887 15498 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.004s	sys 0.007s
I20260709 12:28:50.757820  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.868634  7810 raft_consensus.cc:2410] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:28:50.869354  8413 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:28:50.726460 15489 raft_consensus.cc:740] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:50.774173  7804 raft_consensus.cc:2368] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:50.690285 15642 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:50.769547 15399 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.752s	user 0.008s	sys 0.004s
I20260709 12:28:50.724391 15688 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.784997  7655 rpcz_store.cc:276] 0709 12:28:49.381509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.661412 (+279903us) service_pool.cc:224] Handling call
0709 12:28:49.727091 (+ 65679us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:49.747958 (+ 20867us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:49.807423 (+ 59465us) log.cc:844] Serialized 14 byte log entry
0709 12:28:50.141044 (+333621us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:50.142480 (+  1436us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.142704 (+   224us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:50.143043 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 12:28:50.143467 (+   424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.143575 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:50.728101 (+584526us) raft_consensus.cc:1685] finished
0709 12:28:50.728137 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:50.728429 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31070,"prepare.queue_time_us":54319,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":131,"raft.queue_time_us":78429,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":27016,"spinlock_wait_cycles":28160128,"thread_start_us":203147,"threads_started":1,"wal-append.queue_time_us":646758}
I20260709 12:28:50.788662  7731 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:50.755072 15757 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:50.783171  7653 rpcz_store.cc:276] 0709 12:28:48.776608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.776783 (+   175us) service_pool.cc:224] Handling call
0709 12:28:48.831256 (+ 54473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:48.854691 (+ 23435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:48.854723 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:49.917171 (+1062448us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:49.917754 (+   583us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.117026 (+199272us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:50.117065 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.117139 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:50.715417 (+598278us) raft_consensus.cc:1685] finished
0709 12:28:50.715464 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:50.715784 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18161,"raft.queue_time_us":24581,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":807,"spinlock_wait_cycles":8523648,"thread_start_us":198942,"threads_started":1,"wal-append.queue_time_us":384798}
I20260709 12:28:50.774950 15693 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:50.811318  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 374064us
W20260709 12:28:50.865130 15755 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:28:50.813248  7808 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
W20260709 12:28:50.790766 15718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.010s	sys 0.023s
I20260709 12:28:50.863832  8412 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:50.771513 15748 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:50.873013  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 128966us
I20260709 12:28:50.886137 15657 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:50.870081  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.954895  7810 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:50.955942  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.956584  7810 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:28:50.957442  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:50.864449 15750 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.792706 15735 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:50.885936  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:50.766258 15733 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.892740  7735 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:50.793638 15683 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.895632  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:50.897433  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:50.979907  7658 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:50.874665 15530 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.981333  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:50.811569  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 146287us
I20260709 12:28:50.980036  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:50.863306 15646 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 12:28:50.899168  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:50.984256  7655 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:50.984700  7728 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:50.794487 14906 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.813869 15581 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
W20260709 12:28:50.886047 15749 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:28:50.948025  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:50.999264  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:50.864746 15675 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:50.948908  7654 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:50.943187  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:50.958153  7810 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:50.962011  8415 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:50.892521 15399 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.891511 15642 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.010615  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:51.011699  8413 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:50.968834  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:50.782091 15604 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:50.974231  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:51.016714  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:50.862900 15732 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.017869  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:51.018522  7735 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:51.018616  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:50.986768  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:51.001561  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:51.020781  7808 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:28:50.942298 15597 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:50.903595 15693 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.024096  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:51.024757  7731 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:51.024749  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:50.997345  7655 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:51.006534  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:50.867262  7551 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:51.032819  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:51.033838  7657 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:51.044265  7626 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:51.044417  7733 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:28:51.045351  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:51.045332  7658 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:51.054127  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:51.070300  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.082212  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63690.2us
I20260709 12:28:51.035866 15520 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.094278  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:50.998840 15646 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.973132 15750 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:50.981212 15735 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.002465 15683 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.104242  7658 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:28:51.114748  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 108326us
I20260709 12:28:51.123660  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:51.124158  7808 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 12:28:51.091765 15513 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.833s	user 0.015s	sys 0.000s
I20260709 12:28:51.120045  7735 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:51.117985  7731 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:51.127489  7778 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:51.127381  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:51.035701 15690 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:28:51.130074  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:51.128537  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:51.100741  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:51.053651 15771 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.132994  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:51.120168 15605 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 12:28:51.136801 15737 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:28:51.139505 15753 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 12:28:51.102486 15759 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.141315 15759 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.105834 15693 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:51.143473 15767 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:28:51.143872 15746 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:51.143931 15775 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:51.144500 15746 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.144755 15775 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.145009 15746 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:51.145352 15775 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:51.113287 15747 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:28:51.095757 15613 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:28:51.149076 15731 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:51.154017 15399 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:51.165545 15642 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:51.166283  7809 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:51.155046 15605 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.026013 15760 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.103s
I20260709 12:28:51.204779  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:51.153246 15750 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:51.112185 15744 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 12:28:51.115605 15657 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.105924 15766 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:28:51.116338 15520 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.150305 15581 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:51.172914  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49341.2us
I20260709 12:28:51.145282 14906 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.157071 15683 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:51.174391  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1181) took 1562 ms. Trace:
I20260709 12:28:51.176782  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3157) took 1400 ms. Trace:
I20260709 12:28:51.192229  7728 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:51.194350  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:51.360030  7731 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:51.195847  7730 raft_consensus.cc:1275] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:51.155344 15735 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.205785  8412 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:51.202106  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:51.190521 15688 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:51.207907 15605 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.283239 15530 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.350435 15766 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:51.321949 15684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.014s	sys 0.013s
W20260709 12:28:51.386152 15684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.295s	user 0.015s	sys 0.014s
W20260709 12:28:51.349048  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73093.7us
I20260709 12:28:51.386679 15765 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:51.388260 15776 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:51.388876 15776 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:51.389935 15776 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.351212 15520 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.179935 15513 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.351647 15657 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.267300 15399 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:51.395754 15399 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:51.396257 15399 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:51.209282 15731 raft_consensus.cc:697] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:51.419514  7662 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:51.349221 15750 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:51.269523  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:51.149111 15770 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:51.271023 15762 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:28:51.357196  8405 rpcz_store.cc:276] 0709 12:28:49.611816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.642941 (+ 31125us) service_pool.cc:224] Handling call
0709 12:28:49.648849 (+  5908us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:49.660698 (+ 11849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:49.660741 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:49.687915 (+ 27174us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.226990 (+539075us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:50.227058 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.227153 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:51.138120 (+910967us) raft_consensus.cc:1685] finished
0709 12:28:51.138153 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.174289 (+ 36136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36786,"raft.queue_time_us":22178,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":3959,"thread_start_us":425611,"threads_started":1,"wal-append.queue_time_us":1142228}
W20260709 12:28:51.205559 15736 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.521s	user 0.014s	sys 0.000s
I20260709 12:28:51.298806 15687 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.194626 15771 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.369295 15646 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:51.167171 15675 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.461683 15750 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.476601 15692 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:51.477167 15692 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:51.477582 15513 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.448916 15731 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:51.480602  8409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1172) took 2651 ms (client timeout 2098 ms). Trace:
I20260709 12:28:51.480876 15690 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:51.481133  8409 rpcz_store.cc:269] 0709 12:28:48.829390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.829548 (+   158us) service_pool.cc:224] Handling call
0709 12:28:51.480355 (+2650807us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400770bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:51.480503 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323294592}
I20260709 12:28:51.481565 15690 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.485191 15690 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.471411 15736 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.486980 15489 consensus_queue.cc:260] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.488389  8405 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:51.488914 15489 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:51.490964  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:51.491907  8412 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:51.472031 15733 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:51.493556 15749 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:28:51.456377 15732 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:51.464857 15770 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.474444 15759 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:51.500774 15605 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:51.478106 15692 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.505038 15581 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:51.518141 15788 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:51.529910 15788 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.530493 15788 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:51.531877 15581 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.142067  7551 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 16, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:28:51.347456  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.348754  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 215800us
I20260709 12:28:51.353905  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3154) took 1586 ms. Trace:
I20260709 12:28:51.358883  7659 rpcz_store.cc:276] 0709 12:28:49.774341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.849431 (+ 75090us) service_pool.cc:224] Handling call
0709 12:28:49.849816 (+   385us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:50.722000 (+872184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:50.722096 (+    96us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:50.729767 (+  7671us) log.cc:844] Serialized 78 byte log entry
0709 12:28:50.885879 (+156112us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:50.885928 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.886028 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:51.141345 (+255317us) raft_consensus.cc:1685] finished
0709 12:28:51.141380 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.175094 (+ 33714us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85819,"raft.queue_time_us":200226,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":26401,"thread_start_us":917816,"threads_started":2,"wal-append.queue_time_us":253664}
W20260709 12:28:51.738695  7657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.003s	sys 0.009s
W20260709 12:28:51.996083  7657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.005s	sys 0.009s
W20260709 12:28:51.444051  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 241837us
W20260709 12:28:51.999682  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 86528.6us
I20260709 12:28:52.002537  7626 leader_election.cc:400] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:51.458571  7808 raft_consensus.cc:606] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:51.450906  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1284) took 1613 ms. Trace:
I20260709 12:28:52.003813  7808 raft_consensus.cc:614] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:51.813282  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:52.004652  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:51.786525  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.517802 15656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.389s	user 0.010s	sys 0.016s
I20260709 12:28:51.835373 15604 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:51.841423  7737 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:51.554900 15692 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:51.525585 15734 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.407s	user 0.006s	sys 0.007s
I20260709 12:28:51.575161 15746 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.617502 15775 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:51.653743 15788 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:51.786406  7731 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:52.003171  7626 leader_election.cc:403] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:28:52.006680 15688 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:52.003916  7807 rpcz_store.cc:276] 0709 12:28:49.837875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.838019 (+   144us) service_pool.cc:224] Handling call
0709 12:28:49.838396 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:49.838847 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:49.838876 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:49.839624 (+   748us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.649189 (+809565us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:50.649263 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.649350 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:51.450521 (+801171us) raft_consensus.cc:1685] finished
0709 12:28:51.450555 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.450809 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":244153,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":18046,"thread_start_us":406852,"threads_started":1,"wal-append.queue_time_us":814217}
W20260709 12:28:52.100183  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:51.912719  7656 rpcz_store.cc:276] 0709 12:28:49.767087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.848806 (+ 81719us) service_pool.cc:224] Handling call
0709 12:28:49.927585 (+ 78779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:49.928027 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:49.928069 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:49.928583 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:28:50.256831 (+328248us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:50.256896 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.257009 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:51.353637 (+1096628us) raft_consensus.cc:1685] finished
0709 12:28:51.353671 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.353838 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":327247,"threads_started":1,"wal-append.queue_time_us":853007}
I20260709 12:28:52.615316  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3618) took 1521 ms. Trace:
I20260709 12:28:52.016028  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:52.656495  7737 rpcz_store.cc:276] 0709 12:28:51.094221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.840083 (+745862us) service_pool.cc:224] Handling call
0709 12:28:51.840819 (+   736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:52.027788 (+186969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:52.615175 (+587387us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:52.037912 15656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.887s	user 0.011s	sys 0.016s
I20260709 12:28:52.673135  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:52.673983 15399 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.277s	user 0.016s	sys 0.007s
I20260709 12:28:52.674703  7807 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 12:28:52.247613 15489 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.758s	user 0.016s	sys 0.007s
I20260709 12:28:52.676375 15489 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:28:52.679574  7807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3547) took 1506 ms (client timeout 1790 ms). Trace:
W20260709 12:28:52.680164  7807 rpcz_store.cc:269] 0709 12:28:51.172763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:52.672239 (+1499476us) service_pool.cc:224] Handling call
0709 12:28:52.679414 (+  7175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:52.044345  7551 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 17, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 17 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.077663 15584 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.098582 15604 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.684571 15656 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:52.037518 15766 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.660s
I20260709 12:28:52.686566 15656 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:52.682142  7551 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 18, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 18 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.656731  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1650) took 1621 ms. Trace:
I20260709 12:28:52.699265  7551 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:28:52.702006 15584 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.044771 15644 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:52.695334  7731 rpcz_store.cc:276] 0709 12:28:51.008001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.451868 (+443867us) service_pool.cc:224] Handling call
0709 12:28:51.785174 (+333306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:52.612858 (+827684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:52.629421 (+ 16563us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:52.679071 15489 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:28:52.674597 15399 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:52.717005 15775 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:52.718370 15584 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:52.720242 15692 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.705s	user 0.015s	sys 0.003s
I20260709 12:28:52.702354  7551 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 14, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 14 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:52.727108  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 725067us
I20260709 12:28:52.725777  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:52.731707  8413 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:52.725664 15746 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:52.729380 15766 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
W20260709 12:28:52.737342  7734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1634) took 3340 ms (client timeout 1979 ms). Trace:
W20260709 12:28:52.738052  7734 rpcz_store.cc:269] 0709 12:28:49.396301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.396530 (+   229us) service_pool.cc:224] Handling call
0709 12:28:52.736695 (+3340165us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b540039b2d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:52.737026 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937390336}
I20260709 12:28:52.737203  7551 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 19, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 19 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.741313  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3546) took 1569 ms. Trace:
I20260709 12:28:52.741978  7808 rpcz_store.cc:276] 0709 12:28:51.172402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:52.696842 (+1524440us) service_pool.cc:224] Handling call
0709 12:28:52.741177 (+ 44335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:52.743656 15399 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:52.733556 15040 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:52.735762  8380 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.006s	sys 0.027s
W20260709 12:28:52.747433  8380 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.007s	sys 0.027s
I20260709 12:28:52.748648 15770 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.730907 15604 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:52.729478 15775 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.725225 15688 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:52.752683 15775 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.737563 15584 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:52.755786  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.019s	sys 0.046s
W20260709 12:28:52.756407  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.020s	sys 0.047s
I20260709 12:28:52.766691  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:52.768314 15746 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:52.769582 15746 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:52.770498 15746 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:28:52.756845  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 711136us
W20260709 12:28:52.760437  8412 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.213s	user 0.018s	sys 0.019s
I20260709 12:28:52.786212  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3619) took 1492 ms. Trace:
I20260709 12:28:52.792233  7731 rpcz_store.cc:276] 0709 12:28:51.281103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:52.722668 (+1441565us) service_pool.cc:224] Handling call
0709 12:28:52.733693 (+ 11025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:52.770989 (+ 37296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:52.771027 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:52.771119 (+    92us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:52.771178 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:52.771303 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:52.771483 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:52.797557 15750 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.794481 15766 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:52.800011 15766 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:52.801030 15399 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:52.801878 15766 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:52.787115 15746 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:52.793923  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.29707e+06us
I20260709 12:28:52.820101 15788 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.822000 15775 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.809019 15688 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:52.823055  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:52.823758 15803 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.824082 15688 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:52.824743 15803 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:52.751371 15644 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.826664 15688 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:52.827913 15802 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:28:52.833557  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:52.829311 15803 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:52.840395 15766 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:52.837083 15802 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:52.833653  8413 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:52.814486  8412 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:52.842952 15802 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:52.844564  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1412) took 1362 ms. Trace:
I20260709 12:28:52.845098  8412 rpcz_store.cc:276] 0709 12:28:51.481714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.490574 (+  8860us) service_pool.cc:224] Handling call
0709 12:28:52.844449 (+1353875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:52.845808 15644 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.845739 15802 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.848418 15802 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:52.849285  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3612) took 1951 ms. Trace:
I20260709 12:28:52.849804  7735 rpcz_store.cc:276] 0709 12:28:50.897389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.131161 (+233772us) service_pool.cc:224] Handling call
0709 12:28:51.131560 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.179331 (+ 47771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:51.179395 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:51.179475 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:51.179543 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:52.848783 (+1669240us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b5400144bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:52.848865 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:52.849086 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107785600}
W20260709 12:28:52.852452  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2959) took 2198 ms (client timeout 2098 ms). Trace:
I20260709 12:28:52.834888 15688 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.879503 15656 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.879964 15746 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.887404 15788 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:52.887866  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:52.888496 15788 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:52.861333  7804 rpcz_store.cc:269] 0709 12:28:50.653334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.671013 (+ 17679us) service_pool.cc:224] Handling call
0709 12:28:52.851933 (+2180920us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54003ea3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:52.852191 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163164928}
I20260709 12:28:52.891542 15788 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.891736  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:52.892377  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.892665  8413 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:52.880251 15656 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.896296  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:52.870668 15688 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:52.888010 15681 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.913995 15581 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:52.915800 15656 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:52.918323 15681 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:52.912626  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45762.5us
I20260709 12:28:52.920763  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:52.923699  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:52.926707 15645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.946s	user 0.045s	sys 0.065s
W20260709 12:28:52.927368 15645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.996s	user 0.046s	sys 0.065s
I20260709 12:28:52.927181 15808 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.929646 15808 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:52.933187 15808 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:52.933238 15809 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
W20260709 12:28:52.951454  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.958801  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:52.962599  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 66860.9us
I20260709 12:28:52.964210  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:52.965652 15677 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:52.966145 15677 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.966565 15677 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:52.968755  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:52.974347 15710 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.975255 15710 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:52.975823  7624 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:52.992097  8413 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:53.003780  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1293) took 1825 ms. Trace:
I20260709 12:28:53.008917  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.012439  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:53.016247  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:53.018579  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50098.6us
I20260709 12:28:53.020450 15651 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.020720 15766 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:53.021543 15766 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:53.022708  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1473) took 3640 ms. Trace:
I20260709 12:28:53.023175  7654 rpcz_store.cc:276] 0709 12:28:49.382497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.641476 (+258979us) service_pool.cc:224] Handling call
0709 12:28:49.641791 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.007857 (+1366066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.022064 (+2014207us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540021a7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:53.022471 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995548544}
I20260709 12:28:53.017702 15677 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.037846 15677 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.012679  7807 rpcz_store.cc:276] 0709 12:28:51.171950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:52.717340 (+1545390us) service_pool.cc:224] Handling call
0709 12:28:52.718066 (+   726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:52.718789 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:52.806534 (+ 87745us) log.cc:844] Serialized 14 byte log entry
0709 12:28:52.996861 (+190327us) log.cc:844] Serialized 14 byte log entry
0709 12:28:52.997075 (+   214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:52.997129 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:52.997205 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:52.997276 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:52.997417 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119227,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":157,"raft.queue_time_us":821,"raft.run_wall_time_us":9240,"spinlock_wait_cycles":12208768,"thread_start_us":195261,"threads_started":2}
I20260709 12:28:53.040517  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.005708 15399 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:28:53.004964 15802 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:53.012041 15806 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:53.054131  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1401) took 3670 ms (client timeout 2198 ms). Trace:
I20260709 12:28:53.056368 15681 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:53.058646  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1292) took 3131 ms (client timeout 1667 ms). Trace:
W20260709 12:28:53.059286  7811 rpcz_store.cc:269] 0709 12:28:49.927187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.021474 (+ 94287us) service_pool.cc:224] Handling call
0709 12:28:53.058163 (+3036689us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003728d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:53.058398 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497423104}
I20260709 12:28:53.060149  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.061481  8413 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 12:28:53.054736  8414 rpcz_store.cc:269] 0709 12:28:49.383207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.387708 (+  4501us) service_pool.cc:224] Handling call
0709 12:28:53.028452 (+3640744us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540046d7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:53.053881 (+ 25429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809256320}
I20260709 12:28:53.063674  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:53.063925  7806 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:53.076804  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.077675  7808 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:53.022985 15766 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.078059 15736 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.096609  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3152) took 3346 ms. Trace:
I20260709 12:28:53.097546  7660 rpcz_store.cc:276] 0709 12:28:49.749705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.749945 (+   240us) service_pool.cc:224] Handling call
0709 12:28:49.750302 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:49.774878 (+ 24576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:49.774937 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:49.775968 (+  1031us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.227259 (+451291us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:50.653343 (+426084us) log.cc:844] Serialized 14 byte log entry
0709 12:28:53.056517 (+2403174us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b18003638e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:53.095554 (+ 39037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.095712 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:53.095811 (+    99us) raft_consensus.cc:1685] finished
0709 12:28:53.095962 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.096260 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10453,"prepare.queue_time_us":423993,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":11341,"raft.queue_time_us":132817,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":32429,"spinlock_wait_cycles":17732864,"thread_start_us":409703,"threads_started":2,"wal-append.queue_time_us":892746,"wal-append.run_cpu_time_us":4290,"wal-append.run_wall_time_us":1194699}
I20260709 12:28:53.073303 15806 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.074091 15802 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:53.094543  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.121455  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:28:53.121605  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:53.131930 15724 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 3.345s	user 0.024s	sys 0.042s
I20260709 12:28:53.132854 15802 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.133814 15802 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.137233  7660 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:28:53.139334 15677 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:53.139559  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.140446 15677 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:53.141723  7627 connection.cc:256] RPC call timeout handler was delayed by -1.9929102709998006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
W20260709 12:28:53.142407  7627 connection.cc:256] RPC call timeout handler was delayed by -1.9425102709997191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 12:28:53.142786  7627 connection.cc:256] RPC call timeout handler was delayed by -1.92451027099969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 12:28:53.111011 15806 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:53.134524 15491 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:53.152988  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3527) took 3050 ms. Trace:
I20260709 12:28:53.147784  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.174161  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.174518  8413 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:53.154778  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.175686 15592 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.176055  7808 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:53.146679  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.160724 15818 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:53.143695  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.177616 15818 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.148856  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:53.178548  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1281) took 3344 ms (client timeout 2109 ms). Trace:
W20260709 12:28:53.179103  7805 rpcz_store.cc:269] 0709 12:28:49.834169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.834298 (+   129us) service_pool.cc:224] Handling call
0709 12:28:53.178146 (+3343848us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:53.178347 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280040704}
I20260709 12:28:53.143103 15677 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.148483 15771 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.178607  7654 raft_consensus.cc:2368] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19.
I20260709 12:28:53.144644 15802 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.148356  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.144109  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.184085  7804 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:53.184432  8405 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
W20260709 12:28:53.171468  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 120525us
I20260709 12:28:53.198261  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.180419  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.167867  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:53.199344  7654 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:53.199946  8409 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:53.161720  7656 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
W20260709 12:28:53.160127  8379 connection.cc:256] RPC call timeout handler was delayed by -2.3701439520004897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
W20260709 12:28:53.204988  8379 connection.cc:256] RPC call timeout handler was delayed by -2.3539439520004635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 12:28:53.205600  8379 connection.cc:256] RPC call timeout handler was delayed by -2.0233952830003545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
W20260709 12:28:53.205981  8379 connection.cc:256] RPC call timeout handler was delayed by -1.9520081720002054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 12:28:53.206372  8379 connection.cc:256] RPC call timeout handler was delayed by -1.8886439520001659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2644s for a response
I20260709 12:28:53.207585 15490 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:53.208346  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3605) took 2325 ms. Trace:
I20260709 12:28:53.212494  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:53.221275  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.162297  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.223865  7659 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:53.224785  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.225447  7654 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:53.225852  7803 raft_consensus.cc:2368] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
W20260709 12:28:53.226436  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 2966) took 2571 ms (client timeout 1949 ms). Trace:
I20260709 12:28:53.227030  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.229287 15588 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:53.232528  7659 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:28:53.232990  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3261) took 2218 ms (client timeout 1785 ms). Trace:
W20260709 12:28:53.233707  7662 rpcz_store.cc:269] 0709 12:28:51.014744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.024519 (+  9775us) service_pool.cc:224] Handling call
0709 12:28:53.232419 (+2207900us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400116a54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:53.232697 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852071808}
I20260709 12:28:53.235078 15692 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:53.227104  7803 rpcz_store.cc:269] 0709 12:28:50.655135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.669738 (+ 14603us) service_pool.cc:224] Handling call
0709 12:28:53.224391 (+2554653us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b5800003f98. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:28:53.226329 (+  1938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3993472}
I20260709 12:28:53.237825  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.238026  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1641) took 2465 ms. Trace:
I20260709 12:28:53.238610  7733 rpcz_store.cc:276] 0709 12:28:50.772183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.772348 (+   165us) service_pool.cc:224] Handling call
0709 12:28:50.772725 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.045050 (+272325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.237313 (+2192263us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540007f3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:53.237718 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623628544}
I20260709 12:28:53.239576 15778 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:28:53.239991  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.88509e+06us
I20260709 12:28:53.241308  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.240221 15778 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:53.207149  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5845653330002278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 12:28:53.199044  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:53.208953  7729 rpcz_store.cc:276] 0709 12:28:50.883011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.883202 (+   191us) service_pool.cc:224] Handling call
0709 12:28:50.903635 (+ 20433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:50.950048 (+ 46413us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:50.965554 (+ 15506us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.129578 (+164024us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.484995 (+355417us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b1800093460. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:53.207037 (+1722042us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b1800093460. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:28:53.207455 (+   418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:53.207594 (+   139us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:53.207737 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.207891 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.208101 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106808,"prepare.queue_time_us":22541,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":1539,"raft.queue_time_us":45982,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"spinlock_wait_cycles":66840448,"thread_start_us":117240,"threads_started":1,"wal-append.queue_time_us":403159,"wal-append.run_cpu_time_us":2168,"wal-append.run_wall_time_us":1132286}
W20260709 12:28:53.143077  7627 connection.cc:256] RPC call timeout handler was delayed by -1.9074102709996623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
W20260709 12:28:53.261741  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8867102709996288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 12:28:53.262262  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8849102709996259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 12:28:53.222244 15688 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.242528  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:53.264159  7627 connection.cc:256] RPC call timeout handler was delayed by -1.8201102709999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
I20260709 12:28:53.175575  7806 rpcz_store.cc:276] 0709 12:28:50.102878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.273747 (+170869us) service_pool.cc:224] Handling call
0709 12:28:53.063397 (+2789650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:53.071809 (+  8412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.152494 (+ 80685us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b54003728d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:53.152845 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6414648064}
I20260709 12:28:53.277633  7803 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:53.280714  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1285) took 3402 ms. Trace:
I20260709 12:28:53.281104 14906 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.282074 14906 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.282512 15639 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:53.283816 15639 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.284564 15639 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.285717 14906 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.286514 14906 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:53.287253 15692 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.288643 15606 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:53.289402 15606 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:53.211090 15818 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.291917 15613 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:53.292506 15613 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:53.295423  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:53.293939 15613 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.296470  8412 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:53.297010  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 97903.7us
I20260709 12:28:53.297649  8409 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:53.291455  7812 rpcz_store.cc:276] 0709 12:28:49.840039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.845644 (+  5605us) service_pool.cc:224] Handling call
0709 12:28:49.927717 (+ 82073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:50.287497 (+359780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:50.287540 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:50.287589 (+    49us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:50.287633 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.190885 (+2903252us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400132ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:53.190975 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.242099 (+ 51124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826661760}
I20260709 12:28:53.303747  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:53.304425  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 105503us
I20260709 12:28:53.303743  7804 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:53.316385  7808 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:53.298784 15604 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.320473  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.291069 15606 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.323124 15613 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:53.323702  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.324043 15677 consensus_queue.cc:1048] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:53.307289 15808 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:53.324266 14906 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:53.324975  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46170.5us
W20260709 12:28:53.326761  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.327168  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1652) took 2215 ms. Trace:
I20260709 12:28:53.327965 15285 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:53.184896 15657 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:53.291007 15692 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.348050  7812 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
W20260709 12:28:53.355811  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57907.9us
I20260709 12:28:53.358242  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.250512 15710 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.305939 15688 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.208377 15490 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:53.385548  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70813.8us
I20260709 12:28:53.351161 15489 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:53.389384  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5645102710000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
W20260709 12:28:53.393043  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5117466720002994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:28:53.393442  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3737102709997089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22799999999999998s for a response
W20260709 12:28:53.393781  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3029466720004166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 12:28:53.394098  7627 connection.cc:256] RPC call timeout handler was delayed by -1.0752466720000484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 12:28:53.394685  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.360690 15750 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:53.382429 15788 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:53.380822 15818 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:53.403263  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:53.408844 15500 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:53.409394  7808 raft_consensus.cc:2368] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19.
I20260709 12:28:53.409803 15500 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.388660  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:53.411664 15713 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:53.412443  7803 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:53.413158 15500 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.413322  7803 raft_consensus.cc:614] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:53.413991 15818 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.414844 15818 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:53.382983 15808 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:53.422605  8409 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:53.424520  7803 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:53.361796  7737 rpcz_store.cc:276] 0709 12:28:51.111604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:52.688869 (+1577265us) service_pool.cc:224] Handling call
0709 12:28:52.689666 (+   797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:52.690414 (+   748us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:52.809666 (+119252us) log.cc:844] Serialized 14 byte log entry
0709 12:28:53.326674 (+517008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:53.326743 (+    69us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:53.326824 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.326920 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.327091 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":420752,"prepare.queue_time_us":139301,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":421869,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":184817,"threads_started":2,"wal-append.queue_time_us":122147}
W20260709 12:28:53.426786  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.426635  7812 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:28:53.390110 15639 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.364926 15604 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.435551  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1647) took 2541 ms. Trace:
I20260709 12:28:53.435786  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.436988  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.455169  7806 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:53.457163 15657 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:53.458138  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.458822 15772 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:53.459525  8412 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:53.417812 15285 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.466976  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3528) took 3363 ms. Trace:
I20260709 12:28:53.465214  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.439848  7736 rpcz_store.cc:276] 0709 12:28:50.864382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.866035 (+  1653us) service_pool.cc:224] Handling call
0709 12:28:50.866425 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.010201 (+143776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:51.010263 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:51.010336 (+    73us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:51.010406 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.361427 (+2351021us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:53.374410 (+ 12983us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.405613 (+ 31203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329931008}
I20260709 12:28:53.466286  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3544) took 3310 ms. Trace:
W20260709 12:28:53.501200  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66492.1us
I20260709 12:28:53.439818 15806 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.459611 15772 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:53.430611 15489 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.470145  7809 rpcz_store.cc:276] 0709 12:28:50.103166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.274755 (+171589us) service_pool.cc:224] Handling call
0709 12:28:50.275121 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.195778 (+920657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.466417 (+2270639us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b5400010ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:53.466752 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048805760}
I20260709 12:28:53.483670  7810 rpcz_store.cc:276] 0709 12:28:50.155901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:51.024038 (+868137us) service_pool.cc:224] Handling call
0709 12:28:51.024351 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:51.024571 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:51.194816 (+170245us) log.cc:844] Serialized 14 byte log entry
0709 12:28:53.441139 (+2246323us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b1800159280. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287
0709 12:28:53.465602 (+ 24463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:53.465692 (+    90us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:53.465800 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.465911 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.466075 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18278,"prepare.queue_time_us":400289,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1291,"raft.queue_time_us":109806,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":4230,"spinlock_wait_cycles":4515200,"thread_start_us":408110,"threads_started":3,"wal-append.queue_time_us":264949,"wal-append.run_cpu_time_us":1797,"wal-append.run_wall_time_us":1038275}
I20260709 12:28:53.536713  7660 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
W20260709 12:28:53.590492  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 164366us
I20260709 12:28:53.544351  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:53.597046  8409 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:53.612653  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:53.545037 15821 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:28:53.582636 15828 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:28:53.592933 15606 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:53.617933  7656 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:53.627537  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:53.629979  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 144372us
I20260709 12:28:53.544073 15597 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:28:53.632779  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46546.1us
W20260709 12:28:53.635751  7700 connection.cc:256] RPC call timeout handler was delayed by -1.051768441999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 12:28:53.543790 15718 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:53.638213  7661 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
W20260709 12:28:53.643478  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.592s (SENT)
I20260709 12:28:53.650660  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:53.644219 15639 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.654310 15830 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:53.654556 15638 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.655350 15830 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.656638  7658 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:53.618971 15806 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.513557 15750 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.651352  8411 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
W20260709 12:28:53.659395  7777 connection.cc:256] RPC call timeout handler was delayed by -1.0102020519998405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 12:28:53.644192 15581 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.667570  8411 pending_rounds.cc:85] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 16
I20260709 12:28:53.536978 15778 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.679508  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.682649  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.684520  7808 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:53.674158  8411 pending_rounds.cc:107] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:28:53.683070  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.669373 15597 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.690284 15788 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.686805  8411 raft_consensus.cc:2889] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:53.636214 15808 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.677024 15710 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.654415 15639 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.675201 15713 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.695919 15788 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:53.724251 15834 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:28:53.725309  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.725728  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.725309  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.727880  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:53.730222  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:53.765753  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.766379  7662 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:28:53.731617  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.732934  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:53.737602  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1643) took 2963 ms. Trace:
I20260709 12:28:53.739423 15831 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:53.740208 15823 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:28:53.743275 15770 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:53.745492 15489 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.746435  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:53.747756  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.748864  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.702034 15802 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:28:53.708895 15490 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:53.717079 15692 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:28:53.661161 15772 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.726636  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:53.663398 15638 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.762342 15639 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.676746 15581 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.713495 14906 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.785406  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.817114  7655 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:53.802240  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 160264us
I20260709 12:28:53.767377  7732 rpcz_store.cc:276] 0709 12:28:50.773690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.773820 (+   130us) service_pool.cc:224] Handling call
0709 12:28:50.774193 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:50.792845 (+ 18652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:50.792877 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:50.811512 (+ 18635us) log.cc:844] Serialized 26 byte log entry
0709 12:28:53.212684 (+2401172us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b18004a3fe0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:53.472380 (+259696us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:53.500604 (+ 28224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:53.518109 (+ 17505us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:53.737015 (+218906us) raft_consensus.cc:1685] finished
0709 12:28:53.737047 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:53.737382 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18138,"raft.queue_time_us":90472,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":30901,"spinlock_wait_cycles":70861056,"thread_start_us":172238,"threads_started":1,"wal-append.queue_time_us":2611200}
I20260709 12:28:53.714286 15830 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:53.819906  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.769227 15831 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:53.817370  7657 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:53.819037  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:53.819913  7810 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: [19.14-22.15]
I20260709 12:28:53.815745  7735 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:53.832118 15750 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.833066 15758 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.834029 15758 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.835457  7733 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:53.835925  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.836649  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.837548  7728 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:53.837610  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:53.839854  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:53.847656  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.848057  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.856514  7660 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
W20260709 12:28:53.851809  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.848549  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:53.836671  7731 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:28:53.871467  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 228593us
W20260709 12:28:53.835492 15832 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.067s	user 0.001s	sys 0.001s
I20260709 12:28:53.873057  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:53.879729  7662 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:53.841928  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:53.884512  7656 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:53.844689 15490 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.887480  7734 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:53.841032 15831 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.898434  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:53.896786  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 198600us
I20260709 12:28:53.894940  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.900231  7731 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:53.893452 15613 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.003s
I20260709 12:28:53.853519 15770 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:53.912299  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.648s (SENT)
I20260709 12:28:53.875723 15818 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:53.915719  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.924165 15490 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:28:53.932791 15836 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 12:28:53.938081  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.949743  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.950588  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:53.951669 15490 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:53.952131 15490 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.952515 15490 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:53.956198  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:53.959430 15607 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:53.961192  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.961875 15242 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:53.962051  7730 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:53.962741 15693 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:53.946854 15597 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.963160  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:53.971167  7810 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:28:53.971318  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:54.001642  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:54.002281 15638 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:54.003589  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:54.003928 15581 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:54.004424  8411 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
W20260709 12:28:54.005263  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 163486us
I20260709 12:28:54.005616 15776 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:54.006357  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:54.006426  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:54.007637 15638 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:54.008144 15638 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.008224 15772 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:54.008615 15638 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:53.966356 15285 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:54.011620  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107892us
W20260709 12:28:54.014288  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.745s (SENT)
I20260709 12:28:54.014822  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:53.976934  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.981741  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.996050  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.982160  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:53.987531  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:53.995121  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:53.974015  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:54.015914  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:54.019707  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:54.020413  7657 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:54.020998  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.021804  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:54.022773 15638 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.098271 15638 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:53.978569 15806 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:53.968974 15750 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:28:54.022775 15490 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.099838  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:54.100601  8411 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:28:54.025362  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:54.026322  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 187105us
I20260709 12:28:54.026892 15731 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:54.030340 15604 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:53.968452 15758 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:54.032210  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4683695970002191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 12:28:54.113608  7702 connection.cc:256] RPC call timeout handler was delayed by -2.4062695970001187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
W20260709 12:28:54.114063  7702 connection.cc:256] RPC call timeout handler was delayed by -2.33606959700046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:28:54.114392  7702 connection.cc:256] RPC call timeout handler was delayed by -2.29286959700039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
W20260709 12:28:54.114668  7702 connection.cc:256] RPC call timeout handler was delayed by -2.2370695970003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 12:28:54.114961  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1794695970002067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 12:28:54.115269  7702 connection.cc:256] RPC call timeout handler was delayed by -2.1578695970001718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 12:28:54.115573  7702 connection.cc:256] RPC call timeout handler was delayed by -2.102969597000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 12:28:54.115859  7702 connection.cc:256] RPC call timeout handler was delayed by -2.0237695970004097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 12:28:54.116153  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4720695970004272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2711s for a response
I20260709 12:28:54.111622  7732 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:54.116415  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4201697180001247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2295s for a response
I20260709 12:28:54.032764 15718 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:54.033305 15613 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.033769 14906 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:54.034313 15285 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.035420 15831 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:54.035527 15606 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:54.036289 15775 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.040799 15770 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:54.041349  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55643.1us
I20260709 12:28:54.132018  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:54.050705  7655 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:28:54.051445  7729 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:53.997848  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:54.061597  7658 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:54.063488 15818 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.063797  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:54.135874  7654 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:54.067597  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.068297 15778 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:54.074473  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:54.075546  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:54.077014  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:54.080310  7661 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:54.080801 15692 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.081378 15838 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:54.138460  7657 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:54.081421  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67185.8us
I20260709 12:28:54.138636 15692 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:54.139938  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.784s (SENT)
I20260709 12:28:54.139971  7733 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:54.140592  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.141299 15692 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.142140  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.088759 15788 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.143471 15699 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.090173  7728 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:54.144204 15699 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.045979  7662 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:54.147557  7733 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:54.090947  7656 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:54.095623  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:54.103672  7734 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:54.106925  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:54.107616  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 120272us
I20260709 12:28:54.152542  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:54.101477 15490 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.124998  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.117779 15776 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:54.126304 15831 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.126423 14906 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.128064 15285 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.165334  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.129347 15606 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.130940 15613 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.160584  7653 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:54.162515  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:54.182276  7735 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 23
W20260709 12:28:54.166384  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59592.4us
I20260709 12:28:54.166465  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.183648  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:54.173379  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.132911 15802 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:54.178521  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:54.192371  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:54.160423 15776 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.175212 15731 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:54.195034  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.209074  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:54.136854 15775 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.208312  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:54.210304  7806 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:54.210398  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:54.212236  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:54.213164  7812 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:54.200558 15491 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:54.200873 15490 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:54.148645 15718 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.208987 15638 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:54.220569  7731 raft_consensus.cc:2376] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20. Re-sending same reply.
I20260709 12:28:54.205793 15776 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:54.213110 15597 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:54.259434  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.232859 15831 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:54.234879  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97646.2us
I20260709 12:28:54.236372  7804 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:54.238062  7734 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:28:54.239411  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61410.7us
I20260709 12:28:54.267436  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.217690 15818 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:54.239919  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 102298us
I20260709 12:28:54.240414 15803 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:54.241045 15731 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:54.242055 15802 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.220901  7732 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:54.247504  7808 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:54.248983  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:54.256721  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.246248  7805 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:28:54.252332  7733 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:54.230705 15242 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:54.257170 15491 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:54.293115 15491 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.294366 15285 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.245625 15613 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.261585 15644 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:28:54.296218 15606 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:54.208163 15742 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:28:54.299263  8411 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:28:54.283766  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 160873us
I20260709 12:28:54.285254 15802 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:54.286182 15718 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.252403 15750 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:28:54.248546 15490 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:54.279085  7730 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:54.280424 15731 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.302711  7728 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:28:54.303012  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 221516us
I20260709 12:28:54.308380  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:54.317737  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:54.312129 15490 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:54.323690  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56369us
I20260709 12:28:54.316174 15657 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:54.313472 15699 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:54.338505  7700 leader_election.cc:336] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.098s (SENT)
I20260709 12:28:54.352205  7728 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:54.328183 15500 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:54.331285 15581 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:54.359184  7658 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:54.335705 15613 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:54.368973  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63882.2us
I20260709 12:28:54.369762  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:54.319813 15818 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.374555  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.371243  8409 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:54.376262 14906 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.391785 15597 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:54.392380 15597 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:54.392839 15597 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:54.356231 15750 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.360594 15500 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:54.395566 15750 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.396504 15500 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.397866 15500 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:54.366120 15838 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:54.419934  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:54.421748  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:54.372818 15788 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:54.431692  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.425977  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:54.440722 15607 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:54.452099 15581 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:54.483945  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 234247us
W20260709 12:28:54.486081  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.486768  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:54.462035 15718 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.466847 15607 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:28:54.514721  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77805.3us
W20260709 12:28:54.517091  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 243620us
I20260709 12:28:54.471789 15750 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.476401 15581 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:54.504581 15638 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:54.541512  7735 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:28:54.475579 15584 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:28:54.510648 15776 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.479480 15806 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:28:54.523311 15607 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:54.595415  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:54.601472  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.536550 15581 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:54.583494  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:54.537567  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.660835  7729 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:28:54.611196  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 78280.7us
W20260709 12:28:54.627197  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 264264us
W20260709 12:28:54.640511  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 254731us
I20260709 12:28:54.665686  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:54.529170 15718 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.681242 15776 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:54.672793  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 108280us
I20260709 12:28:54.684281  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.673664 15788 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:54.687873 15788 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:54.689369 15788 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:54.674825 15778 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.513s	user 0.014s	sys 0.002s
W20260709 12:28:54.697389  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 324310us
I20260709 12:28:54.604789 15613 raft_consensus.cc:2764] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:28:54.698928  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:54.604755 15692 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:54.676141  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.639856 15588 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:54.700598  7729 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:54.695269 15584 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:54.703583  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.694034  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:54.699009 15845 log.cc:927] Time spent T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 12:28:54.724648  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:28:54.715629  7735 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:28:54.716310 15846 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.219s	user 0.001s	sys 0.001s
W20260709 12:28:54.729374  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65251us
I20260709 12:28:54.731567  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:54.733847  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.741866  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:54.742429  7729 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:54.745626  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.717053 14906 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.714375  7803 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:28:54.751693  7809 raft_consensus.cc:606] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.752130  7809 raft_consensus.cc:614] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:54.753969 15872 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:54.756230 15872 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:54.757124 15872 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:54.758307 15872 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.760025 15873 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:54.768013  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 287178us
W20260709 12:28:54.769343  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.760705 15873 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.730365 15692 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:54.733851 15718 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.743513 15588 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.795532  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.768412 15818 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.767767 15858 raft_consensus.cc:2764] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
W20260709 12:28:54.817579 15857 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.217s	user 0.001s	sys 0.003s
W20260709 12:28:54.808382  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58260.4us
I20260709 12:28:54.783376 15873 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:54.809451 15863 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:54.810559  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111806us
I20260709 12:28:54.813512 15588 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:54.816211 15843 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:54.773221 15852 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.828394 15775 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:54.829690  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 136894us
I20260709 12:28:54.771970 15638 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.771337 14906 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:54.854918 15490 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.005s
I20260709 12:28:54.802376 15770 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:54.816176 15772 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:54.827638 15692 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:54.840340  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.875597 14906 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:54.881304  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50788 (request call id 54) took 1084 ms. Trace:
I20260709 12:28:54.881862  7809 rpcz_store.cc:276] 0709 12:28:53.797538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:53.805014 (+  7476us) service_pool.cc:224] Handling call
0709 12:28:54.881016 (+1076002us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b5400133dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f1c853 00007fc006eea60a 00007fc0004707d4 00007fc000470762 00007fc000470691 00007fc00047060c 00007fc00046f8c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:54.881175 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2139160320}
W20260709 12:28:54.884008  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3557) took 1744 ms (client timeout 1958 ms). Trace:
W20260709 12:28:54.884555  7805 rpcz_store.cc:269] 0709 12:28:53.139976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:53.180082 (+ 40106us) service_pool.cc:224] Handling call
0709 12:28:54.883785 (+1703703us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:54.883915 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407036160}
I20260709 12:28:54.829627 15584 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:28:54.848335  7806 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.013s
I20260709 12:28:54.895025  7806 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
W20260709 12:28:54.853832  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58206.3us
W20260709 12:28:54.859573  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 110723us
I20260709 12:28:54.870992 15490 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.867753 15597 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.914234 15876 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.006s
W20260709 12:28:54.914780  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54345.8us
I20260709 12:28:54.915041 15584 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:54.915643 15772 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:54.916356  7778 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.935828  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:54.936036  7778 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.936573 15806 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:54.937093 15806 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.937486 15806 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:54.937788  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:54.938804 15588 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:54.917017 15489 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:54.939265 15588 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.939559 15489 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:54.939738 15588 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:54.940816 15489 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.942219 15489 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:54.943463 15879 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:54.943987 15879 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:54.944281 15597 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:54.944429 15879 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:54.945169  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.860353 15863 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:54.867762 15852 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.932430  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:54.963166  8411 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:54.933240 15490 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:54.934927  7803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:54.965797  7803 raft_consensus.cc:606] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:54.945361  7729 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:28:54.933286  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.927172 15500 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:54.968220  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:54.970329 15843 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:54.970526 15693 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:54.971709 15693 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:54.944548 15584 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:54.897374 15581 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:54.990726  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57540.6us
W20260709 12:28:54.993534  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 126927us
I20260709 12:28:54.995347  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:54.991125  8411 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:28:54.996691  8409 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:54.994745  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:54.998829  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:54.999691  7803 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:55.000034  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.001590  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:55.004621  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:55.004675  8379 leader_election.cc:400] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19.
I20260709 12:28:55.009830  8379 leader_election.cc:403] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:28:54.988849 15772 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:55.015415  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51828.9us
I20260709 12:28:55.019501  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:55.020141  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:55.021653  7806 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:55.024252  7807 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:55.023087  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:55.029366  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:55.030125  7728 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:55.035744  7810 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:54.999014 15776 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:55.006745 15818 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.011935 15597 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:55.013458 15871 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.037504  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:55.047258  7803 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:55.048173  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:55.016089 15866 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:55.048813  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:54.983397 15500 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:55.059403  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 163575us
I20260709 12:28:55.032779 15876 raft_consensus.cc:2764] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:28:55.037634 15867 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:55.068192  7735 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:28:55.050164 15597 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:28:55.086838  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.073496  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:55.080832  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.052577 15713 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19.
I20260709 12:28:55.103077  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:55.106423  8411 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 16: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:55.060688 15491 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:55.105192  7735 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:55.098924  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.060s (SENT) [suppressed 34 similar messages]
I20260709 12:28:55.107800  7728 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:28:55.108800  7731 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:54.997905 15693 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.115320  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:55.054664 15875 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:28:55.067369 15866 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:55.063304 15731 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:55.106887 15597 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:55.130151  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:55.131204  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:55.132933  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:55.054032 15818 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.124884 15491 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:55.137059 15607 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.137660 15491 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:55.142941  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:55.147069  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.148756  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:55.154192  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.158885  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:55.061071 15871 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.165132  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:55.168694  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:55.171743  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.172354  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.179324 15607 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:55.179831  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.181813  7735 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:55.067685 15843 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.172748 15770 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:55.214424  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:55.127909 15731 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.215955  7735 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:28:55.218169  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:55.214712  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 158772us
I20260709 12:28:55.222287  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.207440 15607 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:28:55.233587  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 126002us
I20260709 12:28:55.235890  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:55.236351  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:28:55.207880 15867 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:55.237243 15876 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:55.237567 15867 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:55.237936 15876 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:55.238312 15802 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:55.238123 15867 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:55.238603 15876 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:55.240736 15802 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:55.241654 15802 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:55.242022 15802 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:55.209590 15889 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:28:55.249538  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:55.211078 15581 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:55.254307  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 89295.9us
I20260709 12:28:55.257047  8407 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:55.220993 15863 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:55.261976 15884 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:28:55.263024 15890 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:28:55.263453 15775 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.265069 15818 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:55.266258 15899 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:55.224609 15898 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:55.290902  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.290501 15898 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.283923  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:55.228614 15882 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:55.299153 15581 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:28:55.300164 15893 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:28:55.242988 15802 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:55.303355 15802 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:55.304425  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68421.8us
I20260709 12:28:55.278431 15242 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:55.305032  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:55.305492  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:55.305763  7811 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:28:55.307785  7805 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:55.249377 15866 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:55.309036 15860 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.507s	user 0.000s	sys 0.003s
I20260709 12:28:55.313076  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.316875  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:55.290503 14906 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:55.317595  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3642) took 1470 ms. Trace:
I20260709 12:28:55.296010 15731 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:55.330718  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46946.7us
I20260709 12:28:55.321961 14906 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:55.336651 14906 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:55.328941 15818 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:28:55.335752 15772 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:55.306748 15882 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.335882  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.328474  7805 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:28:55.341185  7658 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:55.341711  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.338009 15818 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:55.295322 15898 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:55.322415  7736 rpcz_store.cc:276] 0709 12:28:53.854681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.257734 (+403053us) service_pool.cc:224] Handling call
0709 12:28:54.298671 (+ 40937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.303965 (+  5294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:54.304002 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:54.335050 (+ 31048us) log.cc:844] Serialized 26 byte log entry
0709 12:28:54.566861 (+231811us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:54.566925 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:54.568179 (+  1254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:55.308346 (+740167us) spinlock_profiling.cc:243] Waited 9.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:55.310750 (+  2404us) raft_consensus.cc:1685] finished
0709 12:28:55.310789 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.317521 (+  6732us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":369,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":6358,"spinlock_wait_cycles":22319872,"thread_start_us":189919,"threads_started":1,"wal-append.queue_time_us":386861}
I20260709 12:28:55.340456  7805 raft_consensus.cc:606] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:55.323942 15770 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:55.290417 15775 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:55.288574 15873 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.305410 15242 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.327754 15750 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:55.368023  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.309075 15588 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:55.375864  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:56.120402  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.741886  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.121999  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:28:56.122659  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.507962  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:56.020195  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.006s
I20260709 12:28:55.539687 15644 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.123848  8414 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:55.883996  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1436) took 1081 ms. Trace:
W20260709 12:28:55.532437  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.489431  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.124722  8409 rpcz_store.cc:276] 0709 12:28:54.803253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.803513 (+   260us) service_pool.cc:224] Handling call
0709 12:28:54.834055 (+ 30542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:54.997139 (+163084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.883511 (+886372us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:55.883840 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362110848}
I20260709 12:28:55.569169 15873 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:55.821759 15867 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.582s	user 0.022s	sys 0.004s
W20260709 12:28:56.122785  7803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.010s
W20260709 12:28:55.453810  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 148430us
W20260709 12:28:56.127075  7803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.010s
I20260709 12:28:55.419087  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:55.880051 15856 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.063s	user 0.000s	sys 0.004s
I20260709 12:28:56.119112  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:56.128893  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:55.492831 15892 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.129755  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1433) took 2023 ms. Trace:
I20260709 12:28:56.121063  7811 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:56.130553 15892 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.124814 15588 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:56.130505  8412 rpcz_store.cc:276] 0709 12:28:54.106504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.106698 (+   194us) service_pool.cc:224] Handling call
0709 12:28:54.121638 (+ 14940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.140366 (+ 18728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:54.140397 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:54.333129 (+192732us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b2c000e5b88. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:54.333607 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:28:54.506075 (+172468us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:54.506137 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:54.538401 (+ 32264us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b4800090258. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:54.914368 (+375967us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:54.914441 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:55.871016 (+956575us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:28:56.129422 (+258406us) raft_consensus.cc:1685] finished
0709 12:28:56.129456 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.129671 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1888,"raft.run_cpu_time_us":5640,"raft.run_wall_time_us":401559,"spinlock_wait_cycles":311713536,"thread_start_us":171894,"threads_started":1,"wal-append.queue_time_us":434466}
I20260709 12:28:56.124136 15644 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:55.890082 15852 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:55.545984  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:56.134662 15644 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:56.142784  7657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.013s	sys 0.015s
W20260709 12:28:55.864984 15876 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.003s
I20260709 12:28:55.415431  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:55.451738  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:56.150511  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.007s	sys 0.009s
W20260709 12:28:56.151054  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.007s	sys 0.009s
I20260709 12:28:56.150817  7655 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
W20260709 12:28:56.151510  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.978457  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:56.151988  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:56.123679  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.005s	sys 0.008s
I20260709 12:28:55.352802 15818 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:28:56.143421  7657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.013s	sys 0.015s
I20260709 12:28:55.451391  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:28:55.793350 15788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.005s	sys 0.007s
W20260709 12:28:55.565426 15897 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 12:28:55.892807 15879 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.948s	user 0.016s	sys 0.000s
I20260709 12:28:55.557716  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:55.351331 15882 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:56.117937 14906 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.002s
I20260709 12:28:56.189189  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:55.457878 15742 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:56.149111 15802 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.845s	user 0.014s	sys 0.000s
W20260709 12:28:56.151075  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.493952 15896 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:28:56.191190  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:56.136843 15866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.010s	sys 0.007s
W20260709 12:28:55.869071 15908 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.090s	user 0.002s	sys 0.000s
I20260709 12:28:55.457742 15500 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:56.137185 15750 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.417673 15806 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:55.668582 15847 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.146s	user 0.008s	sys 0.000s
I20260709 12:28:55.367563 15770 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.137768 15606 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:55.570914 15242 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:56.372570  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:56.197079 14906 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.221833  7654 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:28:56.198864  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 220534us
I20260709 12:28:56.348718  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:56.344583  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:56.423751 15489 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.481s	user 0.013s	sys 0.004s
W20260709 12:28:56.433265  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 304500us
I20260709 12:28:56.433924  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:56.224886  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:56.447706  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 325336us
I20260709 12:28:56.444706  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:56.447584  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:56.467737  7811 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:56.469367  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3287) took 1094 ms. Trace:
I20260709 12:28:56.186328 15584 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.470036  7654 rpcz_store.cc:276] 0709 12:28:55.352622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.357861 (+  5239us) service_pool.cc:224] Handling call
0709 12:28:56.446754 (+1088893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:56.223008 15858 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.439416 15770 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:56.243414  7653 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:28:56.485230  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 212125us
I20260709 12:28:56.485486  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3285) took 1133 ms. Trace:
I20260709 12:28:56.156380 15644 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.486256 15644 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.488055  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:56.488890  7732 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:55.588753 15887 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:28:56.220661 15861 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.499s	user 0.010s	sys 0.012s
W20260709 12:28:56.490067 15861 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.793s	user 0.017s	sys 0.012s
W20260709 12:28:56.255934  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97124.4us
W20260709 12:28:56.190514 15788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.009s	sys 0.008s
I20260709 12:28:56.491468 15788 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.492141  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:56.492033 15788 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.493753  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:56.494563 15788 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:56.494647  7808 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:28:56.407958 15892 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:56.495627  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:56.496654  7730 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:28:56.496809 15917 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.497634  7737 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:56.495745  7700 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.700s (SENT)
W20260709 12:28:56.497995  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 50503us
I20260709 12:28:56.253181  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:56.171180 15613 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:56.478880  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:56.440267 15581 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:56.329362  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 76253.9us
I20260709 12:28:56.185822 15876 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.329774 15775 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:56.468575 15491 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.254792  7807 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:28:56.477747  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:56.274905 15873 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:56.275554 15802 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.208s
I20260709 12:28:56.501004  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:56.504102  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:56.142356 15867 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.444389 15750 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:56.505860  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:56.476212 15597 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.330s	user 0.014s	sys 0.001s
I20260709 12:28:56.506750  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:56.446527 15852 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.510505  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:56.501924  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.162s (SENT)
I20260709 12:28:56.489141 15644 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:56.352324 15871 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:56.606020  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.586555  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:56.511904  7809 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
W20260709 12:28:56.438975 15866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.010s	sys 0.034s
I20260709 12:28:56.608096  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3659) took 1561 ms. Trace:
I20260709 12:28:56.519090  8407 raft_consensus.cc:2410] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:28:56.490159  7653 rpcz_store.cc:276] 0709 12:28:55.351977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.352156 (+   179us) service_pool.cc:224] Handling call
0709 12:28:56.150232 (+798076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.485154 (+334922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.485396 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9856}
I20260709 12:28:56.444726 15588 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:56.522323  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3574) took 1252 ms. Trace:
I20260709 12:28:56.523813  7808 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:56.519300  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:56.533378  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:28:56.548760  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.504280 15584 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.223448 15742 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.436887 15606 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.506021 15581 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:56.556164 15911 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:28:56.399295 15818 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:56.557235 15921 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:28:56.566516  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:56.194202 15879 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.519913 15731 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.483377 15858 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.576025 15918 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:56.440433 15891 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:28:56.514060 15917 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:56.580572 15909 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.456s	user 0.004s	sys 0.000s
I20260709 12:28:56.583557 14906 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:56.583752 15491 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.555590 15775 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:56.588183  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.600682  7658 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:28:56.601524  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85914.1us
I20260709 12:28:56.472074 15242 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:56.614499  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:56.545303 15613 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.608796  7734 rpcz_store.cc:276] 0709 12:28:55.047276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.047465 (+   189us) service_pool.cc:224] Handling call
0709 12:28:55.047958 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:55.063068 (+ 15110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.063121 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:55.101438 (+ 38317us) log.cc:844] Serialized 52 byte log entry
0709 12:28:55.224961 (+123523us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:55.234930 (+  9969us) log.cc:844] Serialized 14 byte log entry
0709 12:28:55.253704 (+ 18774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.253836 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.607725 (+1353889us) raft_consensus.cc:1685] finished
0709 12:28:56.607756 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.607996 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17969,"prepare.queue_time_us":10035,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":55256,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":38558,"thread_start_us":89916,"threads_started":1,"wal-append.queue_time_us":152757}
I20260709 12:28:56.615854  7654 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:56.676506  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:56.642371  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 123215us
I20260709 12:28:56.681270  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3296) took 1229 ms. Trace:
I20260709 12:28:56.616245  7804 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:56.637779  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:56.614506  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:56.681563 15775 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:56.688942 15775 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:56.682143  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:56.640111  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:56.616884  7732 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:56.693845  7657 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:56.707681  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:56.732257  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 303044us
I20260709 12:28:56.616607  7803 rpcz_store.cc:276] 0709 12:28:55.260010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.260166 (+   156us) service_pool.cc:224] Handling call
0709 12:28:55.260549 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:55.282623 (+ 22074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.282659 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:55.597797 (+315138us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:55.598754 (+   957us) log.cc:844] Serialized 52 byte log entry
0709 12:28:56.127859 (+529105us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:56.440941 (+313082us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.496302 (+ 55361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.496387 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.497296 (+   909us) raft_consensus.cc:1685] finished
0709 12:28:56.511635 (+ 14339us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.511810 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111462,"prepare.queue_time_us":439663,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":10379,"raft.queue_time_us":88835,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":27785,"spinlock_wait_cycles":13772032,"thread_start_us":840981,"threads_started":2,"wal-append.queue_time_us":650964}
W20260709 12:28:56.714331  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 212520us
W20260709 12:28:56.733556  7627 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.175s (SENT)
I20260709 12:28:56.734130  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:56.714875  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 231018us
I20260709 12:28:56.685626  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:56.696457  7809 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
W20260709 12:28:56.716502  7700 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 2.170s (SENT)
I20260709 12:28:56.490085 15887 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.682957  7658 rpcz_store.cc:276] 0709 12:28:55.452370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.125779 (+673409us) service_pool.cc:224] Handling call
0709 12:28:56.224628 (+ 98849us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:28:56.681196 (+456568us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29963520}
I20260709 12:28:56.615631 15919 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:56.739274  7735 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:28:56.691829 15581 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:56.642562 15900 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:56.739601  7810 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:56.612800 15873 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.739583  8411 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:28:56.745074  7730 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
W20260709 12:28:56.746940  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 61469.5us
I20260709 12:28:56.609054 15597 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.611464 15852 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.759569  7655 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:56.631714 15871 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 12:28:56.759850  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:28:56.764721  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 266022us
I20260709 12:28:56.631994 15588 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.697315 15731 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.771445  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3291) took 1399 ms. Trace:
I20260709 12:28:56.766331  7777 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:56.743520 15876 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:56.777778  7660 rpcz_store.cc:276] 0709 12:28:55.371618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.371890 (+   272us) service_pool.cc:224] Handling call
0709 12:28:56.755750 (+1383860us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:28:56.755979 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.770906 (+ 14927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.770952 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.771014 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:56.771074 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.771162 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.771359 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13546880}
I20260709 12:28:56.755110 15644 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:56.783392  7662 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:56.750666 15242 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.785185  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:56.779904  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:56.782975  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:56.790987  7812 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:56.771111  7624 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:56.794667  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3298) took 1321 ms. Trace:
I20260709 12:28:56.796515  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:56.758462 15806 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:56.797446  7655 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:56.746737 15803 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:28:56.795842  7661 rpcz_store.cc:276] 0709 12:28:55.473698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.349954 (+876256us) service_pool.cc:224] Handling call
0709 12:28:56.410044 (+ 60090us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.410409 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.410452 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.440314 (+ 29862us) log.cc:844] Serialized 26 byte log entry
0709 12:28:56.778515 (+338201us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:56.778553 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.778625 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.794362 (+ 15737us) raft_consensus.cc:1685] finished
0709 12:28:56.794392 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.794585 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":261246,"threads_started":1,"wal-append.queue_time_us":303450}
I20260709 12:28:56.800379  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:56.774104 15892 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:56.776451 15607 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:56.813207  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:56.814082  7660 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:56.815891  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82465.4us
I20260709 12:28:56.812222  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1197) took 1209 ms. Trace:
I20260709 12:28:56.820523  8415 rpcz_store.cc:276] 0709 12:28:55.603009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.781177 (+1178168us) service_pool.cc:224] Handling call
0709 12:28:56.812141 (+ 30964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:56.783115 15644 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.814350  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:56.813668  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1663) took 2881 ms. Trace:
W20260709 12:28:56.821599  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.513s (SENT)
I20260709 12:28:56.800482 15866 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:56.832918  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:56.799393 15891 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:56.805460 15902 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:56.811820 15607 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.829617  7733 rpcz_store.cc:276] 0709 12:28:53.932055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.297172 (+365117us) service_pool.cc:224] Handling call
0709 12:28:54.297564 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.298106 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:54.298135 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:54.308505 (+ 10370us) log.cc:844] Serialized 26 byte log entry
0709 12:28:55.270178 (+961673us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b48002001d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:55.321423 (+ 51245us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b18004a3c20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:28:55.361949 (+ 40526us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:55.362016 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.362125 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.813437 (+1451312us) raft_consensus.cc:1685] finished
0709 12:28:56.813461 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.813597 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20198,"raft.queue_time_us":1708,"raft.run_cpu_time_us":13079,"raft.run_wall_time_us":966849,"spinlock_wait_cycles":15336960,"thread_start_us":40247,"threads_started":1,"wal-append.queue_time_us":1907127}
W20260709 12:28:56.826573 15898 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.444s	user 0.015s	sys 0.002s
I20260709 12:28:56.813127 15242 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:56.861366  7661 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:56.806001 15490 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
W20260709 12:28:56.863895  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 96746.7us
I20260709 12:28:56.831378 15644 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:56.807538 15918 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:28:56.799978 15905 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:28:56.870515  7655 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:56.856371 15882 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.609s	user 0.014s	sys 0.000s
I20260709 12:28:56.892341  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:56.893147  7702 leader_election.cc:400] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
I20260709 12:28:56.893607  7702 leader_election.cc:403] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:28:56.853634 15607 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:56.900688 15925 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:28:56.901593 15928 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:28:56.903429 15606 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:56.904740 15887 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:56.908812 15882 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.910326  8405 raft_consensus.cc:2368] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 20.
I20260709 12:28:56.855118 15867 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:56.915154  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93613.3us
W20260709 12:28:56.917455  7627 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.668s (SENT)
W20260709 12:28:56.918766  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.687s (SENT)
I20260709 12:28:56.923513  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:56.856196 15866 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:56.877280 15876 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:56.925946  7658 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:56.930197  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.880784 15872 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:56.935986  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.710s (SENT)
I20260709 12:28:56.936672  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:56.938733  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3300) took 1464 ms. Trace:
I20260709 12:28:56.934464 15584 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:56.937544 15802 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:56.939471  7653 rpcz_store.cc:276] 0709 12:28:55.474450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.718792 (+1244342us) service_pool.cc:224] Handling call
0709 12:28:56.747433 (+ 28641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.747751 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:56.764023 (+ 16272us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.902749 (+138726us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b1800404140. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:28:56.938004 (+ 35255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.938222 (+   218us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:56.938318 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.938417 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.938647 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11683,"prepare.queue_time_us":16229,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":57115,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":63,"spinlock_wait_cycles":10227072,"thread_start_us":155775,"threads_started":2}
I20260709 12:28:56.864718 15898 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.947156  7657 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:56.972559  7805 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:56.945928 15892 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:56.982322  7812 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:28:56.929325 15918 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17.
W20260709 12:28:56.991950  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 189603us
I20260709 12:28:56.939600 15742 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:56.946421 15867 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.957247 15938 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.961170 15613 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:28:56.976729 15927 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:28:56.968122 15588 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.004308 15657 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:57.004933 15657 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:56.994980 15873 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.007630  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:57.010929 15935 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:28:57.001155  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 142973us
I20260709 12:28:57.012735  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3299) took 1538 ms. Trace:
I20260709 12:28:57.013741  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:57.013207  7654 rpcz_store.cc:276] 0709 12:28:55.474036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.697601 (+1223565us) service_pool.cc:224] Handling call
0709 12:28:56.730266 (+ 32665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.763577 (+ 33311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.763614 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.776573 (+ 12959us) log.cc:844] Serialized 26 byte log entry
0709 12:28:56.931964 (+155391us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:56.932020 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.932104 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.012473 (+ 80369us) raft_consensus.cc:1685] finished
0709 12:28:57.012498 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.012657 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12174,"raft.queue_time_us":22070,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":17075,"thread_start_us":120570,"threads_started":1,"wal-append.queue_time_us":176546}
I20260709 12:28:56.974663 15902 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:56.974035 15917 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.018471  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:57.020602  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84659.2us
W20260709 12:28:57.037829  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.712s (SENT)
I20260709 12:28:57.038365  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:57.022965 15242 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:57.019876 15900 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:28:57.023617  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130633us
I20260709 12:28:56.982924 15858 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.013556 15597 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.032012 15657 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.036366  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:57.052289  7658 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:57.020704  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:57.040515  7660 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:28:57.053954  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:28:57.055510 15941 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:28:57.055747  7702 leader_election.cc:400] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:28:57.056068  7702 leader_election.cc:403] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:57.057333 15873 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:57.044191  7662 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:57.059160  7660 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:57.053730  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:57.059443  7657 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:28:57.061264  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.065078  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46698.1us
I20260709 12:28:57.003126 15938 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.068508  7734 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:57.069711  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.070037  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.072322  7728 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:57.072337  7732 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:57.074409  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.074932  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.075940  7735 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:28:57.087929  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.088891  7730 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:57.015537 15867 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.070991 15917 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:57.058522 15581 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.074265 15242 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:28:57.170449  7728 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:28:57.198799  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.208921 15898 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.171372 15775 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:57.211334  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 173541us
W20260709 12:28:57.213030  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.214272  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.219691  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.221800  7733 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:28:57.170017 15902 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.225498  7735 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:28:57.226352  7658 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:57.225318  7733 pending_rounds.cc:85] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:28:57.227152  7733 pending_rounds.cc:107] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:28:57.227540  7733 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:57.229125  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.230312  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 174754us
I20260709 12:28:57.131744 15917 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:57.242857 15917 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:57.247038 15931 raft_consensus.cc:993] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:28:57.250392  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:57.251530 15938 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:57.251638  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3582) took 1063 ms. Trace:
I20260709 12:28:57.252395  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:57.253048  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:57.254346  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:57.256063  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:57.052078 15866 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.262141  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:57.263584  7730 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:28:57.264842  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:57.233342  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.509s (SENT)
I20260709 12:28:57.234004  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.236599 15919 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:57.237277 15613 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:57.238935 15775 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:57.256343 15770 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.003s
I20260709 12:28:57.241468 15900 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:57.232686  7777 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:57.252142  7805 rpcz_store.cc:276] 0709 12:28:56.188642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.188822 (+   180us) service_pool.cc:224] Handling call
0709 12:28:57.251240 (+1062418us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54003c46d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:57.251428 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794421632}
I20260709 12:28:57.266520  7731 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:28:57.266983 15949 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.267268  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.319650 15949 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.320369  7728 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:57.320816 15607 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.322109 15949 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.266983  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:57.323424  8413 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:57.323490 15607 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.241497 15936 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:57.324846 15936 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:57.252218 15948 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.327785 15948 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.297359 15919 consensus_queue.cc:1243] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:28:57.331895 15954 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:28:57.335112  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.274950  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:57.277199  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43881.5us
I20260709 12:28:57.240694  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:57.339636  7627 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.572s (SENT)
I20260709 12:28:57.304010 15931 raft_consensus.cc:1081] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:28:57.340143  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:57.304165 15942 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:28:57.341928  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 101324us
I20260709 12:28:57.344218 15613 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:57.344928 15613 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:57.346339 15882 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.345979 15613 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:57.350716 15750 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.003s
I20260709 12:28:57.280004 15597 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:28:57.351688 15597 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.278718 15938 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:57.354445 15879 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.354874 15938 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.355439 15938 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:57.356176  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.306147  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1682) took 1056 ms. Trace:
I20260709 12:28:57.357617  7737 rpcz_store.cc:276] 0709 12:28:56.250012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.255360 (+  5348us) service_pool.cc:224] Handling call
0709 12:28:56.273066 (+ 17706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.508586 (+235520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.305866 (+797280us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400248bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:57.306071 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485336960}
I20260709 12:28:57.359193  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.360656  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:57.362689  7731 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:28:57.308204 15581 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.309909 15644 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:57.310541  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77957.9us
I20260709 12:28:57.311568 15945 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:57.311949 15770 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.312788 15491 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.313185 15900 consensus_queue.cc:1243] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:57.314208  7805 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:57.314072  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:57.315155  8379 connection.cc:256] RPC call timeout handler was delayed by -1.8066077569997105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 12:28:57.397684  8379 connection.cc:256] RPC call timeout handler was delayed by -1.6869077569999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 12:28:57.398258  8379 connection.cc:256] RPC call timeout handler was delayed by -1.5744077569997899s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 12:28:57.398710  8379 connection.cc:256] RPC call timeout handler was delayed by -1.4097077569995236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 12:28:57.404836 15490 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:57.321890  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.412629  7728 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:57.268456  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:57.414956 15882 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:57.415712 15882 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:28:57.416281  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3292) took 1965 ms (client timeout 1600 ms). Trace:
W20260709 12:28:57.416920  7659 rpcz_store.cc:269] 0709 12:28:55.450639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.457521 (+  6882us) service_pool.cc:224] Handling call
0709 12:28:57.268268 (+1810747us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:28:57.416122 (+147854us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36786432}
I20260709 12:28:57.406107 15905 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:57.407015  8414 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:57.315944  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:28:57.422940  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107088us
I20260709 12:28:57.326344 15936 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.424499  8380 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:57.424896 15936 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:57.425596 15887 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:57.426033 15887 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.426432 15887 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:57.433302 15858 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:57.438428 15858 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:57.389895 15942 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:57.449399 15942 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.451820 15917 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.436486 15905 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:57.437022 15892 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:57.390970 15750 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.456687 15750 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.457497 15644 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:57.457906 15905 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:57.443638 15806 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:57.444573 15931 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.444694 15945 raft_consensus.cc:697] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:57.459301 15931 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.459903 15945 consensus_queue.cc:237] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.409615  8379 leader_election.cc:400] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19.
I20260709 12:28:57.442028  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:57.775862  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 366481us
W20260709 12:28:57.777489  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.617405 15882 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.779444 15882 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:57.682293 15938 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:57.461123 15905 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.784693 15938 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:57.451954 15858 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:57.686110 15887 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:57.339948 15948 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.776180  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:57.552474  7728 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:28:57.788795  7735 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:57.789175 15867 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:57.790836 15770 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:57.791407  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:57.507110  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.815929  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.793756  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:28:57.785926  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 446182us
I20260709 12:28:57.813989  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:57.821066  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.762s (SENT)
I20260709 12:28:57.821902  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:57.822178  7729 raft_consensus.cc:2368] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23.
W20260709 12:28:57.817646  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.792157  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3577) took 2498 ms. Trace:
I20260709 12:28:57.794168  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:57.823081  7806 rpcz_store.cc:276] 0709 12:28:55.293442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.293557 (+   115us) service_pool.cc:224] Handling call
0709 12:28:55.304530 (+ 10973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:55.509238 (+204708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.509288 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.156164 (+646876us) log.cc:844] Serialized 26 byte log entry
0709 12:28:56.621945 (+465781us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:56.621993 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.791515 (+1169522us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000686d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:57.791585 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.791677 (+    92us) raft_consensus.cc:1685] finished
0709 12:28:57.791697 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.792062 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":572992,"raft.queue_time_us":33418,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":32638,"spinlock_wait_cycles":356858112,"thread_start_us":349983,"threads_started":1,"wal-append.queue_time_us":561496}
W20260709 12:28:57.836719  7627 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.765s (SENT)
I20260709 12:28:57.837436  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:57.846295  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:57.850170  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.850844  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.850911  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:57.851316  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.887068  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:57.910387  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:57.911571  7806 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:28:57.915870  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:57.920634  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 127001us
I20260709 12:28:57.921943  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:57.917932  7737 raft_consensus.cc:2368] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:57.793336 15887 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:57.928419  8415 raft_consensus.cc:2410] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:57.929541  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:57.817546 15693 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.797572 15597 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:28:57.942301  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.045543  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:58.038925  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 217101us
I20260709 12:28:58.048281  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:28:58.040174  7730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.035s	sys 0.029s
W20260709 12:28:58.036618  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 229106us
I20260709 12:28:57.832657 15938 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:57.793339 15902 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.134s
W20260709 12:28:58.066504  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 229827us
W20260709 12:28:58.067389  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.829s (SENT)
I20260709 12:28:58.067816  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:28:58.051437  7730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.036s	sys 0.029s
I20260709 12:28:58.069775  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.070621  7737 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:58.058524  7806 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:28:58.073403  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1313) took 1470 ms. Trace:
I20260709 12:28:58.074004  7806 rpcz_store.cc:276] 0709 12:28:56.602946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.928160 (+1325214us) service_pool.cc:224] Handling call
0709 12:28:58.058201 (+130041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.072868 (+ 14667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.073301 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:58.077204  8407 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:28:58.059643  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 168519us
W20260709 12:28:58.060875  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 139079us
I20260709 12:28:58.080753  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:28:57.937887 15803 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:58.115389  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.118286  7732 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:58.135730  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:28:58.138844  7806 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:58.139846  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3589) took 1500 ms. Trace:
I20260709 12:28:58.140300  7806 rpcz_store.cc:276] 0709 12:28:56.639189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.083469 (+1444280us) service_pool.cc:224] Handling call
0709 12:28:58.138546 (+ 55077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:58.139513 (+   967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.139775 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:57.786158 15858 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.069176 15947 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:58.159185  7730 raft_consensus.cc:1240] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 20. Ops: []
I20260709 12:28:58.057503 15938 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:58.105322 15936 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.680s	user 0.015s	sys 0.009s
I20260709 12:28:58.084684 15887 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:58.084292 15750 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:58.124210 15644 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:58.169917 15808 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:28:58.171680 15958 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:28:58.175768  7733 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:58.162242 15905 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:58.183089 15778 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:58.076776 15693 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:58.184775  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1196) took 2657 ms (client timeout 1700 ms). Trace:
W20260709 12:28:58.185297  8405 rpcz_store.cc:269] 0709 12:28:55.527259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.527476 (+   217us) service_pool.cc:224] Handling call
0709 12:28:58.184522 (+2657046us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54005b04d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:58.184697 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795974400}
I20260709 12:28:58.185948 15693 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:58.186869  8409 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:28:58.187832  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:28:58.189240 15900 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:58.137045 15742 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:58.231417  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:58.283300  7806 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:58.286038  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.288703  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3688) took 1016 ms. Trace:
I20260709 12:28:58.295804  7730 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:58.164307 15887 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:28:58.232421 15956 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
W20260709 12:28:58.232882 15964 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 12:28:58.199517 15938 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:58.307003  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.302150  7731 rpcz_store.cc:276] 0709 12:28:57.272250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.363818 (+ 91568us) service_pool.cc:224] Handling call
0709 12:28:57.459968 (+ 96150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.460236 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:57.789474 (+329238us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.288132 (+498658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.288206 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:58.288310 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.288419 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.288606 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328756,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":126426,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":13462,"thread_start_us":383681,"threads_started":2}
I20260709 12:28:58.319921  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:58.324851  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 189193us
I20260709 12:28:58.278491 15770 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:58.331697  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.305812 15887 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:58.341168  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.342011  7730 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:58.346948  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:58.350910  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:28:58.352283  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.370390  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.377524  7806 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:28:58.347728 15742 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.344907 15931 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:58.370748 15750 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:28:58.369517  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.385839  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.401086  7731 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:58.408735  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:58.409804  7803 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:58.416034  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 367902us
I20260709 12:28:58.419809  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.421429  7731 raft_consensus.cc:2368] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
W20260709 12:28:58.422784  8411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1445) took 1951 ms (client timeout 2196 ms). Trace:
I20260709 12:28:58.349781 15770 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:58.431493  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 249851us
W20260709 12:28:58.440382  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.448860  8411 rpcz_store.cc:269] 0709 12:28:56.471503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.473772 (+  2269us) service_pool.cc:224] Handling call
0709 12:28:58.422562 (+1948790us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400320ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:58.422711 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497568000}
W20260709 12:28:58.451793  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.452710 15905 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:58.454195 15750 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:58.454934 15750 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.457736 15750 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:58.459671  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:58.462329  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3691) took 1144 ms. Trace:
I20260709 12:28:58.382674 15742 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:58.463321 14906 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:58.462798  7729 rpcz_store.cc:276] 0709 12:28:57.317792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.937948 (+620156us) service_pool.cc:224] Handling call
0709 12:28:57.938493 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.938858 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.938898 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.037340 (+ 98442us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.337738 (+300398us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:58.337793 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.337905 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.462028 (+124123us) raft_consensus.cc:1685] finished
0709 12:28:58.462050 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.462259 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":179517,"threads_started":1,"wal-append.queue_time_us":351268}
I20260709 12:28:58.463991 14906 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:28:58.464874  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.465071 14906 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.465740 15742 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:58.466578 14906 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:58.467186  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:58.469682 15963 log.cc:927] Time spent T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 12:28:58.470335 15965 log.cc:927] Time spent T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:28:58.418736 15597 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:28:58.471269 15597 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.474018 15597 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:58.477566 15858 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:58.478847  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3314) took 1480 ms. Trace:
I20260709 12:28:58.479377  7653 rpcz_store.cc:276] 0709 12:28:56.998125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.021272 (+ 23147us) service_pool.cc:224] Handling call
0709 12:28:57.052522 (+ 31250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.073564 (+ 21042us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:57.562655 (+489091us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.063718 (+501063us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.119937 (+ 56219us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.120184 (+   247us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:58.120257 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.120359 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.478586 (+358227us) raft_consensus.cc:1685] finished
0709 12:28:58.478612 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.478776 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163694,"prepare.queue_time_us":49924,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":148531,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":21812,"thread_start_us":777138,"threads_started":2,"wal-append.queue_time_us":540937}
W20260709 12:28:58.481426  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.482162  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.482627  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.482969 15962 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:28:58.483330 15974 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:58.419181 15489 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:58.484362 15489 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:28:58.483982 15974 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.486518  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1304) took 1999 ms. Trace:
W20260709 12:28:58.488765  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.468987  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.487043  7810 rpcz_store.cc:276] 0709 12:28:56.486829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.502237 (+ 15408us) service_pool.cc:224] Handling call
0709 12:28:56.522523 (+ 20286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.759100 (+236577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.486233 (+1727133us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:58.486449 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480659840}
I20260709 12:28:58.490198  7729 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:58.486740  7732 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:58.486375  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:28:58.494845  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77839us
I20260709 12:28:58.496592  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:28:58.530794  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:28:58.532613  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:58.533813  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 453170us
I20260709 12:28:58.533632  7729 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:28:58.549412  8413 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:58.549882  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:58.554069  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:58.554585 15966 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 12:28:58.556392 15960 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:28:58.524184 15742 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:58.563592  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:58.563316 15742 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:58.565218  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:28:58.565889  7810 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:58.527626 15882 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.006s
I20260709 12:28:58.553020  7777 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:58.477622 15770 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:58.579293  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.587322  7730 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
W20260709 12:28:58.599787  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73330.4us
I20260709 12:28:58.588320  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3664) took 2101 ms. Trace:
I20260709 12:28:58.551602 15750 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:58.640288  7803 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:58.580055  7729 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
W20260709 12:28:58.644551  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.020s	sys 0.017s
W20260709 12:28:58.664319  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.020s	sys 0.018s
I20260709 12:28:58.646366  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.651397  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:58.665983  7731 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
W20260709 12:28:58.652951  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.661003  8380 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:28:58.576458 15597 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:58.660944  7736 rpcz_store.cc:276] 0709 12:28:56.486641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.491214 (+  4573us) service_pool.cc:224] Handling call
0709 12:28:56.491556 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.732517 (+240961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.732549 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.732591 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:56.732630 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.587890 (+1855260us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001c52d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:58.587971 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.588224 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553610496}
W20260709 12:28:58.689743  7777 connection.cc:256] RPC call timeout handler was delayed by -1.3872689340000761s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 12:28:58.672109  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:58.689407  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:58.671322  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.694880  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.691679  7655 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:58.673321  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 605925us
I20260709 12:28:58.698474  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.699451  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:58.699879  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.701756  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:58.702960  7654 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:58.707255  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:58.708711  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:58.605573 15932 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:58.701774  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.760344  7729 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:58.761407  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:58.762566  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:58.688867 15803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.014s	sys 0.010s
I20260709 12:28:58.690230 15882 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:58.835537  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:58.772010  7657 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:58.667186 15750 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.013355  7730 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:59.036603 15750 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:59.037362  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:59.037650  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.039340  7657 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:28:58.781111  7655 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:28:58.829108 15863 raft_consensus.cc:2764] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:28:59.037362  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.052415  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.049350  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.046353  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.045323  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.054487  7730 raft_consensus.cc:2410] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:28:59.052634  7660 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:28:59.055083  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:58.772989  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:28:58.784189 15742 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:59.064412  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3594) took 1757 ms. Trace:
I20260709 12:28:59.012473  7731 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
W20260709 12:28:58.884850  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.065029  7805 rpcz_store.cc:276] 0709 12:28:57.306755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.063000 (+1756245us) service_pool.cc:224] Handling call
0709 12:28:59.063562 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.063869 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:59.063912 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:59.063985 (+    73us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:59.064052 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.064139 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.064312 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:59.066092  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:59.066738  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:59.067088  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.844218  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.934424  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 235790us
W20260709 12:28:59.068584  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.908s (SENT)
I20260709 12:28:59.069067  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:58.773617  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.013691  7736 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:28:59.070353  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.070797  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:59.071264  7728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3468) took 1635 ms (client timeout 2162 ms). Trace:
I20260709 12:28:59.020393  7654 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:28:58.976239 15972 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:28:59.026633 15944 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:58.912555  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.089952  7736 raft_consensus.cc:2368] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20.
I20260709 12:28:58.690443 15490 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.085s
I20260709 12:28:58.844350  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:59.040715 15971 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.103878 15971 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.106637 15971 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:59.039716 15803 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:59.044803  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1311) took 2466 ms. Trace:
I20260709 12:28:59.045903 15882 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:59.108348 15803 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.108647 15882 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.109637 15731 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:28:58.776216  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:28:59.113366  7658 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
W20260709 12:28:59.071676  7728 rpcz_store.cc:269] 0709 12:28:57.435728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.070064 (+1634336us) service_pool.cc:224] Handling call
0709 12:28:59.071169 (+  1105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:59.054410 15597 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:28:59.116317  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.116648 15597 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:59.117679  7728 raft_consensus.cc:2410] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:59.052659 15852 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:28:59.079360  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.084981  7655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:28:59.118719  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.119161  7655 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.119668  7655 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:59.092854  7627 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.998s (SENT)
W20260709 12:28:59.073834  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50722.3us
I20260709 12:28:59.100347  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1309) took 2542 ms. Trace:
I20260709 12:28:59.120289  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:28:59.076646  7805 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:59.120569  7804 rpcz_store.cc:276] 0709 12:28:56.557865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.577863 (+ 19998us) service_pool.cc:224] Handling call
0709 12:28:56.586960 (+  9097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.683869 (+ 96909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.100023 (+2416154us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54003e9ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:28:59.100270 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66559360}
I20260709 12:28:59.123875  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.124995  7736 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:28:59.126328  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.118721  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:59.130550  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3597) took 1317 ms. Trace:
I20260709 12:28:59.131322  7805 rpcz_store.cc:276] 0709 12:28:57.813276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.066420 (+1253144us) service_pool.cc:224] Handling call
0709 12:28:59.066814 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.121544 (+ 54730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.130479 (+  8935us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:59.086529 15980 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.250s	user 0.011s	sys 0.014s
I20260709 12:28:59.080519 15974 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:59.107723 15750 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.132117  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.131616  7654 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:59.204257  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.456606  7731 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 24 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:28:59.457196  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3483) took 1278 ms. Trace:
I20260709 12:28:59.457576  7731 rpcz_store.cc:276] 0709 12:28:58.178525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.075845 (+897320us) service_pool.cc:224] Handling call
0709 12:28:59.457130 (+381285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:59.153992  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:28:59.263080  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.136459  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:59.172448  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.475261  7730 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:28:59.476334  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 340118us
I20260709 12:28:59.327378  7809 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:59.144999  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 300790us
I20260709 12:28:59.477387  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:59.197768  7810 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.002s
I20260709 12:28:59.478448  7810 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:28:59.479269  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3602) took 1345 ms. Trace:
W20260709 12:28:59.447115  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.008s	sys 0.012s
I20260709 12:28:59.479961  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:28:59.480603  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.009s	sys 0.013s
I20260709 12:28:59.479780  7810 rpcz_store.cc:276] 0709 12:28:58.133662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.564943 (+431281us) service_pool.cc:224] Handling call
0709 12:28:59.479147 (+914204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:59.445087  7803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.012s
W20260709 12:28:59.483608  7803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.005s	sys 0.013s
W20260709 12:28:59.380898  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72325us
W20260709 12:28:59.446853  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 332996us
W20260709 12:28:59.487874  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.072222 15932 raft_consensus.cc:697] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:28:59.493113  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 400292us
I20260709 12:28:59.481035  7655 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:59.495141  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3598) took 1674 ms. Trace:
W20260709 12:28:59.121868 15982 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 12:28:59.532183  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.533438  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3476) took 1321 ms. Trace:
I20260709 12:28:59.534042  7732 rpcz_store.cc:276] 0709 12:28:58.176473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.369202 (+192729us) service_pool.cc:224] Handling call
0709 12:28:59.497243 (+1128041us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b5400072bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:28:59.497473 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209599360}
I20260709 12:28:59.535626  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:59.334139 15983 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.019s	sys 0.021s
I20260709 12:28:59.505030  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3599) took 1419 ms. Trace:
I20260709 12:28:59.543538  7809 rpcz_store.cc:276] 0709 12:28:58.085590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.203999 (+1118409us) service_pool.cc:224] Handling call
0709 12:28:59.326957 (+122958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.504660 (+177703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.504956 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:59.507059  7627 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.068s (SENT)
I20260709 12:28:59.200716  7736 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:28:59.511607  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:28:59.433207 15959 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.038s	user 0.005s	sys 0.000s
W20260709 12:28:59.525524  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48263.4us
I20260709 12:28:59.525724  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.569221  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:28:59.212561 15985 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:28:59.526748  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.571732  7728 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:28:59.116451 15490 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:59.498867  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:59.100188 15742 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:59.538760  7662 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:28:59.542122  7808 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.007s	sys 0.031s
W20260709 12:28:59.585498  7808 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.008s	sys 0.031s
I20260709 12:28:59.586470  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3588) took 2956 ms. Trace:
I20260709 12:28:59.487836  7729 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
W20260709 12:28:59.542708  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52096.9us
I20260709 12:28:59.542059  7732 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 17: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:28:59.545974  7810 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:28:59.257032 15882 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.538817 15916 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:59.494382 15971 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.547704  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.599371  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:59.599978  7730 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:28:59.549430 15750 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:59.147454 15863 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:59.550940 14906 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.084s	user 0.010s	sys 0.005s
I20260709 12:28:59.561009 15932 consensus_queue.cc:237] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:28:59.543330 15818 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:59.131814 15500 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:28:59.447717 15899 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:28:59.566066  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:59.187361 15891 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:59.569793  7731 raft_consensus.cc:2368] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:28:59.504374 15900 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:59.575430  7655 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:28:59.576399  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3462) took 2232 ms. Trace:
I20260709 12:28:59.583325  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:28:59.586983  7808 rpcz_store.cc:276] 0709 12:28:56.629838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.933713 (+1303875us) service_pool.cc:224] Handling call
0709 12:28:57.934217 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:58.083294 (+149077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.083351 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:58.125307 (+ 41956us) log.cc:844] Serialized 52 byte log entry
0709 12:28:58.606147 (+480840us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:58.813414 (+207267us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.585987 (+772573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.586101 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.586166 (+    65us) raft_consensus.cc:1685] finished
0709 12:28:59.586186 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.586393 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":197332,"prepare.queue_time_us":61596,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":5603,"raft.queue_time_us":379856,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":52950,"thread_start_us":1304634,"threads_started":3,"wal-append.queue_time_us":650784}
I20260709 12:28:59.600071  7728 raft_consensus.cc:2376] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17. Re-sending same reply.
I20260709 12:28:59.596468  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:59.602067  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:28:59.602142  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:28:59.605338  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 127509us
I20260709 12:28:59.606765 15882 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.607776  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:28:59.608152  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:28:59.122108 15944 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:28:59.582755 15858 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.105s	user 0.015s	sys 0.000s
W20260709 12:28:59.684839  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 67868.5us
W20260709 12:29:00.055008  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 458600us
W20260709 12:29:00.058152  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.027035 15489 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.035756  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:00.037277  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.038422  7729 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 12:28:59.843493  7735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.006s	sys 0.019s
W20260709 12:29:00.068550  7735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.012s	sys 0.020s
I20260709 12:29:00.039096 15500 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:28:59.656926  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.040473  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3509) took 1008 ms. Trace:
I20260709 12:29:00.075421  7731 rpcz_store.cc:276] 0709 12:28:59.032589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.458366 (+425777us) service_pool.cc:224] Handling call
0709 12:29:00.040380 (+582014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.041023 15899 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:28:59.584128 15490 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:28:59.610904 15983 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.437s
I20260709 12:29:00.047962  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45918 (request call id 3327) took 1379 ms. Trace:
I20260709 12:28:59.783913  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3603) took 1444 ms. Trace:
I20260709 12:29:00.049201 15992 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:29:00.049808  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 447795us
I20260709 12:28:59.614833  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:00.051841  7732 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:28:59.617162  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:00.029477  7737 rpcz_store.cc:276] 0709 12:28:57.343749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.089657 (+745908us) service_pool.cc:224] Handling call
0709 12:28:58.148019 (+ 58362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.160883 (+ 12864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.160923 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.242204 (+ 81281us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.671732 (+429528us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:58.671770 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.671845 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.576017 (+904172us) raft_consensus.cc:1685] finished
0709 12:28:59.576048 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.576295 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":264190,"threads_started":1,"wal-append.queue_time_us":596290}
I20260709 12:28:59.761117  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1303) took 3274 ms. Trace:
I20260709 12:29:00.073529  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:00.067685  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:28:59.993420  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.102011  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 594942us
I20260709 12:29:00.122558  7728 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 12:29:00.129101  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.280s (SENT)
I20260709 12:29:00.129333  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3464) took 2785 ms. Trace:
I20260709 12:29:00.130225  7728 rpcz_store.cc:276] 0709 12:28:57.344350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.068249 (+2723899us) service_pool.cc:224] Handling call
0709 12:29:00.122186 (+ 53937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:00.128844 (+  6658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.129226 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.131343  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3516) took 1096 ms. Trace:
I20260709 12:29:00.124430  7806 raft_consensus.cc:2376] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26. Re-sending same reply.
W20260709 12:29:00.126645  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53213us
W20260709 12:28:59.681447  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.104014  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:00.103466  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3493) took 1545 ms. Trace:
W20260709 12:28:59.651835  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.028765 15900 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:00.112558  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3497) took 1539 ms. Trace:
I20260709 12:29:00.127051  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3522) took 1013 ms. Trace:
I20260709 12:29:00.134590  7730 rpcz_store.cc:276] 0709 12:28:58.558271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.600893 (+1042622us) service_pool.cc:224] Handling call
0709 12:29:00.103389 (+502496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.135087  7729 rpcz_store.cc:276] 0709 12:28:58.573364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.600811 (+1027447us) service_pool.cc:224] Handling call
0709 12:29:00.112430 (+511619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.135141  7736 rpcz_store.cc:276] 0709 12:28:59.113566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.036963 (+923397us) service_pool.cc:224] Handling call
0709 12:29:00.126925 (+ 89962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.140563  7731 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 12:29:00.114751  7655 rpcz_store.cc:276] 0709 12:28:58.710949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.066345 (+355396us) service_pool.cc:224] Handling call
0709 12:29:00.047801 (+981456us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":575}
W20260709 12:28:59.937119 15887 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.584s	user 0.015s	sys 0.001s
I20260709 12:29:00.123991  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:00.141754  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3465) took 2797 ms. Trace:
I20260709 12:29:00.132727  7732 rpcz_store.cc:276] 0709 12:28:59.056041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.601742 (+545701us) service_pool.cc:224] Handling call
0709 12:29:00.131167 (+529425us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.113674  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47807.5us
I20260709 12:29:00.142362  7731 rpcz_store.cc:276] 0709 12:28:57.344655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.076377 (+2731722us) service_pool.cc:224] Handling call
0709 12:29:00.130196 (+ 53819us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:00.141303 (+ 11107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.141666 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.143440  7778 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.833s (SENT) [suppressed 30 similar messages]
I20260709 12:28:59.600024 16005 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:00.145845  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.103343  7700 connection.cc:256] RPC call timeout handler was delayed by -2.2516078120002021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2821s for a response
I20260709 12:29:00.067559 15778 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:00.115919  7805 rpcz_store.cc:276] 0709 12:28:58.339664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.461222 (+1121558us) service_pool.cc:224] Handling call
0709 12:28:59.484690 (+ 23468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.485052 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:59.613532 (+128480us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.782967 (+169435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:59.783083 (+   116us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:59.783189 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.783330 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.783734 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59389,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":64,"raft.queue_time_us":1660,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":74497,"thread_start_us":243725,"threads_started":2}
I20260709 12:29:00.135759  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:00.156622  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1699) took 1924 ms. Trace:
I20260709 12:29:00.157493  7729 rpcz_store.cc:276] 0709 12:28:58.232126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.155257 (+1923131us) service_pool.cc:224] Handling call
0709 12:29:00.155648 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.155935 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.155974 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.156045 (+    71us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:00.156105 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.156206 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.156382 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.160338  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:00.161376  7806 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:29:00.161856  7806 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:00.164326  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1702) took 1931 ms. Trace:
I20260709 12:29:00.165156  7732 rpcz_store.cc:276] 0709 12:28:58.233275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.163156 (+1929881us) service_pool.cc:224] Handling call
0709 12:29:00.163613 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.163870 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.163908 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.163970 (+    62us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:00.164029 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.164101 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.164248 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.067296 16002 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:00.173228  7736 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:29:00.174618  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3482) took 1996 ms. Trace:
I20260709 12:29:00.175144  7736 rpcz_store.cc:276] 0709 12:28:58.178254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.154249 (+1975995us) service_pool.cc:224] Handling call
0709 12:29:00.154836 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:00.174089 (+ 19253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.174521 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:59.613293 15993 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:00.130882 15770 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.022s
I20260709 12:29:00.110438 15981 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:00.199208  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1701) took 1966 ms. Trace:
W20260709 12:29:00.139750 15938 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.811s	user 0.012s	sys 0.005s
I20260709 12:29:00.184871  7728 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:29:00.199931  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.017s	sys 0.046s
I20260709 12:29:00.201666  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3467) took 2856 ms. Trace:
I20260709 12:29:00.137476 15863 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:00.193714  7737 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 15. Current term is 17. Ops: []
I20260709 12:29:00.120045 15489 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:00.182816  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:00.202064  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.017s	sys 0.047s
I20260709 12:29:00.125309  7811 rpcz_store.cc:276] 0709 12:28:56.486517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.489821 (+  3304us) service_pool.cc:224] Handling call
0709 12:28:56.490222 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.490714 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:56.764517 (+273803us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.983599 (+219082us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.983795 (+   196us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:57.236897 (+253102us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4800091158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:58.146745 (+909848us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b4800091158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e42b37 00007fc006e429ca 00007fc006f3a26f 00007fc006f39f37 00007fc006f4e55c 00007fc006f4e508 00007fc006f4e488 00007fc006f4e443 00007fc006f4d6ec 00007fc005e77a96
0709 12:28:59.115778 (+969033us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800450b20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a
0709 12:28:59.116899 (+  1121us) spinlock_profiling.cc:243] Waited 9.78 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:59.117502 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:28:59.445630 (+328128us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:59.446066 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.446364 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.446606 (+   242us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.760696 (+314090us) raft_consensus.cc:1685] finished
0709 12:28:59.760724 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.761002 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126102,"prepare.queue_time_us":93646,"prepare.run_cpu_time_us":191,"prepare.run_wall_time_us":31665,"raft.queue_time_us":94131,"raft.run_cpu_time_us":8354,"raft.run_wall_time_us":2023900,"spinlock_wait_cycles":63844224,"thread_start_us":740880,"threads_started":4,"wal-append.queue_time_us":579677,"wal-append.run_cpu_time_us":2863,"wal-append.run_wall_time_us":1144406}
I20260709 12:29:00.207371  7810 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19. Re-sending same reply.
I20260709 12:29:00.203666  7736 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:59.869956 15999 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:00.188776  7806 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.223870  7806 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:00.200095  7731 rpcz_store.cc:276] 0709 12:28:58.232912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.162977 (+1930065us) service_pool.cc:224] Handling call
0709 12:29:00.198385 (+ 35408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.198690 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.198730 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.198795 (+    65us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:29:00.198857 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.198944 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.199119 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.202412  7728 rpcz_store.cc:276] 0709 12:28:57.345271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.131575 (+2786304us) service_pool.cc:224] Handling call
0709 12:29:00.184175 (+ 52600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.201202 (+ 17027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.201582 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.207324  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 694668us
I20260709 12:29:00.209753  7730 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
W20260709 12:29:00.213368  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84299.2us
I20260709 12:29:00.208751  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3505) took 1177 ms. Trace:
W20260709 12:29:00.231339  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.191576 15770 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.004s	sys 0.023s
I20260709 12:28:59.612860 16003 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.600s
I20260709 12:29:00.120555 15992 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:00.216125  7732 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:00.261482  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3501) took 1549 ms. Trace:
I20260709 12:29:00.262069  7732 rpcz_store.cc:276] 0709 12:28:58.711684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.215266 (+1503582us) service_pool.cc:224] Handling call
0709 12:29:00.215829 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.261018 (+ 45189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.261380 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.262290  7810 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:28:59.606778 15971 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.207670 15981 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.231380  7736 rpcz_store.cc:276] 0709 12:28:59.030847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.194620 (+1163773us) service_pool.cc:224] Handling call
0709 12:29:00.195409 (+   789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.208440 (+ 13031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.208687 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.239032  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1698) took 2468 ms. Trace:
I20260709 12:29:00.279906  7730 rpcz_store.cc:276] 0709 12:28:57.770559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.150021 (+2379462us) service_pool.cc:224] Handling call
0709 12:29:00.209552 (+ 59531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.238686 (+ 29134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.238961 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.212296 15944 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:00.241585  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7383774040004027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
I20260709 12:29:00.196686 15891 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:00.265340  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
W20260709 12:29:00.285796  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.957s (SENT)
I20260709 12:29:00.285976  7809 raft_consensus.cc:606] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:00.265683  7655 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:00.286976  7655 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:00.213872  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3466) took 2868 ms. Trace:
I20260709 12:29:00.286482  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:00.279256  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1705) took 1148 ms. Trace:
I20260709 12:29:00.262223  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:00.288486  7736 rpcz_store.cc:276] 0709 12:28:59.130420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.277785 (+1147365us) service_pool.cc:224] Handling call
0709 12:29:00.278392 (+   607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.278703 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.278744 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.278813 (+    69us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:00.278876 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.278959 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.279128 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.273243  7806 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:29:00.291946  7806 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:00.287904  7737 rpcz_store.cc:276] 0709 12:28:57.344966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.125654 (+2780688us) service_pool.cc:224] Handling call
0709 12:29:00.181251 (+ 55597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.213405 (+ 32154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.213774 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.282267  7728 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:29:00.294356  7728 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:00.239384  7811 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:00.295965  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3605) took 1801 ms. Trace:
I20260709 12:29:00.296470  7811 rpcz_store.cc:276] 0709 12:28:58.494571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.207562 (+1712991us) service_pool.cc:224] Handling call
0709 12:29:00.239137 (+ 31575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.295546 (+ 56409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.295883 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.284070  7733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3684) took 3057 ms (client timeout 1709 ms). Trace:
I20260709 12:29:00.284241  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1704) took 1147 ms. Trace:
W20260709 12:29:00.297740  7733 rpcz_store.cc:269] 0709 12:28:57.226187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.262298 (+ 36111us) service_pool.cc:224] Handling call
0709 12:29:00.275163 (+3012865us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54002ab554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:00.283942 (+  8779us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1918518528}
I20260709 12:29:00.297966  7732 rpcz_store.cc:276] 0709 12:28:59.120088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.263501 (+1143413us) service_pool.cc:224] Handling call
0709 12:29:00.263953 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.264214 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.264255 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.264335 (+    80us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:00.264400 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.264481 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.267482 (+  3001us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.301414  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 154074us
I20260709 12:29:00.325721  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1703) took 1206 ms. Trace:
I20260709 12:29:00.241503 15597 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:00.327637  7810 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: []
I20260709 12:29:00.200639 16002 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.266683 15992 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:00.179055 15887 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:00.332300  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 172566us
I20260709 12:29:00.329736  7655 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.352219  7655 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:00.334067  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1706) took 1199 ms. Trace:
I20260709 12:29:00.353144  7737 rpcz_store.cc:276] 0709 12:28:59.134810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.294039 (+1159229us) service_pool.cc:224] Handling call
0709 12:29:00.333443 (+ 39404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.333667 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.333698 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.333749 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:00.333814 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.333876 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.333996 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.196871 16005 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.215056 15938 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.340811  7731 rpcz_store.cc:276] 0709 12:28:59.119779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.261001 (+1141222us) service_pool.cc:224] Handling call
0709 12:29:00.325084 (+ 64083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.325320 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.325346 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.325396 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:00.325447 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.325506 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.325652 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.338311 15916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.015s	sys 0.072s
I20260709 12:29:00.331122  7728 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.286708 15981 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:00.346611  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.371501  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.372112  7730 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:00.397080  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 111327us
I20260709 12:29:00.400537  7728 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:29:00.398893  7627 connection.cc:256] RPC call timeout handler was delayed by -2.6051774040001874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260709 12:29:00.402956  7627 connection.cc:256] RPC call timeout handler was delayed by -2.5844774040001539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 12:29:00.404129  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 261608us
W20260709 12:29:00.404347  7627 connection.cc:256] RPC call timeout handler was delayed by -2.5538774040001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
W20260709 12:29:00.404873  7627 connection.cc:256] RPC call timeout handler was delayed by -2.4926774040004602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
I20260709 12:29:00.409854  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.410897  7737 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:29:00.411710  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.412597  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.413101  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.431602  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.433122  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.434767  8414 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:00.438067  7729 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [18.10-19.11]
W20260709 12:29:00.443701  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 261676us
I20260709 12:29:00.453222  7731 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:00.360425 15916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.015s	sys 0.072s
I20260709 12:29:00.236886 15999 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.350315 15852 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:00.468124  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1498) took 1793 ms. Trace:
I20260709 12:29:00.473258  7657 raft_consensus.cc:2393] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 25: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:00.469388  7655 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
W20260709 12:29:00.470552  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.474344  7655 pending_rounds.cc:85] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 14
I20260709 12:29:00.474822  7655 pending_rounds.cc:107] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:29:00.475167  7655 raft_consensus.cc:2889] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:00.476305  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1504) took 1767 ms. Trace:
I20260709 12:29:00.476810  7655 rpcz_store.cc:276] 0709 12:28:58.709225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.227618 (+1518393us) service_pool.cc:224] Handling call
0709 12:29:00.265483 (+ 37865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.475861 (+210378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.476211 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:00.480008  7627 connection.cc:256] RPC call timeout handler was delayed by -1.7310261830002673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 12:29:00.276089 15490 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:00.486747  7627 connection.cc:256] RPC call timeout handler was delayed by -1.5443900810000741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:29:00.490950  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4997261830003481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 12:29:00.491703  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4898261830003321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 12:29:00.492285  7627 connection.cc:256] RPC call timeout handler was delayed by -1.4385261830002491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 12:29:00.492833  7627 connection.cc:256] RPC call timeout handler was delayed by -1.3319900810001855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 12:29:00.493397  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1226261830001931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25470000000000004s for a response
I20260709 12:29:00.493245  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.495762  7730 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:29:00.497704  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.620s (SENT)
I20260709 12:29:00.500782  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:00.506346  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.507426  7737 raft_consensus.cc:2376] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19. Re-sending same reply.
I20260709 12:29:00.472100  7660 rpcz_store.cc:276] 0709 12:28:58.675000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.064847 (+389847us) service_pool.cc:224] Handling call
0709 12:28:59.069398 (+  4551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.069644 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:59.609336 (+539692us) log.cc:844] Serialized 14 byte log entry
0709 12:29:00.467487 (+858151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.467588 (+   101us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:00.467689 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.467823 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.468025 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1083015,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":117392,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":6078,"thread_start_us":1088278,"threads_started":3,"wal-append.queue_time_us":765273}
I20260709 12:29:00.500828  7661 pending_rounds.cc:85] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 13
I20260709 12:29:00.511817  7661 pending_rounds.cc:107] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:29:00.512187  7661 raft_consensus.cc:2889] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:00.518505  7728 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:29:00.519009  7728 pending_rounds.cc:85] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 9
I20260709 12:29:00.520812  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.521999  7731 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:29:00.522069  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:00.324178 15900 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:00.501446 16001 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.456s	user 0.000s	sys 0.002s
I20260709 12:29:00.576327  7728 pending_rounds.cc:107] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:29:00.577216  7728 raft_consensus.cc:2889] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:00.578238  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3514) took 1544 ms. Trace:
I20260709 12:29:00.578624  7728 rpcz_store.cc:276] 0709 12:28:59.034168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.236280 (+1202112us) service_pool.cc:224] Handling call
0709 12:29:00.282076 (+ 45796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.577950 (+295874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.578170 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.579910  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.433893 16009 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:29:00.572008  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.571445  7627 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.958s (SENT)
I20260709 12:29:00.595369  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3332) took 1803 ms. Trace:
I20260709 12:29:00.519761 15852 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.595993  7655 rpcz_store.cc:276] 0709 12:28:58.792072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.569513 (+1777441us) service_pool.cc:224] Handling call
0709 12:29:00.594626 (+ 25113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.594860 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.594888 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.594990 (+   102us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:00.595022 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.595084 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.595291 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.604694 16026 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:00.606259 16026 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:29:00.606662  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3591) took 3300 ms. Trace:
I20260709 12:29:00.607100  7803 rpcz_store.cc:276] 0709 12:28:57.305866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.697019 (+1391153us) service_pool.cc:224] Handling call
0709 12:28:58.699507 (+  2488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.771461 (+ 71954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.771498 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.772184 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 12:28:59.484585 (+712401us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:59.484654 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.484777 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:00.606399 (+1121622us) raft_consensus.cc:1685] finished
0709 12:29:00.606422 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.606593 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68253,"raft.queue_time_us":68747,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":40970,"thread_start_us":654685,"threads_started":1,"wal-append.queue_time_us":967412}
I20260709 12:29:00.608772 16023 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:00.609351 16023 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:00.506083 15858 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.096s
I20260709 12:29:00.553215 15882 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:00.612948 15882 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:00.610839 16023 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.613407 15882 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:00.615446 15891 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:00.617043 15891 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:00.618206 16015 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
W20260709 12:29:00.619494 16014 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 12:29:00.571354 15597 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:00.614537 15882 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.786028  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.761181  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1503) took 2056 ms. Trace:
I20260709 12:29:00.781185  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1695) took 3470 ms. Trace:
W20260709 12:29:00.789183  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.804335  7654 rpcz_store.cc:276] 0709 12:28:58.704882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.564051 (+859169us) service_pool.cc:224] Handling call
0709 12:28:59.593878 (+ 29827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.637819 (+ 43941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:59.637873 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:59.637954 (+    81us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:59.638022 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.760847 (+1122825us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b54000f3554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:00.760901 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.761102 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100127616}
I20260709 12:29:00.805027  7735 rpcz_store.cc:276] 0709 12:28:57.310745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.084858 (+1774113us) service_pool.cc:224] Handling call
0709 12:28:59.085380 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:59.085855 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:59.085890 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:59.086476 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:29:00.069192 (+982716us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:00.069254 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.069364 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:00.780893 (+711529us) raft_consensus.cc:1685] finished
0709 12:29:00.780920 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.781108 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":961,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":13006,"thread_start_us":883649,"threads_started":1,"wal-append.queue_time_us":1016182}
W20260709 12:29:00.845405  7776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.023s	sys 0.007s
W20260709 12:29:00.846776  7776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.024s	sys 0.008s
W20260709 12:29:00.793215  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.992s (SENT)
I20260709 12:29:00.847572  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:00.786324  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.774664 16019 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 12:29:00.857595  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.857963  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.619920 15489 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:00.868204  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.579424 15500 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:00.856699  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 285849us
I20260709 12:29:00.871106  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:00.767622 15971 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:00.803484 16027 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:00.756583 15967 log.cc:927] Time spent T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.361s	user 0.006s	sys 0.013s
I20260709 12:29:00.866228  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:00.621208 15944 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.634829 15863 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.241s
I20260709 12:29:00.753247 15999 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:00.898758  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:00.860580  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:00.882066  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.876655  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.902158  7730 raft_consensus.cc:2376] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22. Re-sending same reply.
I20260709 12:29:00.910251  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.911348  7737 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:29:00.879333 16023 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:00.778048 15981 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:00.897356 15778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.019s	sys 0.004s
I20260709 12:29:00.645284 15852 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:00.880079 15489 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.922277  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:00.943904 16033 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:29:00.891264 16027 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.892577 15500 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:00.916163 15938 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:00.932389  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1691) took 3828 ms. Trace:
W20260709 12:29:00.976646 15778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.020s	sys 0.005s
I20260709 12:29:00.899493 15887 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:00.957535 15818 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.927s	user 0.009s	sys 0.005s
I20260709 12:29:00.967132  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:00.931200 16030 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:00.897264 15932 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:00.968859 15992 consensus_meta.cc:287] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
I20260709 12:29:00.936156 15597 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:00.961668  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1615) took 2743 ms. Trace:
W20260709 12:29:01.065548  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 272458us
W20260709 12:29:01.066452  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.060s (SENT)
I20260709 12:29:01.065968  8414 rpcz_store.cc:276] 0709 12:28:58.218501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.218714 (+   213us) service_pool.cc:224] Handling call
0709 12:28:58.296175 (+ 77461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.453543 (+2157368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.961171 (+507628us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b540060f9d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:00.961509 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4917763456}
I20260709 12:29:01.067142  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:00.964613  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3328) took 2200 ms. Trace:
I20260709 12:29:00.940907  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.930409  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:00.960425  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3336) took 1924 ms. Trace:
I20260709 12:29:01.069841  7658 rpcz_store.cc:276] 0709 12:28:59.014838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.938153 (+1923315us) service_pool.cc:224] Handling call
0709 12:29:00.938573 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.938823 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.938856 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.938910 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:00.938960 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.939020 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.939165 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:59.508719  7804 rpcz_store.cc:276] 0709 12:28:57.820838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.121733 (+1300895us) service_pool.cc:224] Handling call
0709 12:28:59.155581 (+ 33848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.155875 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:59.156208 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.494581 (+338373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:59.494694 (+   113us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:59.494778 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.494876 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.495060 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34098,"prepare.queue_time_us":16644,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":44141,"raft.queue_time_us":35042,"thread_start_us":303792,"threads_started":1}
I20260709 12:29:00.975385  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:01.072110  7737 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:00.915263  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.008s	sys 0.019s
I20260709 12:29:00.978483  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1707) took 1774 ms. Trace:
W20260709 12:29:00.935042 16028 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
I20260709 12:29:01.073577  7733 rpcz_store.cc:276] 0709 12:28:59.203932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.969403 (+1765471us) service_pool.cc:224] Handling call
0709 12:29:00.977754 (+  8351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.978050 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.978090 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:00.978148 (+    58us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:00.978192 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.978271 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.978412 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:00.929806 15905 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.979946  7730 raft_consensus.cc:2376] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23. Re-sending same reply.
I20260709 12:29:00.890736 15971 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.957468 16005 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:00.990306  7734 rpcz_store.cc:276] 0709 12:28:57.103616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.104027 (+   411us) service_pool.cc:224] Handling call
0709 12:28:57.844868 (+740841us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:28:57.845183 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.848666 (+  3483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.848698 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:57.909833 (+ 61135us) log.cc:844] Serialized 26 byte log entry
0709 12:29:00.431561 (+2521728us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b18004a35c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:00.588106 (+156545us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:00.588184 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:00.588318 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:00.932050 (+343732us) raft_consensus.cc:1685] finished
0709 12:29:00.932078 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:00.932303 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44539,"raft.queue_time_us":47992,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":30042,"spinlock_wait_cycles":14192640,"thread_start_us":156136,"threads_started":1,"wal-append.queue_time_us":2849330}
W20260709 12:29:00.992823  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 187629us
I20260709 12:29:00.900816 15944 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:00.997521  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.001986 15489 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.075991 15992 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.098336  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.099941 15992 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:01.100394  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.011667  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1708) took 1474 ms. Trace:
I20260709 12:29:01.102500  7728 rpcz_store.cc:276] 0709 12:28:59.537409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.010672 (+1473263us) service_pool.cc:224] Handling call
0709 12:29:01.011078 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.011322 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:01.011345 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:01.011387 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:01.011426 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.011479 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.011603 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:01.013294 15852 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.014211  7731 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:29:01.015111 15981 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.017129 16027 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.018543  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1620) took 1540 ms. Trace:
I20260709 12:29:01.080709 15818 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:01.139613  8411 rpcz_store.cc:276] 0709 12:28:59.478165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.505558 (+ 27393us) service_pool.cc:224] Handling call
0709 12:28:59.566996 (+ 61438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.698673 (+131677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:59.698723 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:59.698788 (+    65us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:59.698834 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.018229 (+1319395us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:01.018302 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.018466 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302038912}
W20260709 12:29:01.025539  7662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1501) took 2345 ms (client timeout 1655 ms). Trace:
W20260709 12:29:01.141283  7662 rpcz_store.cc:269] 0709 12:28:58.679968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.699168 (+ 19200us) service_pool.cc:224] Handling call
0709 12:29:01.025258 (+2326090us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540021b1d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:01.025445 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927287680}
I20260709 12:29:01.026702  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1448) took 4173 ms. Trace:
I20260709 12:29:01.144522  8409 rpcz_store.cc:276] 0709 12:28:56.853113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.853268 (+   155us) service_pool.cc:224] Handling call
0709 12:28:58.186554 (+1333286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.187347 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.026417 (+2839070us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005b04d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:01.026599 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2994864896}
I20260709 12:29:01.034212  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:01.042172  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.146739  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.041894  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:01.063957  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:01.073367  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.009s	sys 0.019s
W20260709 12:29:01.149150  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 886976us
I20260709 12:29:01.150202  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:01.075884  7736 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:29:01.067992  7657 rpcz_store.cc:276] 0709 12:28:58.764542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.049001 (+284459us) service_pool.cc:224] Handling call
0709 12:29:00.964386 (+1915385us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400218ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:00.964537 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3218955648}
W20260709 12:29:01.080199  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.128s (SENT)
I20260709 12:29:01.043957 15731 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.999791 15778 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:01.068123 15772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.013s	sys 0.010s
I20260709 12:29:01.112255  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.138522  7656 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
W20260709 12:29:01.152753  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72616.1us
W20260709 12:29:01.167446  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 125626us
I20260709 12:29:01.168064  7735 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:29:01.168222  7627 leader_election.cc:336] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.139s (SENT)
I20260709 12:29:01.168735  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:01.170130  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57989.9us
W20260709 12:29:01.170956  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.157047  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.158954  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.171249  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.152961  7657 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 16. Current term is 17. Ops: []
I20260709 12:29:01.169971  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:01.173058  7808 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:01.095489 15607 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:01.168481  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:01.066293 15932 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:01.167207 15772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.013s	sys 0.011s
I20260709 12:29:01.201923  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3333) took 2409 ms. Trace:
I20260709 12:29:01.202574  7660 rpcz_store.cc:276] 0709 12:28:58.792449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.569953 (+1777504us) service_pool.cc:224] Handling call
0709 12:29:00.599183 (+ 29230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:00.599526 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:29:00.599907 (+   381us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.201401 (+601494us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:01.201482 (+    81us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:01.201556 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.201651 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.201836 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5056,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":84081,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":136,"thread_start_us":600976,"threads_started":2,"wal-append.queue_time_us":462676}
I20260709 12:29:01.088585 15905 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.090372 15971 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.169509 15838 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.266906  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:01.267788  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.165357  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:01.186067 15981 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.176153 16005 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:29:01.199843 15916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.014s	sys 0.052s
W20260709 12:29:01.280282  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 111908us
I20260709 12:29:01.185958 16027 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:01.139061 15775 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:29:01.282477  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.285713  7657 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:29:01.285759  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120508us
I20260709 12:29:01.287525  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:01.217743 15942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.785s	user 0.013s	sys 0.029s
I20260709 12:29:01.149912 16049 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.158s
I20260709 12:29:01.295936  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:01.300269  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.174376 15900 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 12:29:01.347246  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.184378 15731 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.421653  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.384508  7657 raft_consensus.cc:1275] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:29:01.291756  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 141711us
W20260709 12:29:01.423983  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43742.7us
I20260709 12:29:01.424772  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:01.427510  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:01.199558 15863 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.148s
I20260709 12:29:01.301458 15852 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:29:01.356961  7700 leader_election.cc:336] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.740s (SENT)
I20260709 12:29:01.444609  7811 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.280262 15887 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:29:01.423281  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 255085us
I20260709 12:29:01.418743  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.195098 15607 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:01.443631  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 246244us
I20260709 12:29:01.199810 15968 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.216s
I20260709 12:29:01.307629 15772 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:29:01.406832 15942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.953s	user 0.014s	sys 0.031s
W20260709 12:29:01.443902  7624 connection.cc:256] RPC call timeout handler was delayed by -1.1101721439999892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 12:29:01.444657  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 87818.7us
W20260709 12:29:01.322686  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.425122 16027 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:01.443825  7809 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: []
I20260709 12:29:01.449432  7660 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:29:01.345512  7808 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:01.369603 15818 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:29:01.459333  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.460171  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:01.462488  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.463519  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:01.438495 16005 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:01.290973 15931 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:01.420601 15916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.024s	sys 0.052s
I20260709 12:29:01.447625  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:01.473003  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.495916  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.478322  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.468428 15879 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:01.497279  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.297864 15838 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:01.499401  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51874.4us
W20260709 12:29:01.501470  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.501967  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.503615  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.476876 15772 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:01.507938  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:01.508836 15879 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:01.509642 16044 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:29:01.510526 16027 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.511559 16016 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:01.468212 16049 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:29:01.500319 15931 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.514287  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:01.515825 16041 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:29:01.516428  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.518045 15942 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.518946 15731 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:01.554401  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.521452 15607 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.523193 15905 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.472004 15852 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:01.468559 16045 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:01.572333 15942 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.572715 16045 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.526152  7660 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:01.536291 15947 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:01.572026 16040 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:01.529716  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3342) took 1113 ms. Trace:
I20260709 12:29:01.421969 15778 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.579610  7662 rpcz_store.cc:276] 0709 12:29:00.415882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.142812 (+726930us) service_pool.cc:224] Handling call
0709 12:29:01.143336 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.143622 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:29:01.462901 (+319279us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.529312 (+ 66411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:01.529373 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:01.529436 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.529510 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.529649 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":405568,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":119,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":2621,"thread_start_us":383969,"threads_started":2}
I20260709 12:29:01.490898 16030 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.582873 15852 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.583352 16030 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.511250  7624 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:01.534827  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72018.1us
W20260709 12:29:01.585918  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74810.6us
I20260709 12:29:01.586839  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.587877 15644 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:01.588773 15500 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.477679 15863 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:29:01.593348 15931 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.533812 16016 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:01.595620 16016 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:29:01.597431  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.598388  7660 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:29:01.542977  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 98452.8us
I20260709 12:29:01.600555  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.485828 16051 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:01.472637 16036 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
W20260709 12:29:01.519980 16039 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:29:01.612087 16036 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:01.532128 15803 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:01.612676 16036 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:01.528191 15772 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:01.568295  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.615422 15644 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.615631 15932 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.571206 15607 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:29:01.572016 16054 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:29:01.549902 16023 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.628s	user 0.014s	sys 0.002s
I20260709 12:29:01.491439 16005 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:01.613891 16036 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.606851  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:01.616029  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47793.4us
I20260709 12:29:01.627755  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3498) took 3014 ms. Trace:
I20260709 12:29:01.644135  7729 rpcz_store.cc:276] 0709 12:28:58.613506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.186599 (+1573093us) service_pool.cc:224] Handling call
0709 12:29:00.187489 (+   890us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:00.438836 (+251347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.627175 (+1188339us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540037ee54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:01.627627 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575125248}
I20260709 12:29:01.631997  7732 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:29:01.622481 15879 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.616324 15932 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:01.647971  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:01.649152 16005 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.649473  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:01.649628  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:01.652179  7731 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:01.654245 16002 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.616122 15644 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.651659  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:01.649195  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.668382  7729 raft_consensus.cc:1275] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:29:01.670738  8414 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:01.671792  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:01.672762  7809 raft_consensus.cc:606] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.680154  7811 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:01.684335  8411 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:29:01.620488 16057 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:29:01.649194 16036 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:01.626006 15917 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.652169 15942 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.705396  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:01.715708 16010 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.683s	user 0.003s	sys 0.007s
I20260709 12:29:01.719640 15644 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.704545 15932 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:29:01.699049 15731 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.721370 15932 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:01.721917 15932 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:01.722345 15942 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:01.723397 15942 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:01.725085 15942 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:01.727419 16055 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.237s	user 0.000s	sys 0.003s
I20260709 12:29:01.729619 15490 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.736972 15882 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:01.742671  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.743870  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3334) took 2930 ms. Trace:
I20260709 12:29:01.744465  7655 rpcz_store.cc:276] 0709 12:28:58.813143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.603885 (+1790742us) service_pool.cc:224] Handling call
0709 12:29:00.646221 (+ 42336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:01.062561 (+416340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:01.062619 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:01.063571 (+   952us) log.cc:844] Serialized 26 byte log entry
0709 12:29:01.272203 (+208632us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:01.272273 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.272399 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:01.743471 (+471072us) raft_consensus.cc:1685] finished
0709 12:29:01.743509 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.743771 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":283513,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":568331,"threads_started":2,"wal-append.queue_time_us":410021}
I20260709 12:29:01.722967 15932 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.753790 15932 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:01.756822  8415 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:01.750784 15944 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.764384  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3698) took 1436 ms. Trace:
I20260709 12:29:01.785992  7732 rpcz_store.cc:276] 0709 12:29:00.328242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.040703 (+712461us) service_pool.cc:224] Handling call
0709 12:29:01.631450 (+590747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.645889 (+ 14439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.764071 (+118182us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b540037ee50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:01.764285 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357620992}
W20260709 12:29:01.783718  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.787763  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.788296 15983 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.789198 15983 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.789364 16005 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.790777  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.791625  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:01.791818  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.794340 16023 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:01.795805 15692 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:01.796252 15692 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:01.797144 15692 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.798549 15692 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:01.798281 15772 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.801595 15983 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:28:59.107586  7812 rpcz_store.cc:276] 0709 12:28:56.577931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.000824 (+422893us) service_pool.cc:224] Handling call
0709 12:28:57.001212 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.002320 (+  1108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.002362 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.470119 (+1467757us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b2c001427d8. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:28:58.470668 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.708731 (+238063us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:58.815174 (+106443us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.044279 (+229105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.044370 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.044422 (+    52us) raft_consensus.cc:1685] finished
0709 12:28:59.044540 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.044729 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44093,"prepare.queue_time_us":108291,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":270358,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"spinlock_wait_cycles":7531264,"thread_start_us":431978,"threads_started":2,"wal-append.queue_time_us":543564}
I20260709 12:29:01.805816 16051 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:01.807629  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:01.775822 15887 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:01.820559  7662 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:29:01.812979  7658 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:29:01.819751  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:01.845885  7657 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:29:01.846597  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:01.851966  7656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3295) took 6360 ms (client timeout 2093 ms). Trace:
W20260709 12:29:01.852736  7656 rpcz_store.cc:269] 0709 12:28:55.452003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.142678 (+690675us) service_pool.cc:224] Handling call
0709 12:29:01.811932 (+5669254us) spinlock_profiling.cc:243] Waited 4.61 s on lock 0x7b540033e5d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:01.812145 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10593193216}
I20260709 12:29:01.816186  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:01.862588  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:01.862843  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 46818.5us
W20260709 12:29:01.863453  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46297.9us
I20260709 12:29:01.864753  7657 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:29:01.869210  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:01.870898  7657 raft_consensus.cc:2393] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:01.872041  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:01.874208  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:01.874890  7660 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:29:01.875617  7660 pending_rounds.cc:85] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 12
I20260709 12:29:01.875926  7660 pending_rounds.cc:107] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:29:01.876305  7660 raft_consensus.cc:2889] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:01.878153  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:01.811187 15887 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:01.884073  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178805us
I20260709 12:29:01.763111 15981 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.886188  7808 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:29:01.778398 15838 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:01.887663  8411 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:01.844823 15772 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:01.889313  7808 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:01.895282 15713 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:01.895018  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.908880  7812 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:01.868335 15944 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:01.781561 16016 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:01.884310 16064 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:01.899631  7811 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:29:01.901280  7806 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: []
W20260709 12:29:01.902069  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.902114  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.938762  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:01.940816  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.941344  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 292256us
I20260709 12:29:01.953009  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:01.955572  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.956976  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:01.957439  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:01.927835  8411 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:01.958148  7808 raft_consensus.cc:606] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:01.958621  7808 raft_consensus.cc:614] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:01.962224  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:01.962287  8409 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 12:29:01.948290 16011 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.900s	user 0.006s	sys 0.000s
I20260709 12:29:01.963148  7810 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:01.963785  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:01.916361 15905 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
W20260709 12:29:01.965514  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.973506 16076 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:29:01.974150  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.974274 16076 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.974841 16076 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:01.932966 15713 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:01.980312 15713 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:01.981129 15994 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:01.982230  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3331) took 3209 ms. Trace:
I20260709 12:29:01.969798 16025 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:29:01.983724 15788 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:29:01.982710  7661 rpcz_store.cc:276] 0709 12:28:58.791615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.453403 (+1661788us) service_pool.cc:224] Handling call
0709 12:29:00.500520 (+ 47117us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:00.816826 (+316306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.816880 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:01.733815 (+916935us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:29:01.734486 (+   671us) log.cc:844] Serialized 52 byte log entry
0709 12:29:01.928298 (+193812us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:01.946765 (+ 18467us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.947097 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.947193 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:01.981887 (+ 34694us) raft_consensus.cc:1685] finished
0709 12:29:01.981998 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:01.982154 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23271,"raft.queue_time_us":358896,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":213,"spinlock_wait_cycles":54390912,"thread_start_us":439017,"threads_started":2,"wal-append.queue_time_us":244813}
I20260709 12:29:01.985366 16045 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.986570 15903 raft_consensus.cc:993] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: : Instructing follower fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:29:01.987346 15903 raft_consensus.cc:1081] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Signalling peer fc963afbbf4c4b34b161448ef79bd2ed to start an election
I20260709 12:29:01.970788  7660 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:01.946256 16049 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:01.932085 16064 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.996527  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:01.997325  7810 raft_consensus.cc:606] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:01.992640  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:02.016324  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 415823us
I20260709 12:29:01.951342 15858 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:29:01.989827 16005 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:01.997864 16064 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:01.964402 15644 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:29:01.951267 15742 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 12:29:02.038493  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1623) took 2253 ms. Trace:
I20260709 12:29:02.015847 15852 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:02.049458  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:02.038839 16072 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:02.064002 16005 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:01.997478 16049 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:02.039019  8406 rpcz_store.cc:276] 0709 12:28:59.785112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.809540 (+ 24428us) service_pool.cc:224] Handling call
0709 12:28:59.810274 (+   734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:59.810601 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:29:00.265044 (+454443us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.773645 (+1508601us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b180046b280. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:29:02.023425 (+249780us) log.cc:844] Serialized 14 byte log entry
0709 12:29:02.038043 (+ 14618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:02.038111 (+    68us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:29:02.038220 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:02.038290 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:02.038428 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21502,"prepare.queue_time_us":740808,"prepare.run_cpu_time_us":284,"prepare.run_wall_time_us":13448,"raft.queue_time_us":33621,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":68038272,"thread_start_us":785381,"threads_started":3,"wal-append.queue_time_us":260433}
I20260709 12:29:02.074539  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:02.077080  7810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:02.078019  7810 raft_consensus.cc:606] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:02.078518  7810 raft_consensus.cc:614] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.017860 15584 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:29:02.085316  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.851s (SENT)
I20260709 12:29:02.063261 15852 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.088847  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:02.080660 16049 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:02.085812 16072 raft_consensus.cc:2764] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:29:02.086644 15731 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:02.096489 15852 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:02.099416 16045 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:02.164425  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 169790us
W20260709 12:29:02.163925 16083 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:29:02.164983 16011 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 12:29:02.165889  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:02.173935  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:02.105260 16005 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:29:02.187052  7627 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.949s (SENT)
I20260709 12:29:02.218233  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:02.162417 16076 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:02.248787  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 311589us
I20260709 12:29:02.250355  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:02.304602  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54443.8us
I20260709 12:29:02.216629 16005 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:02.227818 15944 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:02.325057 16078 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:02.355000 16093 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:29:02.319391 16068 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
W20260709 12:29:02.370074  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:02.306133 16016 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:02.320695 15713 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:29:02.380326 16084 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 12:29:02.381955 16005 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:02.384083 16005 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:29:02.446146 15887 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.002s
I20260709 12:29:02.375754 15863 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:02.378298 15713 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.569540 15713 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:02.566493 15863 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:02.694280  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:02.695477  7805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.003s	sys 0.009s
W20260709 12:29:02.696197  7805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.003s	sys 0.009s
W20260709 12:29:02.724437  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:02.694046 16076 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:02.447012 15887 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:02.701467 16023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.015s	sys 0.007s
W20260709 12:29:02.722664 15882 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.985s	user 0.017s	sys 0.000s
W20260709 12:29:02.738483 16023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.016s	sys 0.008s
I20260709 12:29:02.752235 15882 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:02.756278  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 142970us
I20260709 12:29:02.758462 15692 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:02.837409 15692 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:02.841962 15852 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.005s
I20260709 12:29:02.771977 15863 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:02.843273 15692 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:02.814934 16064 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.766s	user 0.013s	sys 0.003s
I20260709 12:29:02.844807 16064 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:02.816380 15887 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:02.847458 16064 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:02.757751 16086 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:02.848429 16086 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:02.799614 15882 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:02.886422 16064 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:02.887408 16064 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:02.892022 16064 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:02.894448 16005 consensus_meta.cc:287] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.004s
I20260709 12:29:02.913695 16086 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:02.842935 15852 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:02.971621 15852 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.070791  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:29:03.089160  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.090232  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1500) took 4410 ms. Trace:
W20260709 12:29:03.093084  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 905777us
I20260709 12:29:03.095031  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1513) took 1215 ms. Trace:
I20260709 12:29:03.095690  7655 rpcz_store.cc:276] 0709 12:29:01.880005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.880225 (+   220us) service_pool.cc:224] Handling call
0709 12:29:01.889932 (+  9707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:01.890375 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:01.890404 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:01.891281 (+   877us) log.cc:844] Serialized 26 byte log entry
0709 12:29:01.952151 (+ 60870us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:01.952225 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.952332 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:03.094266 (+1141934us) raft_consensus.cc:1685] finished
0709 12:29:03.094326 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.094789 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1704,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":60310,"threads_started":1,"wal-append.queue_time_us":172116}
W20260709 12:29:03.101007  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.980s (SENT)
I20260709 12:29:03.135821  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:03.138258  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.143294  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.147390  7661 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:03.163053  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3628) took 1072 ms. Trace:
I20260709 12:29:03.163833  7805 rpcz_store.cc:276] 0709 12:29:02.090812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:02.091050 (+   238us) service_pool.cc:224] Handling call
0709 12:29:02.091697 (+   647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:02.092025 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:02.092053 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:02.097877 (+  5824us) log.cc:844] Serialized 26 byte log entry
0709 12:29:02.696899 (+599022us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:02.696971 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:02.697101 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:03.161989 (+464888us) raft_consensus.cc:1685] finished
0709 12:29:03.162069 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.162640 (+   571us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":518503,"threads_started":1,"wal-append.queue_time_us":641661}
I20260709 12:29:03.168217  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:03.168682  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3627) took 1220 ms. Trace:
I20260709 12:29:03.172832  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:03.105245  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.05565e+06us
W20260709 12:29:03.105940  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.087908  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.113976  7731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.037s	sys 0.035s
W20260709 12:29:03.200601  7731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.038s	sys 0.036s
I20260709 12:29:03.090945  7653 rpcz_store.cc:276] 0709 12:28:58.679635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.118697 (+439062us) service_pool.cc:224] Handling call
0709 12:28:59.156842 (+ 38145us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:59.173209 (+ 16367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:59.173247 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:59.525250 (+352003us) log.cc:844] Serialized 52 byte log entry
0709 12:28:59.600115 (+ 74865us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:59.897074 (+296959us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.897631 (+   557us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.922199 (+ 24568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.922351 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:01.905594 (+1983243us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:03.089180 (+1183586us) raft_consensus.cc:1685] finished
0709 12:29:03.089267 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.089844 (+   577us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39905,"prepare.queue_time_us":939660,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":7608,"raft.queue_time_us":117960,"raft.run_cpu_time_us":664,"raft.run_wall_time_us":160939,"spinlock_wait_cycles":4765568,"thread_start_us":241869,"threads_started":2,"wal-append.queue_time_us":485299}
I20260709 12:29:03.115947  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:03.118034  7736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:46548
W20260709 12:29:03.125605  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.174144  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:03.178632  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.181038  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.181581  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.181919  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1627) took 2145 ms. Trace:
I20260709 12:29:03.169394  7812 rpcz_store.cc:276] 0709 12:29:01.947718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.947920 (+   202us) service_pool.cc:224] Handling call
0709 12:29:01.966307 (+ 18387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.966611 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:29:01.966927 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:29:03.167367 (+1200440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:03.167698 (+   331us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:03.167904 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.168126 (+   222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.168457 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":610,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":38924,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"spinlock_wait_cycles":3456,"thread_start_us":896237,"threads_started":2,"wal-append.queue_time_us":776356}
W20260709 12:29:03.188896  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1203) took 2106 ms (client timeout 1740 ms). Trace:
W20260709 12:29:03.192239  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.055s (SENT)
I20260709 12:29:03.193167  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:03.194679 15584 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.196239 15489 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:03.198081  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:03.104398  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.214382  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.203930  8414 rpcz_store.cc:269] 0709 12:29:01.082680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.099999 (+ 17319us) service_pool.cc:224] Handling call
0709 12:29:03.188292 (+2088293us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.188592 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309333376}
I20260709 12:29:03.218396 16005 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.218636  7733 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:03.219887  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:03.228910  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.235397  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:03.203619  8407 rpcz_store.cc:276] 0709 12:29:01.035989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.036193 (+   204us) service_pool.cc:224] Handling call
0709 12:29:01.036614 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:02.751654 (+1715040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:02.751710 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:02.751789 (+    79us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:02.751862 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.181211 (+429349us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540013fbd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.181300 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.181662 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5450,"raft.queue_time_us":10743,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":175,"spinlock_wait_cycles":3930282880}
I20260709 12:29:03.243752  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:03.245560  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:03.248624  8412 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:02.938453 15713 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.254372  7736 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:03.272536 15584 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.273197 16076 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.277738  8406 raft_consensus.cc:2368] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20.
I20260709 12:29:03.274384 16076 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.277971  7736 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:03.282346  7662 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:03.274385  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1473) took 1459 ms. Trace:
W20260709 12:29:03.295281  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.296193 15584 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:03.285620  8409 rpcz_store.cc:276] 0709 12:29:01.815352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.827973 (+ 12621us) service_pool.cc:224] Handling call
0709 12:29:03.273755 (+1445782us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b54006310d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.274124 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219315968}
I20260709 12:29:03.296746 15489 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:03.297063 16086 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.298051 16086 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.301519  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3725) took 1652 ms. Trace:
I20260709 12:29:03.302232  7731 rpcz_store.cc:276] 0709 12:29:01.648765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.655037 (+  6272us) service_pool.cc:224] Handling call
0709 12:29:01.655485 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.655745 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:29:01.907865 (+252120us) log.cc:844] Serialized 14 byte log entry
0709 12:29:03.232915 (+1325050us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:03.233088 (+   173us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:03.256298 (+ 23210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.262473 (+  6175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.301344 (+ 38871us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":332249,"prepare.run_cpu_time_us":212,"prepare.run_wall_time_us":202,"raft.queue_time_us":47699,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":224,"thread_start_us":1575942,"threads_started":3,"wal-append.queue_time_us":832275}
I20260709 12:29:03.305693  7731 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:03.306119  7731 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:03.306715  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.313740 16027 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:03.314522 16027 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:03.360503  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 107146us
I20260709 12:29:03.376904 16086 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:03.377514  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 185477us
I20260709 12:29:03.369935 16027 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.401328  7548 maintenance_manager.cc:419] P 004cbdb090d14f8ca89ed535ed34a625: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033560
I20260709 12:29:03.403293  7731 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.405236  7731 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:03.408056 15772 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:03.408926 15772 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.409528  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3613) took 2253 ms. Trace:
I20260709 12:29:03.410079  7807 rpcz_store.cc:276] 0709 12:29:01.155495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.155703 (+   208us) service_pool.cc:224] Handling call
0709 12:29:01.184950 (+ 29247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:03.407562 (+2222612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:03.407617 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:03.407676 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:03.407711 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.408988 (+  1277us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54001333d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.409041 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.409300 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5013543168}
I20260709 12:29:03.425060 16027 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:03.444093  7733 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:29:03.449148 16102 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 12:29:03.416513 16005 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.450776  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3339) took 4432 ms. Trace:
I20260709 12:29:03.451898 16094 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.452917  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1626) took 2419 ms. Trace:
I20260709 12:29:03.453692 16094 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.455458  8408 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:29:03.446116 15891 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.457237 15891 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.462433 16005 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:03.453487  8415 rpcz_store.cc:276] 0709 12:29:01.033725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.033909 (+   184us) service_pool.cc:224] Handling call
0709 12:29:03.452396 (+2418487us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54007a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.452672 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403923200}
I20260709 12:29:03.464787 16045 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.451435  7658 rpcz_store.cc:276] 0709 12:28:59.018665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.088877 (+2070212us) service_pool.cc:224] Handling call
0709 12:29:01.812428 (+723551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.821832 (+  9404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.450229 (+1628397us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540033e5d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.450531 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547679616}
I20260709 12:29:03.458135  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1471) took 1667 ms. Trace:
I20260709 12:29:03.467664  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:03.469453 15916 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:03.466879 15838 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.471426 15838 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.474776 15838 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.442333 16101 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.477062 16053 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:03.484494  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1474) took 1499 ms. Trace:
I20260709 12:29:03.486244 15971 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:03.466440  8408 rpcz_store.cc:276] 0709 12:29:01.790668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.792231 (+  1563us) service_pool.cc:224] Handling call
0709 12:29:03.454404 (+1662173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:03.456565 (+  2161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.457759 (+  1194us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54007a0150. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.458015 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3821566336}
I20260709 12:29:03.489557 15882 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:03.490480  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.491008  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3337) took 4475 ms. Trace:
I20260709 12:29:03.466169 16045 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.474932  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3335) took 4460 ms. Trace:
I20260709 12:29:03.498773  8405 rpcz_store.cc:276] 0709 12:29:01.985028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.987381 (+  2353us) service_pool.cc:224] Handling call
0709 12:29:01.987827 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:02.067943 (+ 80116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:02.067977 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:02.068028 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:02.068083 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.483955 (+1415872us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.484029 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.484300 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183641856}
I20260709 12:29:03.504500  7659 rpcz_store.cc:276] 0709 12:28:59.015301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.953313 (+1938012us) service_pool.cc:224] Handling call
0709 12:29:00.953667 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.066764 (+113097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:01.066797 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:01.066841 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:01.066879 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.490431 (+2423552us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000f4454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.490542 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.490802 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259491584}
I20260709 12:29:03.498472 16053 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:03.534682  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.540096  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 145846us
I20260709 12:29:03.543402 16101 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.508289  7654 rpcz_store.cc:276] 0709 12:28:59.014280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.884240 (+1869960us) service_pool.cc:224] Handling call
0709 12:29:00.884848 (+   608us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:29:00.990912 (+106064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.990964 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:29:00.994110 (+  3146us) log.cc:844] Serialized 78 byte log entry
0709 12:29:01.283152 (+289042us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:01.529572 (+246420us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.530125 (+   553us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.531071 (+   946us) log.cc:844] Serialized 14 byte log entry
0709 12:29:01.536942 (+  5871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.473797 (+1936855us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540021b6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.474053 (+   256us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:03.474180 (+   127us) raft_consensus.cc:1685] finished
0709 12:29:03.474203 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.474686 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6451,"prepare.queue_time_us":365523,"prepare.run_cpu_time_us":240,"prepare.run_wall_time_us":709,"raft.queue_time_us":14948,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":53154,"spinlock_wait_cycles":286913920,"thread_start_us":347120,"threads_started":2,"wal-append.queue_time_us":432629,"wal-append.run_cpu_time_us":4215,"wal-append.run_wall_time_us":1083280}
I20260709 12:29:03.546728  7729 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:29:03.548041  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:03.549372  7733 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:03.551700  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.552834  7732 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:03.560436 16076 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.561084 15500 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:03.562343  7734 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:03.543476 15838 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:03.564265 15838 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.572705  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.573747  7732 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:03.574999  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:03.576012  7732 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:03.576829 16094 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.577400  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:03.578470  7729 raft_consensus.cc:2376] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21. Re-sending same reply.
I20260709 12:29:03.579398 16045 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.546761 16101 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.580682  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:03.582792  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:03.585592  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:03.590188 16050 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 2.131s	user 0.005s	sys 0.035s
W20260709 12:29:03.607173  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82739.8us
I20260709 12:29:03.610672  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:03.567071 15838 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:03.607582  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80645.8us
I20260709 12:29:03.612875  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.614030  7729 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:29:03.614964  8411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1468) took 1825 ms (client timeout 1556 ms). Trace:
I20260709 12:29:03.614743 16086 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:29:03.615585  8411 rpcz_store.cc:269] 0709 12:29:01.789601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.791331 (+  1730us) service_pool.cc:224] Handling call
0709 12:29:03.614586 (+1823255us) spinlock_profiling.cc:243] Waited 94.6 ms on lock 0x7b5400321c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:03.614766 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217593728}
I20260709 12:29:03.644680  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:03.645641  7735 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:03.646070  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3723) took 1997 ms. Trace:
W20260709 12:29:03.647199 16048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.138s	user 0.020s	sys 0.032s
W20260709 12:29:03.656805  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44286.4us
I20260709 12:29:03.655779  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:03.662962  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52576.9us
I20260709 12:29:03.609071 16053 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.667624  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:03.668718 15916 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.671550 15993 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:03.656229 15882 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.017s
I20260709 12:29:03.675035  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:29:03.656298 16116 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.123s	user 0.002s	sys 0.000s
I20260709 12:29:03.686869 15882 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:03.687687 15882 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:03.674901 16048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.166s	user 0.020s	sys 0.032s
W20260709 12:29:03.688676  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 177907us
I20260709 12:29:03.683315  7731 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:29:03.695055  7731 pending_rounds.cc:85] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 14
I20260709 12:29:03.646874  7737 rpcz_store.cc:276] 0709 12:29:01.648016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.650807 (+  2791us) service_pool.cc:224] Handling call
0709 12:29:01.651188 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:01.652173 (+   985us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:29:03.553620 (+1901447us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b18003444a0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:29:03.553911 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 12:29:03.645081 (+ 91170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:03.645197 (+   116us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:03.645312 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.645435 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.645754 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":316393,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":126,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146,"spinlock_wait_cycles":6154240,"thread_start_us":436095,"threads_started":2,"wal-append.queue_time_us":90941}
I20260709 12:29:03.696283  7736 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.708112 15713 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.709162 15713 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:03.715691 16053 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:03.705634  7731 pending_rounds.cc:107] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:29:03.692610 15993 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:03.717878 15999 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:03.698750 15916 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.726100 15489 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:03.727514 15899 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:03.728327 15838 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.729029 15882 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:03.731704  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.732014 15772 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:03.733078  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:03.734411  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.736783  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.730921  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.723009  7731 raft_consensus.cc:2889] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:03.744163 16064 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:29:03.759204  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 120234us
I20260709 12:29:03.749639  7737 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:03.755054  7736 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:03.754262  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 61758us
W20260709 12:29:03.752503  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.655s (SENT)
I20260709 12:29:03.744593  7654 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:29:03.799921  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.806521  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.741231 15999 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:03.740741 16086 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:03.752624 15891 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:03.803845  7812 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:03.801393  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:03.815296  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.815865  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.816308  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.816835  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.818226  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:03.820295  7655 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:29:03.823287 15916 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:03.837997  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.840586  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84129.5us
I20260709 12:29:03.802376 16064 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.844532 15882 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:03.857308  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.857713 16123 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:03.858249  7806 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:03.858439 16123 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:03.859017 16123 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:03.863453 16086 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:03.867822  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.873991  7627 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.787s (SENT)
W20260709 12:29:03.874536  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.875584 16086 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:29:03.876257  7812 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:29:03.877795  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.879124  7736 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:29:03.880653 16067 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.861s	user 0.020s	sys 0.011s
I20260709 12:29:03.883420  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3612) took 2728 ms. Trace:
I20260709 12:29:03.883625  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.884732  7811 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:03.883970  7804 rpcz_store.cc:276] 0709 12:29:01.155078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.169427 (+ 14349us) service_pool.cc:224] Handling call
0709 12:29:01.197713 (+ 28286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:01.482873 (+285160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:01.482907 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:01.483584 (+   677us) log.cc:844] Serialized 52 byte log entry
0709 12:29:01.843904 (+360320us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:01.843976 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.186123 (+1342147us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003c9e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.186209 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:03.879512 (+693303us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:03.882986 (+  3474us) raft_consensus.cc:1685] finished
0709 12:29:03.883010 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:03.883208 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18190,"raft.queue_time_us":24822,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8037,"spinlock_wait_cycles":642804352,"thread_start_us":311728,"threads_started":1,"wal-append.queue_time_us":457265}
I20260709 12:29:03.888877  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.890021  7806 raft_consensus.cc:2368] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
I20260709 12:29:03.890297  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.891743  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.891834 15838 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:03.895419  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 137420us
I20260709 12:29:03.895581  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.896724  7805 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:03.906554 15838 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:03.906706  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:03.907797  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 159881us
I20260709 12:29:03.856652 16064 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:03.904559 15899 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:29:03.914889  7807 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:03.915210  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:03.919983  7805 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:03.917018  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.924975  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.924321 15993 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:03.926522  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.917853 15584 consensus_queue.cc:1048] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:29:03.927837  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:03.928544 15584 consensus_queue.cc:1243] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:29:03.928714  7809 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:03.915939  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.914762  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.932641  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.939798  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1515) took 2023 ms. Trace:
I20260709 12:29:03.940367  7660 rpcz_store.cc:276] 0709 12:29:01.909881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:01.910700 (+   819us) service_pool.cc:224] Handling call
0709 12:29:01.945497 (+ 34797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:01.989895 (+ 44398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.861759 (+1871864us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b540002f154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:03.933047 (+ 71288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57399680}
I20260709 12:29:03.940927  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:03.943928  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70058.5us
I20260709 12:29:03.945434  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.948064  7803 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:29:03.949436  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.924431 16035 consensus_queue.cc:1048] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 12:29:03.963794  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:03.964470  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:03.951622  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.965785  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:03.966693  7804 raft_consensus.cc:2368] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23.
I20260709 12:29:03.947137  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:03.974728  7736 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:03.990540  7735 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:03.992336  7809 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:29:04.000048  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.000830  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.002344  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.006705  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.009596 15938 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:04.010499 15938 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:04.013796 15938 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.014541 15938 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:04.015641  7737 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:29:04.017413  7737 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:04.011582 16035 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:29:04.019447 16027 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:04.020335 16027 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.023137  7807 raft_consensus.cc:1217] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Deduplicated request from leader. Original: 15.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 12:29:04.023690  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.010857  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:04.025684  7812 raft_consensus.cc:1217] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Deduplicated request from leader. Original: 14.10->[16.11-18.12]   Dedup: 16.11->[18.12-18.12]
I20260709 12:29:04.026384  7812 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [16.11-18.12]
I20260709 12:29:04.028191 16021 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:04.028530  7806 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:29:04.028894 16021 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.029382 16021 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:04.000108 15999 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.023936  7807 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:29:04.032047  7805 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:04.033239  7659 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:04.034734  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.037160  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:04.037257  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 16.
W20260709 12:29:04.038548  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 219052us
I20260709 12:29:04.015641  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:04.067500  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 169544us
I20260709 12:29:04.072142  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.075246  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:04.078181  7803 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:04.079815  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.023092 16027 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:03.963259 16053 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.110s
I20260709 12:29:04.095053  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:04.097857  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:04.106567  7811 raft_consensus.cc:2368] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:04.113652  7655 raft_consensus.cc:2393] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:04.121474 16123 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.102092  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.123687  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.124986  7653 raft_consensus.cc:2410] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:29:04.127985 16035 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:04.129527 16035 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:04.129998 16035 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:04.132136  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.135989  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.136123  7803 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 24. Ops: [19.14-22.15]
I20260709 12:29:04.149637  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:04.136708 16064 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.151995  7804 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:04.105787 15999 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:04.152846  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:04.165870  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:04.168676  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 223508us
W20260709 12:29:04.170481  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 219017us
I20260709 12:29:04.173077  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.078408 15938 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.165588 15772 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:04.183303  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.185009  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:04.194748  7655 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:04.196645  7660 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:04.206914 16125 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 12:29:04.207809  7547 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.375s	user 0.014s	sys 0.016s
I20260709 12:29:04.210785  7736 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:04.211467  7736 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:04.212425 16123 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.212682 16032 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:29:04.219748  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.198494  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:04.216388  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68923.8us
I20260709 12:29:04.224030  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.188620 15772 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:04.227039  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.212468  7736 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:04.243430  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.244287  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.246585  7658 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 17: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:04.131186 16035 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:04.213833 16123 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:29:04.261758 16035 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:04.265833 16064 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.217788 15993 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:29:04.271065  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44111.1us
W20260709 12:29:04.284029  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.271157  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:04.310272  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 225474us
W20260709 12:29:04.313707  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.247920 15772 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:04.315374 15588 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.269567 16053 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.316869  7804 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24. Ops: []
W20260709 12:29:04.317571  7627 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.412s (SENT)
I20260709 12:29:04.320647  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:04.321622 15882 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:04.314699  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.312026  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.280238 15993 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.315490 16123 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:04.327507  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 151293us
I20260709 12:29:04.350405  7737 raft_consensus.cc:2376] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24. Re-sending same reply.
W20260709 12:29:04.366111  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.396471  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:04.402446  7658 raft_consensus.cc:2393] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:04.407730  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.407699  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.411036  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:04.413430  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.414181  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.414640  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.432823  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:04.437469  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:04.372565 15993 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:04.415757  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:04.442160  8415 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:04.449390  7736 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:04.450917  8406 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:04.451038  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.448031  7653 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:04.459782  8408 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:29:04.433475 15891 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.460299  8408 pending_rounds.cc:85] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 16
I20260709 12:29:04.468843  8408 pending_rounds.cc:107] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:29:04.469163  8408 raft_consensus.cc:2889] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.464515  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:04.463737  7777 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:04.480229  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:04.480744  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.502960  7659 raft_consensus.cc:2410] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:29:04.406517 16053 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.505465  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.502425  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3735) took 1350 ms. Trace:
I20260709 12:29:04.415512 15882 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.402395 15588 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:04.443856 16134 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:29:04.462604 16094 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:04.508236  7658 raft_consensus.cc:2410] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:29:04.519905  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 352026us
W20260709 12:29:04.524435  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.507725 16115 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:29:04.533095 15588 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:04.537117  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.456065 16123 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.508044  7730 rpcz_store.cc:276] 0709 12:29:03.101329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:03.306242 (+204913us) service_pool.cc:224] Handling call
0709 12:29:03.307048 (+   806us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:29:03.363728 (+ 56680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:03.363772 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:29:03.476895 (+113123us) log.cc:844] Serialized 78 byte log entry
0709 12:29:04.322105 (+845210us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540001cd20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8c090 00007fc001e996b8 00007fc001f2292b 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c
0709 12:29:04.322199 (+    94us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:04.322655 (+   456us) log.cc:844] Serialized 14 byte log entry
0709 12:29:04.443324 (+120669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:04.443442 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:04.443496 (+    54us) raft_consensus.cc:1685] finished
0709 12:29:04.451506 (+  8010us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:04.451708 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30689,"prepare.queue_time_us":763,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":10304,"raft.queue_time_us":58893,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":47940,"spinlock_wait_cycles":268175232,"thread_start_us":174210,"threads_started":2,"wal-append.queue_time_us":51474}
I20260709 12:29:04.507696 15852 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:04.542399  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.560812  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:04.561848  7736 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:29:04.573486  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 255986us
I20260709 12:29:04.578824  7659 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 15.
W20260709 12:29:04.585542  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 137542us
W20260709 12:29:04.586100 16143 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:29:04.586622 16051 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:04.586763  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:04.587368 16132 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:04.587579 16051 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:04.588356 16132 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:04.589330  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:04.590628 16051 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:04.591856 15588 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:04.561539 15852 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:04.585623  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:04.592185 16144 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:29:04.585904  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.590260 16132 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:04.602831  7658 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:04.611933  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:04.613361  7700 leader_election.cc:400] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20.
I20260709 12:29:04.613771  7700 leader_election.cc:403] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:29:04.600853 15852 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:04.563822 15891 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:04.619958  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.654s (SENT)
I20260709 12:29:04.630100  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:04.608157 16032 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:04.656050  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.619617 16051 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:04.695868  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 84063.5us
I20260709 12:29:04.697690  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.707472  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:04.735965  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.737780  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80879.8us
I20260709 12:29:04.738523  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:04.743580  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 132280us
I20260709 12:29:04.745270  7654 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:04.761368  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:04.781829  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.785094 16047 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:04.785969 16129 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.332s	user 0.002s	sys 0.000s
I20260709 12:29:04.786746 16149 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:04.788116 16136 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.354s	user 0.003s	sys 0.001s
I20260709 12:29:04.799517 16032 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:04.805446  7658 raft_consensus.cc:2368] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
W20260709 12:29:04.806732  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 99568.9us
I20260709 12:29:04.787563 16149 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:04.825578  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.826300  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.829981 16021 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.800s	user 0.019s	sys 0.000s
I20260709 12:29:04.830238  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.830590 16021 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:04.833871 16021 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:04.834057  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.834427  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.836051  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.836107  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:04.837277  7730 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:04.838269  7809 raft_consensus.cc:1217] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Deduplicated request from leader. Original: 24.16->[25.17-25.17]   Dedup: 25.17->[]
I20260709 12:29:04.838934  7809 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:04.763967 15882 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.839704  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.841147  7811 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:29:04.842037 16149 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.842528  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.844132  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.844743  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:04.845360  7730 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:29:04.845605  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.846510 15898 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.847903  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:04.848716  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.849597 15932 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:04.847167 15898 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:04.853454 16146 log.cc:927] Time spent T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.318s	user 0.002s	sys 0.000s
I20260709 12:29:04.855980  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:04.857764  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 78219.7us
W20260709 12:29:04.858852  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.138s (SENT)
W20260709 12:29:04.873579  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.879458 16021 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:04.882244  7654 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:04.884560 15891 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.809566 16053 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.886550 16123 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:04.890031  8380 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:04.763103 16151 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20.
I20260709 12:29:04.892356 15898 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:04.892793  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:04.893694  7737 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:04.898234  7732 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:29:04.900298  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.901176  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.898926  7732 pending_rounds.cc:85] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:29:04.903474  7732 pending_rounds.cc:107] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:29:04.903954  7732 raft_consensus.cc:2889] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.902977  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.905962  7662 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:04.802043  7547 maintenance_manager.cc:643] P 004cbdb090d14f8ca89ed535ed34a625: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 1.399s	user 0.053s	sys 0.025s Metrics: {"bytes_written":4831,"delete_count":0,"lbm_write_time_us":5368,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":1920,"thread_start_us":237929,"threads_started":1,"update_count":186,"wal-append.queue_time_us":337066}
I20260709 12:29:04.952508  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:04.953330  7806 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:04.956707  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:04.959741  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 344000us
W20260709 12:29:04.966320  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.967779  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:04.972154  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 374050us
W20260709 12:29:04.974409  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:04.990056  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:04.991127  7807 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:04.992848  7736 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:04.993142  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:04.994102  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:04.994231  7731 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:05.023576  7548 maintenance_manager.cc:419] P 004cbdb090d14f8ca89ed535ed34a625: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.259743
I20260709 12:29:05.023761  7730 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:05.024317  7662 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:05.025146  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.028940  7658 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:05.031670  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:05.032781  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 173973us
I20260709 12:29:04.958412 16053 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:05.033476  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.031772  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:05.034423  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.035878  7654 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:29:05.032279  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:05.042853 15772 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.011s
I20260709 12:29:05.043448 15772 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:05.045485 15999 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.875s	user 0.000s	sys 0.013s
I20260709 12:29:05.050911 16053 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:05.027163 16132 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:29:05.052211 16053 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.054854  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.055902 16119 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.053s	user 0.003s	sys 0.002s
W20260709 12:29:05.056350  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.056527  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.057605  7736 raft_consensus.cc:2410] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:29:05.058928  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.059607  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.059676  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.060467  7654 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:05.060506  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.061254  7809 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:29:05.062350  7627 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.856s (SENT)
I20260709 12:29:05.062928  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.071878  7662 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:05.063834  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:05.072602  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115428us
I20260709 12:29:05.064612  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:05.065037 16047 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:05.065348  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:05.065560  7732 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:05.066234  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.083426  7657 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:05.066537  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.066148  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.084429  7803 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:05.061336  7656 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:05.068179  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.069097  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:05.058954  7736 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:29:05.073628  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.074815  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.075615  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.076915  7659 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:05.079746  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.079600  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.078974  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.082482  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:05.067124  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:05.093098  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 135465us
W20260709 12:29:05.098232  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.099644  7806 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:05.099773  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.100286 15772 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:05.141073  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48363.4us
I20260709 12:29:05.142659  7733 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:29:05.149000  7732 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:05.145109  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.145116  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.154573  7736 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
W20260709 12:29:05.154961  8379 leader_election.cc:336] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.525s (SENT)
I20260709 12:29:05.162061  7811 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:05.163275  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:05.099835  7804 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:05.174633  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.084973 16047 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:29:05.179100  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.137984 16035 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.876s	user 0.013s	sys 0.003s
I20260709 12:29:05.156875 15489 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:05.184672  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122425us
I20260709 12:29:05.185889  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.191067  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.191862  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.192569  7658 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:05.192613  7736 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19. Re-sending same reply.
I20260709 12:29:05.101902 15999 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:29:05.159421 16045 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:29:05.205336  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.199853  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102227us
I20260709 12:29:05.200632  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.207845  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:05.209131  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.183915 16064 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:05.217408 16047 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 12:29:05.218267  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76805.5us
W20260709 12:29:05.220973  8379 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.589s (SENT) [suppressed 31 similar messages]
I20260709 12:29:05.213699  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.172363 15971 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:29:05.214574 16030 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:05.215399 15852 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.000s
I20260709 12:29:05.215549 15882 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:05.219614  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.214365 16002 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 12:29:05.234370  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.871s (SENT)
I20260709 12:29:05.231046  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.232472  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.233278  7662 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:05.212988 15999 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:05.186647 15898 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:29:05.243893  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.236090  7659 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:05.249812  7736 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 15.
I20260709 12:29:05.254127  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.255113  7733 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:05.255395  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.230567 15932 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:05.237695 15882 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:05.269081  7732 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:29:05.271450  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.244658 16158 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.280683  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.286389  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:05.292816  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.294028  7732 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:05.295480  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.296373  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.297676  7733 raft_consensus.cc:2410] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:29:05.255705 15971 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.238632 15852 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.279814 15882 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:05.305896  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.310105  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.310670  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:05.312000  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.282366  7654 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:05.323789  7732 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:29:05.325228  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.326035  7732 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
W20260709 12:29:05.326707  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122803us
W20260709 12:29:05.327827  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.935s (SENT)
W20260709 12:29:05.329259  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.331544 15867 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
W20260709 12:29:05.336467 15993 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.874s	user 0.007s	sys 0.007s
I20260709 12:29:05.336089 15999 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.320093 16162 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:05.340286 16162 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.343132 15992 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:05.343729 15992 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:05.344882 15992 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:05.356393  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 168797us
W20260709 12:29:05.385106  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.387846  7732 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:29:05.387985  8411 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
W20260709 12:29:05.388906  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.390244  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.391561  7656 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:05.404189  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:05.347733  7627 connection.cc:256] RPC call timeout handler was delayed by -1.1816236720001143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.268s for a response
W20260709 12:29:05.409756  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:05.410221  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.283450 15932 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:05.405273  7736 raft_consensus.cc:2376] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19. Re-sending same reply.
I20260709 12:29:05.441403  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3175) took 1271 ms. Trace:
I20260709 12:29:05.441975  7655 rpcz_store.cc:276] 0709 12:29:04.169884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:04.170078 (+   194us) service_pool.cc:224] Handling call
0709 12:29:05.441128 (+1271050us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b54000f4454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:05.441285 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7024768}
W20260709 12:29:05.450125  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.298442 16158 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.455075  7657 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:05.460304  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.462764  7733 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:29:05.466843  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1740) took 1060 ms. Trace:
I20260709 12:29:05.467530  7729 rpcz_store.cc:276] 0709 12:29:04.406511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:04.507096 (+100585us) service_pool.cc:224] Handling call
0709 12:29:04.526782 (+ 19686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:04.696907 (+170125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:04.696942 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:04.991610 (+294668us) log.cc:844] Serialized 26 byte log entry
0709 12:29:05.177378 (+185768us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:05.177423 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:05.177515 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:05.466551 (+289036us) raft_consensus.cc:1685] finished
0709 12:29:05.466576 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:05.466751 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35761,"raft.queue_time_us":66473,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":11985,"thread_start_us":253916,"threads_started":2,"wal-append.queue_time_us":267254}
W20260709 12:29:05.474288 16147 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.733s	user 0.009s	sys 0.000s
I20260709 12:29:05.476745 15852 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:05.486816  7737 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.584s	user 0.005s	sys 0.005s
I20260709 12:29:05.310446 15971 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:05.480312  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.196s (SENT)
I20260709 12:29:05.456452  7734 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:05.491791  7737 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:05.498755  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.499691  7659 raft_consensus.cc:2376] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18. Re-sending same reply.
I20260709 12:29:05.501137  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.342475 15993 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:05.502068  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.503316  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.507004  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.507702  7729 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:05.504175  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 290282us
I20260709 12:29:05.505565  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.509383  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.510103  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.516014  7806 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:05.510807  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.517642  7734 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:05.519660  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.520391  7659 raft_consensus.cc:2376] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20. Re-sending same reply.
I20260709 12:29:05.478430 16162 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:05.527103  7805 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: [21.13-23.14]
I20260709 12:29:05.530372  7803 pending_rounds.cc:85] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 10
I20260709 12:29:05.531580  7803 pending_rounds.cc:107] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:29:05.531845  7803 raft_consensus.cc:2889] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:05.535581  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.536474  7734 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 16.
I20260709 12:29:05.556262  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.536458  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.541174 15932 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:05.551641  7655 pending_rounds.cc:85] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 11
I20260709 12:29:05.565320  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3753) took 1363 ms. Trace:
I20260709 12:29:05.574019  7655 pending_rounds.cc:107] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:29:05.568580 16158 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:05.574452  7655 raft_consensus.cc:2889] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:05.570351 15971 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:05.571507 15993 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:05.572960  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 119482us
I20260709 12:29:05.580826  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:05.582190  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.581820 15882 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:05.582861  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.583871  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3171) took 1484 ms. Trace:
I20260709 12:29:05.584721 16064 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:05.584520  7660 rpcz_store.cc:276] 0709 12:29:04.099327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:04.099609 (+   282us) service_pool.cc:224] Handling call
0709 12:29:05.583638 (+1484029us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b54000f4e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:05.583800 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129817856}
I20260709 12:29:05.592981 16141 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.593571 16141 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.574131  7728 rpcz_store.cc:276] 0709 12:29:04.201855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:04.202056 (+   201us) service_pool.cc:224] Handling call
0709 12:29:04.204809 (+  2753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:04.205325 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:29:04.205761 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 12:29:05.436663 (+1230902us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b180047b600. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:29:05.534670 (+ 98007us) log.cc:844] Serialized 14 byte log entry
0709 12:29:05.535057 (+   387us) log.cc:844] Serialized 14 byte log entry
0709 12:29:05.547151 (+ 12094us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:05.553139 (+  5988us) log.cc:844] Serialized 26 byte log entry
0709 12:29:05.553368 (+   229us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:05.553423 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:05.564808 (+ 11385us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:05.564871 (+    63us) raft_consensus.cc:1685] finished
0709 12:29:05.564889 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:05.565083 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11681,"prepare.queue_time_us":874,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":36677,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":12284,"spinlock_wait_cycles":90795776,"thread_start_us":120652,"threads_started":2,"wal-append.queue_time_us":151663}
W20260709 12:29:05.594614 16154 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.850s	user 0.008s	sys 0.000s
I20260709 12:29:05.554163  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:05.607332  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.607600  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.608194  7733 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:05.608549  7736 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:29:05.608495  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.610096  7728 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:29:05.614573  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 58428.2us
I20260709 12:29:05.616319  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.601017 16123 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.617374  7807 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:05.628258  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.628530  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.630846  8414 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:05.633248  7806 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:05.633172  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:05.633350  7731 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:05.628258  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.634649  7735 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:29:05.641096  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.643693  7736 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:05.645306  7736 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:05.646238  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.656000  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.661957  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:05.662569  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.665271  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:05.669071  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54233.5us
I20260709 12:29:05.673071  8414 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:05.676501  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.682124  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.683825  7662 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:29:05.685082  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45282.8us
I20260709 12:29:05.695509  7736 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:05.698148  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 99145.1us
I20260709 12:29:05.672564 16064 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:05.705243  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:05.703259  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:05.709167  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.638545 16123 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.703600  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:05.658643  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 105027us
I20260709 12:29:05.712507  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:05.715247  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.709363  7736 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:05.716981  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46980.2us
I20260709 12:29:05.718711  7737 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:05.719271  7731 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:05.719568  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.727592  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.705875 16064 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.733475  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.736645  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:05.708639 16101 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:05.740011  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:05.740767  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75762.4us
I20260709 12:29:05.752939  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:05.750562  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51232.4us
I20260709 12:29:05.732206 16051 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.719103 16141 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:05.761741  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:05.739830 16064 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:05.731876 15938 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:05.753515 15584 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:05.776948  8409 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:29:05.782552  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.800606  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.781647 15882 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:29:05.806699  7733 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:05.809098  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:05.781991 15916 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:05.856405  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:05.857825  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51690.8us
I20260709 12:29:05.805442 16141 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.763113 16101 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:05.868014  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:05.868278 16161 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.060s	user 0.000s	sys 0.005s
I20260709 12:29:05.868877  7661 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:05.869165 16002 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:05.869676 16002 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.870127 16002 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:05.869866  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:05.870929  7733 raft_consensus.cc:2376] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23. Re-sending same reply.
I20260709 12:29:05.776046 16051 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:05.872440  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.873716  7733 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
W20260709 12:29:05.877679  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 124811us
I20260709 12:29:05.879426  7661 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
W20260709 12:29:05.917145  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1247427969997261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 12:29:05.917902  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1247427969997261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 12:29:05.918315  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1103427969997028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:29:05.918726  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1103427969997028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 12:29:05.889366  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.925486  7736 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:29:05.877329 16184 log.cc:927] Time spent T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:29:05.939478  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:05.940833  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45144.9us
W20260709 12:29:05.878297 16174 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:29:05.947570  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.942112  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:05.953227 16123 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:05.955873  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.956799  7730 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:29:05.919124  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0842427969996606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 12:29:05.881388 15838 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
W20260709 12:29:05.946990 16188 log.cc:927] Time spent T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:29:05.948390 16182 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:29:05.972003  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.972880  7661 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:05.974205  7729 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:05.908134 16141 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:05.875169 16051 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:05.958549  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0842427969996606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 12:29:05.978240  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 223408us
W20260709 12:29:05.987674  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.018667  7702 connection.cc:256] RPC call timeout handler was delayed by -1.06534279699963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 12:29:06.018551  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:05.947558 15938 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:29:06.008720  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:06.016891  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 124669us
I20260709 12:29:06.017606  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.019495  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0491427969996039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 12:29:06.026803  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0491427969996039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 12:29:06.028507  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0221427969995602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
W20260709 12:29:06.029721  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0221427969995602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
W20260709 12:29:06.030128  7702 connection.cc:256] RPC call timeout handler was delayed by -1.002342796999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 12:29:06.023095  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.024483  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.031998  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.038312  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.055996  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:06.057369  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:06.059063 16100 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:06.059846 16193 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:29:06.061463 15731 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.010s
I20260709 12:29:06.062839 15731 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:06.063333 15731 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:06.046387 16051 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:06.064451 15731 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.065935 15731 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:06.067413 16040 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:06.068130 16040 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:06.069505 16040 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.071473  7653 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:29:06.071525 16005 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:06.081387  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.084859 16094 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:06.085575 16094 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:06.086740  7654 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:29:06.089267  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:06.089572 16053 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:06.090292 16053 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:06.090744  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:06.092823  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:06.093979  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.022042 15838 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:29:06.107890  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:06.108796  7654 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:06.109519 16101 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:06.091926 15992 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:06.110353  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:06.023928 16021 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:29:06.033742 15852 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:29:06.112094  7654 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:29:06.113559  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.114969  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:06.122430  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.127358  8411 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:06.127180  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:06.135058  7653 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:06.135484  8408 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:29:06.155388  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 137928us
I20260709 12:29:06.156013  8409 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:06.151503  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:06.167069  7728 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.000s
I20260709 12:29:06.167802  7728 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:29:06.175416  8405 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:06.129977 16100 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:06.128510 16051 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:06.133518 16048 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:06.180994 16158 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:06.182391  7654 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
W20260709 12:29:06.182981  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83285.2us
I20260709 12:29:06.183749  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:06.185498  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:06.189332  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:06.191128 16196 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:06.191459 16005 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:29:06.191862 16134 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:06.192368 16005 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:06.192799 16134 consensus_queue.cc:1243] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:29:06.192970 16005 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:29:06.199605 16027 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:06.200054  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.200299 16027 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:06.201648  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.201306 16027 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.201581  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:06.206478  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.187091  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:06.133502 16132 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:29:06.173460 16101 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:29:06.214088  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.217595  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.218133  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 127501us
I20260709 12:29:06.179121 16100 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:06.211138 16181 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.706s	user 0.010s	sys 0.000s
I20260709 12:29:06.107983 16053 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.220093 16158 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:29:06.243079  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.221904 16086 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:29:06.251327  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49851.7us
I20260709 12:29:06.259070  7804 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:06.231115 16100 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:06.234258 15838 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:06.241266 15882 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:29:06.293056  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104323us
W20260709 12:29:06.294986  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.295689  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.297202  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:06.301013  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.298245  7733 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:06.262086 16094 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:06.253513 16086 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.285235 16101 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:06.326900  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:06.328244  7729 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:06.260455 16206 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.088s
I20260709 12:29:06.324513 16101 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:06.322886 16086 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:06.355822  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 104749us
I20260709 12:29:06.332619 16132 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.306037 16126 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
W20260709 12:29:06.366307  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121395us
I20260709 12:29:06.371471  7729 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:06.372076  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:06.353662 16101 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:06.396504  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:06.409144  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 108308us
I20260709 12:29:06.411110  7803 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:29:06.417192  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.348191 16048 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 12:29:06.421655  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49908.3us
I20260709 12:29:06.427874 16209 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:06.428162 16203 log.cc:927] Time spent T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:29:06.428959  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:06.373421 16132 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:06.435748 16215 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:06.436582 16211 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:06.438342 16207 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:29:06.439522  7805 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:06.450335  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.464015  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44193.6us
I20260709 12:29:06.464983  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:06.448700  7812 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
I20260709 12:29:06.454051 16048 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:06.469095 16048 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:06.470191 16048 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:06.467424  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.461117 16064 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.004s
I20260709 12:29:06.473227 16064 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.474426 16141 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:06.474897  8414 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:06.483536  7777 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:06.495533  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.507103  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.511925  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.512586  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.519381  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 339045us
I20260709 12:29:06.478953 16048 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:06.522393  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55174us
W20260709 12:29:06.524513  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 306542us
I20260709 12:29:06.526823  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.527518  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.518770 16196 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:06.521389 16064 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:06.520524  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 333525us
I20260709 12:29:06.482033 15838 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:06.451697 16215 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:06.538419 15882 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:06.558372  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:06.558174  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:06.568097 16002 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.697s	user 0.016s	sys 0.000s
I20260709 12:29:06.568801 16002 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.574270 16214 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:06.575019 16214 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:06.575520 16214 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:06.578611 16173 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:06.579689  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:06.452162 16209 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:06.583781  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.584807  7808 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:06.586560  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.579418 16173 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.596683 16173 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:06.597393 16215 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.598857 16215 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:06.602933  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44659.7us
W20260709 12:29:06.632753  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107196us
I20260709 12:29:06.634805  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:06.539860 15971 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:06.558790 15992 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:06.663175  8405 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:06.556962 16158 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:06.541055 16027 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:06.697516  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:06.698484  7735 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:06.648180 16141 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:29:06.691390  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56739.2us
I20260709 12:29:06.586705 16132 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:06.699570  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 142063us
I20260709 12:29:06.644052 16162 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:06.647938 16094 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:06.711581  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 168727us
I20260709 12:29:06.711794  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:06.718015  7737 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:29:06.663779 15731 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.005s
W20260709 12:29:06.703961  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.684839 16002 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:06.726926  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.704900 16141 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.744828  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:06.734107  7812 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:06.732779  7702 leader_election.cc:400] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:06.717005 16162 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:06.674584 16086 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:06.756356  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 227457us
W20260709 12:29:06.770265  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.799924  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 94642.3us
W20260709 12:29:06.802043  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:06.803755  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59067.9us
I20260709 12:29:06.748287 16158 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:06.809639  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:06.749385 16027 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:06.819582  7702 leader_election.cc:403] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:29:06.840240  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:06.792816  7737 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:06.805826 16162 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:06.742794 16141 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:06.851872  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:06.855096  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.855402  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:06.855887 16236 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:29:06.856891  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:06.857905 16236 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:06.859025  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:06.861747 16233 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:29:06.863765  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.864451  7806 pending_rounds.cc:85] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 13
I20260709 12:29:06.864984  7806 pending_rounds.cc:107] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:29:06.865181  7809 raft_consensus.cc:2410] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:29:06.865371  7806 raft_consensus.cc:2889] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:06.862697  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:06.869189  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104613us
I20260709 12:29:06.857900  7808 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:06.866884  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:06.834044 16005 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.006s
I20260709 12:29:06.872591  7809 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
W20260709 12:29:06.874503  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 141680us
I20260709 12:29:06.876510  7805 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 25: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:06.814229 16228 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:06.883060  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:06.877831  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.886181  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:06.901371  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.901211  7653 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:29:06.904923  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.908301  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:06.909171  8409 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
W20260709 12:29:06.891465  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 358717us
I20260709 12:29:06.910341  7658 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:29:06.911098  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:06.724395 16231 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.167s
I20260709 12:29:06.912292  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:06.923856  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:06.923904  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:06.890462  7812 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:06.912992  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:06.943049  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 104876us
I20260709 12:29:06.886726 16228 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:06.977007  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.900120 15713 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:06.985401  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:06.995486  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 3010) took 2152 ms. Trace:
I20260709 12:29:06.933070 15993 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:06.996057  7811 rpcz_store.cc:276] 0709 12:29:04.842664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:04.869073 (+ 26409us) service_pool.cc:224] Handling call
0709 12:29:04.893378 (+ 24305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:05.168164 (+274786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:06.995079 (+1826915us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54003c4bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:06.995388 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617258240}
I20260709 12:29:06.885952 16005 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:06.887372 16027 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 12:29:07.019570 16223 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:07.020952 16223 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:07.021415 16223 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:07.021809 16223 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:07.025602 16234 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:29:07.026379 16086 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:07.029017 16100 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.010s
I20260709 12:29:06.989837 16228 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:06.997015 15916 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:29:07.052404  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:06.999919 15993 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:07.034662  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:07.056634  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.000366 16126 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:29:07.062126  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.017679 15731 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.076867  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.029639 16100 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:07.081305  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:07.091352  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 104501us
I20260709 12:29:07.058816 16194 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.058974 15993 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:06.999444 15713 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:07.120015  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.075506 15916 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:07.097980 16064 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:07.111526 15882 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:07.098718 15731 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.137138 16064 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:07.125705  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:07.204922 16238 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:29:07.206679 16218 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:29:07.209127 16213 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:07.161234 16210 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:07.210036 16210 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:07.210682 16210 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:07.213092 15713 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:07.208529 15916 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.148453 16246 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:29:07.152680 16241 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:07.139979 16126 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:29:07.171528 15882 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:07.243932  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 331013us
W20260709 12:29:07.248380  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122798us
I20260709 12:29:07.189662 16064 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:07.246613 16048 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.713s	user 0.014s	sys 0.001s
I20260709 12:29:07.242605 16246 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:29:07.256270 16101 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.834s	user 0.015s	sys 0.000s
W20260709 12:29:07.290555  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 152547us
W20260709 12:29:07.294005  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44986.1us
W20260709 12:29:07.304263  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 318932us
I20260709 12:29:07.268358 16173 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:07.305896  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:07.312075  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:07.320060  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:07.317989  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:07.324829  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.323128  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:07.326411  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:07.326642  7661 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:29:07.327527  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 226390us
I20260709 12:29:07.330437  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:07.332046 16224 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:07.332502 16224 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:07.332711  7733 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:29:07.333168 16224 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:07.333789  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:07.150100 16237 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.386s
I20260709 12:29:07.338183  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:07.289680 16246 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:07.271209 16032 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:29:07.346930  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.347687  7732 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:29:07.350092 16236 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:07.350657 16236 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:07.351747 16236 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.353314 16236 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:07.309049 16173 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:07.341526  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:07.341856  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:07.138355 16024 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.244s
I20260709 12:29:07.209955 16213 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.123973 16194 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.329334 16005 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:07.249487 16241 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:07.368577  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72836.7us
I20260709 12:29:07.293515 16101 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:07.382006  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51690.3us
W20260709 12:29:07.430214  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.430938  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:07.329043 16048 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:07.323282 16211 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.398092  7733 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:07.346242 16237 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:07.411374 16251 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:07.411959 16247 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:07.412528  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:07.412708  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:07.401544 15882 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.414112 15938 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:29:07.419065  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77327.8us
W20260709 12:29:07.424063 16141 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.539s	user 0.014s	sys 0.000s
I20260709 12:29:07.427484 16248 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:07.428984  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3388) took 1617 ms. Trace:
I20260709 12:29:07.453167  7656 rpcz_store.cc:276] 0709 12:29:05.811154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.069560 (+258406us) service_pool.cc:224] Handling call
0709 12:29:06.070016 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:06.243005 (+172989us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:29:06.292688 (+ 49683us) log.cc:844] Serialized 14 byte log entry
0709 12:29:07.210772 (+918084us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b180049fba0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a 00007fc001f2ca2a
0709 12:29:07.373715 (+162943us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:07.401488 (+ 27773us) log.cc:844] Serialized 26 byte log entry
0709 12:29:07.401675 (+   187us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:07.401715 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:07.401830 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:07.428453 (+ 26623us) raft_consensus.cc:1685] finished
0709 12:29:07.428615 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:07.428879 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22674,"prepare.queue_time_us":49473,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":139466,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":31067,"spinlock_wait_cycles":3851648,"thread_start_us":272215,"threads_started":2,"wal-append.queue_time_us":1090610}
I20260709 12:29:07.434240  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:07.435580  7732 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:07.376636 16100 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:07.442269  7730 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:29:07.443641  7729 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:07.374646 16173 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:07.458782  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.434113 16256 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:07.473912  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:07.480822  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.452723 16141 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.489825  8413 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:29:07.490942 16141 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:07.526108  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:07.533090  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.534888  7729 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:29:07.465726 15731 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:07.501354 16048 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:07.501729 16158 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:07.446688 16211 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:07.557111  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:07.561094  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 87271.8us
I20260709 12:29:07.558845  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.525455 16005 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:07.539577 16024 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:07.581432 16264 raft_consensus.cc:2764] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:29:07.602900  7627 connection.cc:256] RPC call timeout handler was delayed by -2.0418249890003608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 12:29:07.564883 16158 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.580802 16024 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:07.572960 16005 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:07.580152 16247 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:07.592914 16048 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:07.642581 16257 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.580981 16248 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:07.644868 16250 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
I20260709 12:29:07.672041 16267 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:07.672873 16267 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:07.674075 16267 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.672089 16270 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:07.676617 16270 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.681231 16158 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:07.683503 16257 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:07.688398 16223 consensus_meta.cc:287] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.006s
I20260709 12:29:07.690074 16257 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:07.690287 16248 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.640856 16251 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.645674 16005 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:07.685027 16214 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.109s	user 0.011s	sys 0.005s
I20260709 12:29:07.685173 16247 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.669368 16262 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:07.704281 16261 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:29:07.707605  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:07.723881  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93399.4us
I20260709 12:29:07.670902 16252 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:07.689056 16223 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.754127  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.724360 16194 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:07.737242 16241 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:07.777580  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:07.780248  7733 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:07.786294  7734 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:07.735146 16214 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.788031  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:07.722330 16048 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.792547  7732 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:07.778201 16226 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:29:07.736488 16251 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.743901 16262 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.812182 16262 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:07.817389 16223 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:07.817389 16214 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:07.839668  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92473.8us
I20260709 12:29:07.803548 16125 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:29:07.846904 16162 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.965s	user 0.011s	sys 0.005s
I20260709 12:29:07.876668 16162 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.837862 16211 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:07.885646  7627 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.197s (SENT)
I20260709 12:29:07.788791 16024 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:07.858642  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:07.840351 16226 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:07.862341  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:07.898468 16211 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:07.899070 16211 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:07.899628 16211 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:07.903641  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:07.904469  7734 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
W20260709 12:29:07.875147 16263 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:29:07.910434 15584 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:07.911067 15584 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:07.865439 16215 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.220s	user 0.016s	sys 0.000s
W20260709 12:29:07.870466 16204 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.548s	user 0.000s	sys 0.012s
W20260709 12:29:07.867830 16260 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.281s	user 0.002s	sys 0.001s
W20260709 12:29:07.853459 16228 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.778s	user 0.013s	sys 0.003s
I20260709 12:29:07.914486  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:07.871189 15993 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.000s
I20260709 12:29:07.864596 16048 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:07.848953 16172 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 12:29:07.912348 15584 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.952474 15584 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:07.917124 16210 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.001s
W20260709 12:29:07.953984  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:07.958175  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72584.3us
W20260709 12:29:07.958992  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.242s (SENT)
I20260709 12:29:07.950081 16024 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:07.963544 16064 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.691s	user 0.013s	sys 0.002s
I20260709 12:29:07.960347 16215 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:07.950525 16173 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.953649 15993 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:07.910485 16226 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:07.983706 16224 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.004s
I20260709 12:29:07.948824 16228 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.009944 16228 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.010833  8408 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:07.984839 16248 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.012765 16248 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.016891 16248 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:07.979995 16215 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:08.024835 16215 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:08.027676  7547 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.028s	user 0.010s	sys 0.012s
I20260709 12:29:08.030473 16173 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:08.034698  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 475912us
I20260709 12:29:08.035660  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:08.035863  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3397) took 1947 ms. Trace:
I20260709 12:29:08.036365  7654 rpcz_store.cc:276] 0709 12:29:06.088409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.127111 (+ 38702us) service_pool.cc:224] Handling call
0709 12:29:06.127549 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:06.182859 (+ 55310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:08.035523 (+1852664us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b54003980d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:08.035782 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125618432}
I20260709 12:29:08.036829  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:07.977651 16251 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.037775 16247 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:08.037906  7653 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:08.062052 16247 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:08.062783 16247 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:08.043367  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.055927 16224 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:07.960258 16210 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.081924 16101 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.052477 16211 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.091928  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1493) took 1190 ms. Trace:
I20260709 12:29:08.134126  8408 rpcz_store.cc:276] 0709 12:29:06.902073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.902507 (+   434us) service_pool.cc:224] Handling call
0709 12:29:06.943519 (+ 41012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:08.011462 (+1067943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:08.091529 (+ 80067us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:08.091834 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2453971840}
W20260709 12:29:08.102821  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:08.107625  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:08.053785 16246 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.006s
I20260709 12:29:07.983876 16064 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.116075 16162 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.082827 16101 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:08.158645  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:08.133570  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96842.5us
I20260709 12:29:08.155730  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:08.160322  7807 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:08.156282  7804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3646) took 2633 ms (client timeout 1825 ms). Trace:
W20260709 12:29:08.161199  7804 rpcz_store.cc:269] 0709 12:29:05.500216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:05.500505 (+   289us) service_pool.cc:224] Handling call
0709 12:29:08.087599 (+2587094us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54003f5054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:08.133158 (+ 45559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729708800}
W20260709 12:29:08.164322  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.166185  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:08.167016 16229 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.170s	user 0.006s	sys 0.006s
I20260709 12:29:08.172905  7655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3386) took 2416 ms. Trace:
I20260709 12:29:08.175046  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:08.176977  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:08.178125  7737 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:08.173379  7655 rpcz_store.cc:276] 0709 12:29:05.756750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.056356 (+299606us) service_pool.cc:224] Handling call
0709 12:29:06.070391 (+ 14035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:06.151569 (+ 81178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:06.151615 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:06.173161 (+ 21546us) log.cc:844] Serialized 26 byte log entry
0709 12:29:06.440241 (+267080us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b18002f5240. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:06.527952 (+ 87711us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:06.578718 (+ 50766us) log.cc:844] Serialized 14 byte log entry
0709 12:29:06.643739 (+ 65021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:06.643872 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:08.166170 (+1522298us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:08.172606 (+  6436us) raft_consensus.cc:1685] finished
0709 12:29:08.172633 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:08.172802 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120915,"prepare.queue_time_us":50043,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":249310,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":9370,"spinlock_wait_cycles":42939648,"thread_start_us":87409,"threads_started":1,"wal-append.queue_time_us":659587}
I20260709 12:29:08.180101  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:08.187615  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:08.191550  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:08.195264  7732 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:08.148427 15938 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:08.199543  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:08.212514  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3630) took 4369 ms. Trace:
I20260709 12:29:08.187704 16223 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:08.218636 16211 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:08.220665  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:08.201732 15938 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:08.190960 16276 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:08.201843 16248 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:08.232409  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1645) took 2500 ms. Trace:
I20260709 12:29:08.232918  8414 rpcz_store.cc:276] 0709 12:29:05.732264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:05.733064 (+   800us) service_pool.cc:224] Handling call
0709 12:29:08.232149 (+2499085us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:08.232331 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699278592}
I20260709 12:29:08.155809 16246 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.213150  7810 rpcz_store.cc:276] 0709 12:29:03.833172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:03.841505 (+  8333us) service_pool.cc:224] Handling call
0709 12:29:03.880852 (+ 39347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:04.003892 (+123040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:04.003947 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:04.004769 (+   822us) log.cc:844] Serialized 26 byte log entry
0709 12:29:04.323194 (+318425us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b180029ed80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:05.711777 (+1388583us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b180029ed80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9
0709 12:29:07.783692 (+2071915us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b180029ed80. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:07.801625 (+ 17933us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:07.801672 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:07.801766 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:08.202159 (+400393us) raft_consensus.cc:1685] finished
0709 12:29:08.202184 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:08.202368 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81927,"raft.queue_time_us":110641,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":6553,"spinlock_wait_cycles":143130112,"thread_start_us":17714,"threads_started":1,"wal-append.queue_time_us":3889260}
I20260709 12:29:08.219873 16101 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:08.219769 15993 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:08.244082 16276 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:08.262199 15932 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:08.265046 16162 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:29:08.283280  7736 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:08.292179 16246 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.294901 16064 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:08.317102  7732 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:08.276335 15882 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.313282  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.359884  8411 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: []
W20260709 12:29:08.361069  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.358356 16173 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:29:08.364250  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.365666  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.366714  8409 raft_consensus.cc:2410] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:29:08.362504  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:08.328987  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:08.311771 16005 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 12:29:08.331322  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:08.334822  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:08.277545 16276 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.340577 16224 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.278941 16228 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.304224 16247 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.304867 15932 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.376821  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.379772  7736 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:08.392282 15788 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:08.393764 16209 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.395469 16211 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:08.396278 16211 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.397907 16213 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.398845 16211 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.374178 15938 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.390673 16173 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:08.412977  7730 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:08.362720 16215 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.382056  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:08.360174 15882 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.382043  8408 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:08.414839  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.417085  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3402) took 1114 ms. Trace:
I20260709 12:29:08.417829 15938 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.407742  7736 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:08.418387 16173 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.381793 16005 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.429481  8409 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:08.430298  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:08.436456  7734 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:29:08.418010  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1548) took 1509 ms. Trace:
I20260709 12:29:08.437980  7653 rpcz_store.cc:276] 0709 12:29:06.908907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.932720 (+ 23813us) service_pool.cc:224] Handling call
0709 12:29:08.037501 (+1104781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:08.062402 (+ 24901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:08.417797 (+355395us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003980d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:08.417950 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494136576}
I20260709 12:29:08.427174  7657 rpcz_store.cc:276] 0709 12:29:07.302229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.302698 (+   469us) service_pool.cc:224] Handling call
0709 12:29:07.303186 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:07.451416 (+148230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:07.451453 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:07.451500 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:07.451555 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:08.416776 (+965221us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003994d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:08.416844 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:08.417005 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340174720}
I20260709 12:29:08.424319 16276 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.410851 15788 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:08.441075  8411 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
W20260709 12:29:08.434021  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.444662  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:08.445494 16021 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:08.450438 16035 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:08.451038 16035 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:08.452335 16035 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.453967 16035 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:08.446802 16024 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:08.466957  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 246096us
I20260709 12:29:08.390623 16228 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.427933 16158 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.476004  7729 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:08.482069 16125 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:08.482764 16125 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:08.482939 16215 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:08.485164  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1757) took 1152 ms. Trace:
I20260709 12:29:08.485697  7737 rpcz_store.cc:276] 0709 12:29:07.333089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.333297 (+   208us) service_pool.cc:224] Handling call
0709 12:29:08.484876 (+1151579us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:08.485040 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1937460480}
I20260709 12:29:08.486294 15882 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.472185 15788 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.487886  7734 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
W20260709 12:29:08.472519 16290 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:29:08.501719  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:08.502893  7736 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 15. Current term is 16. Ops: []
I20260709 12:29:08.506235  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:08.470460 16247 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.509665  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:08.512763  7737 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:08.501503 16215 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.508349 16064 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.538226  7737 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:08.480150 16158 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:08.556900  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 267439us
W20260709 12:29:08.571985  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.509546  7547 maintenance_manager.cc:643] P 004cbdb090d14f8ca89ed535ed34a625: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 3.485s	user 0.119s	sys 0.047s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":25980,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":250694,"dirs.run_cpu_time_us":601,"dirs.run_wall_time_us":11641,"lbm_read_time_us":3059,"lbm_reads_lt_1ms":24,"lbm_write_time_us":2399,"lbm_writes_lt_1ms":11,"peak_mem_usage":272498,"reinsert_count":0,"spinlock_wait_cycles":4034944,"thread_start_us":323354,"threads_started":3,"update_count":207,"wal-append.queue_time_us":262782}
W20260709 12:29:08.581231  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 113322us
I20260709 12:29:08.554498 16236 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.537216 16215 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.554906 16252 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.555066 16294 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:08.563745 16291 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.589706  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:08.598718  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.595780  7548 maintenance_manager.cc:419] P 004cbdb090d14f8ca89ed535ed34a625: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:29:08.597934  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.545658 16064 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.601574  7547 maintenance_manager.cc:643] P 004cbdb090d14f8ca89ed535ed34a625: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 12:29:08.600014  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.597096  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:08.554036 16213 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:29:08.601619  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.593281  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:08.607589  7660 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:08.608644  7662 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 16. Current term is 17. Ops: []
I20260709 12:29:08.609442  7737 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:08.607618  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.569432 15838 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.030s
W20260709 12:29:08.627728 16299 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:29:08.631986 16156 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:08.632601 16156 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:08.633849 16156 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.637230  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:08.638872  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.639473 16256 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:08.639937  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:08.640184 16251 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:08.641782  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:08.642184 16251 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:08.595985 16206 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:08.644755 16206 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.605428 16202 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
W20260709 12:29:08.647500  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.096s (SENT)
I20260709 12:29:08.648092  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:08.598747 15938 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.598070 16294 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:29:08.655889  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73388.5us
I20260709 12:29:08.657361  7702 leader_election.cc:400] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
I20260709 12:29:08.657786  7702 leader_election.cc:403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:08.651114  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:08.651862  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.654217  7803 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:29:08.661201 16266 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:08.661933 16256 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:08.663152 16256 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.665400 15932 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.661279 16209 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:08.667948 16173 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.667583 15938 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.613526 16247 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:08.598920 16252 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.672927 16252 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.673907 16294 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:08.675840 16206 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.677781  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.678321 16228 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.679111 16005 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:08.679507  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.680846  7732 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:08.667506 16209 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:08.670312 15938 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:08.682530  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 127607us
I20260709 12:29:08.680868  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.690722 16209 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.692436 16209 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:08.687767 15993 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:08.689666  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:08.594197 16236 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:08.700337  7730 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:08.703675  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:08.710119  7737 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:29:08.702682  8412 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:29:08.723356  8412 pending_rounds.cc:85] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 14
I20260709 12:29:08.723742  8412 pending_rounds.cc:107] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 12:29:08.724063  8412 raft_consensus.cc:2889] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:08.703886  7736 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:29:08.701606  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:08.694291  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.727421  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.730057  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.722039  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:08.732671  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.694152 16210 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:08.742197  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.733292  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.772850  7657 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:08.742273 16021 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:08.742100  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.741700  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.764334 16226 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:08.744215  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:08.712288 16094 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:08.777913  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44031.2us
W20260709 12:29:08.740633  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 83498.6us
I20260709 12:29:08.603475 16291 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.731523 16248 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19.
I20260709 12:29:08.730340 16213 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:08.719751 16202 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:08.777957  7659 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:08.766904 16301 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:29:08.733016 16228 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:29:08.778112  7734 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:08.781039  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:08.795923  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:08.779488  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:08.828444  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49621.4us
I20260709 12:29:08.915979  7660 raft_consensus.cc:1240] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:08.926200  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:08.975690  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:08.985337  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:08.986230  7732 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:09.006067  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1497) took 1801 ms. Trace:
I20260709 12:29:09.002668  7734 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:09.017416  7658 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:08.792029 16296 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:08.812455 16226 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.010267  8413 rpcz_store.cc:276] 0709 12:29:07.204254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.205460 (+  1206us) service_pool.cc:224] Handling call
0709 12:29:07.205954 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:07.490636 (+284682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:08.977757 (+1487121us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400321c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:09.005950 (+ 28193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651664256}
I20260709 12:29:09.017937  7658 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:08.978667 16035 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.005s
W20260709 12:29:09.063566  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 273827us
W20260709 12:29:09.073679  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 294256us
I20260709 12:29:09.039752 16236 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.065172 16303 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:29:09.073055  7658 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.087442  7658 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:09.035162  7657 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:08.915678 16094 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.036968 16296 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.042586 16248 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:09.037876 16215 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:08.918730 16251 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.165s
W20260709 12:29:09.091161  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 118410us
I20260709 12:29:09.086623 16274 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:09.053086 15838 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:09.098747  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.099794  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.109911  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:09.094731 16296 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:09.115674 15838 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:09.118079 16125 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 12:29:09.118888 15838 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.118788 16125 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.120002 16210 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:09.120273  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.121130  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.126241 16215 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:09.126820 15731 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:09.099763 16064 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:09.128271  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:08.916561 16021 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:08.970125 16202 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.134985 16296 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.139411 16307 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:29:09.173461 16307 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:09.174762 16307 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.142089 16291 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.091197 16226 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.142788 16305 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:29:09.143191  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3406) took 1697 ms. Trace:
I20260709 12:29:09.182166  7656 rpcz_store.cc:276] 0709 12:29:07.445806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.512308 (+ 66502us) service_pool.cc:224] Handling call
0709 12:29:07.543953 (+ 31645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:07.557643 (+ 13690us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:29:07.785275 (+227632us) log.cc:844] Serialized 14 byte log entry
0709 12:29:07.887482 (+102207us) log.cc:844] Serialized 14 byte log entry
0709 12:29:08.394728 (+507246us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800372d00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:08.517687 (+122959us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800372d00. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:29:08.550554 (+ 32867us) log.cc:844] Serialized 14 byte log entry
0709 12:29:08.593851 (+ 43297us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:09.131931 (+538080us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:29:09.132479 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:29:09.132689 (+   210us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:09.132987 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 12:29:09.133189 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:09.133271 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:09.142924 (+  9653us) raft_consensus.cc:1685] finished
0709 12:29:09.142951 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:09.143119 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":304508,"prepare.queue_time_us":310333,"prepare.run_cpu_time_us":245,"prepare.run_wall_time_us":4270,"raft.queue_time_us":719457,"raft.run_cpu_time_us":846,"raft.run_wall_time_us":33252,"spinlock_wait_cycles":73591808,"thread_start_us":258379,"threads_started":3,"wal-append.queue_time_us":80160}
I20260709 12:29:09.145524 16205 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:09.145900  7728 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:29:09.185674  7658 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:29:09.151137 16048 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:09.153899  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.155279  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3407) took 1681 ms. Trace:
I20260709 12:29:09.168311  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:09.172498  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51496.6us
I20260709 12:29:09.111621 16274 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.186409  7658 pending_rounds.cc:85] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 9
I20260709 12:29:09.188760  7733 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:09.192009  7658 pending_rounds.cc:107] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:29:09.192340  7658 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:09.188223  7661 rpcz_store.cc:276] 0709 12:29:07.473556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.726324 (+252768us) service_pool.cc:224] Handling call
0709 12:29:07.751324 (+ 25000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:07.751604 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:29:07.825410 (+ 73806us) log.cc:844] Serialized 14 byte log entry
0709 12:29:09.148960 (+1323550us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b18002ba860. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf
0709 12:29:09.149220 (+   260us) log.cc:844] Serialized 14 byte log entry
0709 12:29:09.150774 (+  1554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:09.150841 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:09.150900 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:09.154350 (+  3450us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:09.155128 (+   778us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":274843,"prepare.queue_time_us":119267,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":82,"raft.queue_time_us":257916,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":9442,"spinlock_wait_cycles":4642688,"thread_start_us":541370,"threads_started":3,"wal-append.queue_time_us":537837}
I20260709 12:29:09.194279  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.171514 16247 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:09.179850 16270 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.189389 16205 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:09.192888 16307 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:29:09.223557  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.171869 15731 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.240720  7702 leader_election.cc:400] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23.
I20260709 12:29:09.241357  7702 leader_election.cc:403] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:09.202250 16210 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:09.276252 16021 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.277942 16202 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.284852 16274 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:09.288920  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48467.6us
I20260709 12:29:09.290086  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.266494 16247 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23.
W20260709 12:29:09.307158  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 210018us
I20260709 12:29:09.311936  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3819) took 1687 ms. Trace:
I20260709 12:29:09.325013  7729 rpcz_store.cc:276] 0709 12:29:07.625047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.625288 (+   241us) service_pool.cc:224] Handling call
0709 12:29:07.640990 (+ 15702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:08.476711 (+835721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:09.311453 (+834742us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400282b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:09.311816 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919750016}
I20260709 12:29:09.284837 16296 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:09.302330 16294 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.628s	user 0.005s	sys 0.010s
W20260709 12:29:09.339044  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.691s (SENT)
I20260709 12:29:09.341058  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:09.305037 16210 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:09.335503 16202 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:09.341883 16125 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:09.360280  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 252747us
I20260709 12:29:09.354203 15852 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:09.345044 16294 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.306840 16205 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:09.373417  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 83471.9us
I20260709 12:29:09.376716  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.366428 16296 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:29:09.378417 16303 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:09.380295 16296 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:29:09.380784 16296 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:09.381718 16296 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.383857 16296 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:09.387279 16158 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.394774  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.395884  8405 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:09.352438 16202 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.399467  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.401997  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:09.404096  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 230998us
I20260709 12:29:09.405609  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:09.415007  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:09.421737  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.413748  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.414060 16305 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:09.431507 16305 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.431380  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.432108 16305 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:09.410799 16251 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:29:09.428529  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.412112 16210 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:09.404300 15891 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:29:09.435048 15891 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:09.435647 15891 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:09.436699 15891 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.437659  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.438144 15891 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:09.409832 15882 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:09.446764 16210 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:29:09.448374 16294 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.453423 16305 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.463454 16291 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:09.464457  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.414397 16228 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:29:09.466850  7702 leader_election.cc:400] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
W20260709 12:29:09.486918  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 189783us
W20260709 12:29:09.492491  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:09.492806  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.440229 16247 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:09.498545  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.502000  7732 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:09.427951 16064 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:29:09.485890 15731 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.503885  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.507858  7734 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:09.506316  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.495397 16291 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.513293  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.518055  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:09.519750 16314 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:29:09.517109 16291 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:09.516093 16202 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.530143  7777 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:09.530546  7734 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
W20260709 12:29:09.535588  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.532538 16051 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:09.539062 16051 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.539610 16051 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:09.543769  7777 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:09.545327  8413 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:09.545862 16291 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.546267 16032 consensus_queue.cc:1059] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:29:09.547286 16064 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:09.547900 16032 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:09.548740 16064 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:09.549292 16064 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:09.550328 16064 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.551642 16266 raft_consensus.cc:3055] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:09.552318 16266 raft_consensus.cc:740] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:09.552448 16064 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:09.548672 16291 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.556507  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.559007  7658 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:09.560283  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.561143  7657 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:09.557374  7654 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:09.549304 16305 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.566342 16048 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:09.516185 16228 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:09.567375 16205 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:09.567085 16048 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.567744  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:09.569774 16228 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:09.571059 16312 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:09.571599  7662 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:09.572089  7662 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:09.570986 16228 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.571072  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.572374 16312 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:09.583561  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.561962 16032 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:09.585093 16032 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.586475  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.583807  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.587668  7658 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:09.590655 16228 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:09.550495 16173 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:29:09.593673 16125 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:09.594915 16173 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:09.595454 16173 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:09.572988  7662 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.578332  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:09.598701  7662 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:29:09.579300  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 69628.6us
I20260709 12:29:09.600358  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:09.601976  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.606068  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.606892  7737 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:29:09.606422  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.610941  7731 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:29:09.603556  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:09.569772 16048 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.632053  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.633550 16265 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:09.634119 16265 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.634513 16265 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:09.609555 16210 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:09.637827  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.642310 16094 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.645162  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:09.649314  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.650090  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.650465  8411 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:09.651029  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.651427 16231 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:09.651984  8414 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:09.652165 16231 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.653049  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.653532  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.654366  8412 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:09.654722 16231 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.657768 16156 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:09.651599  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:09.666070  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.668793 16312 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:09.665623  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.068s (SENT)
I20260709 12:29:09.679988  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:09.682721  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 53243.9us
W20260709 12:29:09.688606  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43526.6us
I20260709 12:29:09.689898  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:09.692917  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.609158 16208 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:09.695073  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:09.694988  7737 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:09.699343  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.699483  7662 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:29:09.700807  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.701242  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:09.709265  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.709617  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.710233  7657 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:09.712595  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:09.714722  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9436626110000361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 12:29:09.715744  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.717382  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.718554  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.720605  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.722368  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.723284  7737 raft_consensus.cc:2393] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:09.724162  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.726585  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:09.728645  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:09.726625  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.725446  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.735880 15993 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:09.736079  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.736980  7732 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 16: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:09.736680 15993 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.711064  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.738052  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.738152  7660 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:09.738933  7732 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:09.739598  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.739889  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.740454  7804 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:09.741089  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.742913  7732 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:09.703588 16002 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:09.743937  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.743866 16002 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.745344  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.745522 15971 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:09.746003 15971 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:09.746189  7732 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:09.746426 15971 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:09.747192  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.747575 16048 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:29:09.748000  7732 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:09.750773  7730 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:29:09.752408  7730 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:29:09.752840 16158 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:09.753929  7730 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:29:09.741381  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.755605  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:09.742019 15867 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:09.756477  7729 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:29:09.757042 16247 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:09.757913 16247 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.759004  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.760507 16247 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:09.760591  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.757042 15993 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.762655  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.763563  7734 raft_consensus.cc:2368] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:29:09.764564  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.765393  7658 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:09.764979 15971 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.718190  7810 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:09.742182 16227 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:09.782392  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:09.784512  7730 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:29:09.741937  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:09.741585  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45773.3us
I20260709 12:29:09.785708  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:09.756872  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.739609  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.788533  7661 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:09.740238  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.790045  7656 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:29:09.741035  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.741473  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.791618  7805 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:09.776678  7657 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:09.769904  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:09.784651  7659 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:09.813603  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.814481  7810 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:09.819583  7810 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:09.820890  7731 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:09.822793  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:09.818383  8379 connection.cc:256] RPC call timeout handler was delayed by -1.4975595739997516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:29:09.824681  8379 connection.cc:256] RPC call timeout handler was delayed by -1.3211595739999211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 12:29:09.825220  8379 connection.cc:256] RPC call timeout handler was delayed by -1.2806595739998556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 12:29:09.825742  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.826860  7812 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:09.825973  8379 connection.cc:256] RPC call timeout handler was delayed by -1.2242337709999447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:29:09.829152  8379 connection.cc:256] RPC call timeout handler was delayed by -1.1519595739996475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 12:29:09.832000  8379 connection.cc:256] RPC call timeout handler was delayed by -1.036759573999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
I20260709 12:29:09.701725 16208 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:09.701079 16215 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:09.834102  7657 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:29:09.835639  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50041.2us
W20260709 12:29:09.822706 16301 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.025s	sys 0.044s
I20260709 12:29:09.836185  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:09.740975 16170 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:09.837466  7803 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
W20260709 12:29:09.840346  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 174787us
I20260709 12:29:09.856160  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:09.746434 16002 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.853281  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.673868 16266 consensus_queue.cc:260] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.663511 16173 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:09.741400 16235 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:09.867024  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:09.924882  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.989329  7810 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:09.989916  7658 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:09.881561  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:09.974694  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 229649us
I20260709 12:29:09.858749  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:09.962404  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:09.918031  7812 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:29:09.962788  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:09.990649  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.996052  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:09.905138  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:09.997762  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:09.868393  7660 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:09.991928  7731 raft_consensus.cc:2368] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22.
I20260709 12:29:09.870419  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:09.998250  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.833s (SENT)
I20260709 12:29:09.993716  7807 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:09.999987  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.779052 16094 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:09.912575  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:09.961802  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:10.011035  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:10.014871  7661 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:10.002658  7655 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:10.014546  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.017306  7812 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:09.999127  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.021078  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.834533 16227 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:09.853317 16303 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:10.030007  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:10.032615  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 173954us
I20260709 12:29:10.031104  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:09.841113 16208 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:10.034801  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:09.991375 16301 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.026s	sys 0.046s
I20260709 12:29:10.001317 16235 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:10.042137 16125 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:09.952806 15971 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:09.918816 15882 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:10.023439  7805 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:10.045475  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:10.001035 15891 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:09.919654 16215 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:10.046437  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:10.022336 16307 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.004s
I20260709 12:29:10.048972  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.049413  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.053069  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.055595  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:10.013419 15916 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:10.058167  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.071060  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.077330  7730 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:10.030982 16170 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:10.071740  7803 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:10.042399 16235 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:10.038779 16048 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:10.059303 16125 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.076031  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.077790  7658 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:29:10.072335  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.088668  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.046064  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:10.089588  7729 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:10.080279  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.090263  7812 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:10.072512  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:10.067818  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:10.091020  7810 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:10.091419  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.074843  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:10.057112 16173 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:10.062914 16202 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:10.078694  7730 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:10.089802  7731 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:10.092974  7811 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:10.092269  7812 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:09.823017 16247 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.215s
I20260709 12:29:10.094492  7805 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:10.048511 16156 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:29:10.066126 16303 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:10.092092  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:10.101483  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.117581 15852 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:10.118103 15852 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.118501 15852 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.121474  7805 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:10.123893 16251 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:29:10.084543 16213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.001s	sys 0.020s
I20260709 12:29:10.138800  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:10.104729  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55883.7us
I20260709 12:29:10.141295  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.106928  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.058506 16257 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:29:10.024713 16094 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.094094 16125 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:10.160563  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:10.126253 16251 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:10.096109 15838 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:10.144212  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.096664 16202 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.154546  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.072024 15891 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:10.170867  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:10.170914  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:10.145210 16213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.002s	sys 0.021s
I20260709 12:29:10.137377 15882 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:10.175596  7627 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.854s (SENT)
I20260709 12:29:10.179981  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.180986  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.181910  7731 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:10.182443  7659 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:29:10.156260 16312 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:10.180630  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.169495 15838 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.189312 16303 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:09.859277  7550 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:10.197764  7627 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.070s (SENT)
I20260709 12:29:10.132405 16227 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.198190 15838 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:10.199429 16027 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:10.198339  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:10.174500 16202 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:10.183727 16051 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.641s	user 0.005s	sys 0.010s
I20260709 12:29:10.207818  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:10.201288  7803 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:10.150302 16274 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:29:10.211900 16322 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:29:10.212809 15891 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.262642  7807 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:10.213572 16094 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.174048 16213 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:10.266273 16209 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:10.020991 16226 consensus_queue.cc:1175] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" responder_term: 20 status { last_received { term: 19 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:29:10.215762 16247 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:29:10.186215  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126316us
I20260709 12:29:10.178479  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.185532 16329 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:29:10.259522  7627 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.611s (SENT) [suppressed 27 similar messages]
I20260709 12:29:10.260397  8405 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
I20260709 12:29:10.231959 15932 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.264757  7803 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26. Ops: [23.17-25.18]
I20260709 12:29:10.204730 16210 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:10.279390  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:10.291990  7656 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.501s	user 0.016s	sys 0.000s
W20260709 12:29:10.276053  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97710.4us
I20260709 12:29:10.295469  7656 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 16.
W20260709 12:29:10.297806  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 90116.4us
W20260709 12:29:10.264012 16064 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.711s	user 0.015s	sys 0.000s
I20260709 12:29:10.174896 16215 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.312997  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:10.318023  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:10.267832 16205 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.700s	user 0.006s	sys 0.007s
I20260709 12:29:10.327353  7803 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:10.276839 16311 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:10.333026  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 216541us
I20260709 12:29:10.340260  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.333590  7653 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:10.315007  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:10.341609  7803 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:29:10.345808 16342 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:10.346624 16342 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.347245 16342 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.196976 16170 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.270304 16247 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:10.314215 15772 raft_consensus.cc:2764] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:29:10.258730 16051 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.276997 15852 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.331487 16210 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.269603 16209 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.339448 16327 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:10.300439 15838 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.277396 15932 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.368414  7810 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:10.368031 16205 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:10.387575 16338 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:10.388633 16344 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.327108 16156 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:29:10.429064  7812 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:10.360157 16247 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:29:10.436803  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 75269.9us
I20260709 12:29:10.268894 16213 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:10.422976 16326 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:29:10.445221  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.395481 16210 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:10.435606 16332 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:29:10.451674  7657 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:10.451613  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.459635  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:10.455497  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 140591us
I20260709 12:29:10.459777 16347 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.461488  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:10.446354 16343 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:10.466428  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:10.473673  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.454727 16307 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:10.494019 16231 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:10.494604 16231 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:10.495723 16231 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.498481 16231 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.465340 16228 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:10.475467 16064 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:10.477420 15882 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:10.578946  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.580684  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.597318  7731 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:10.584331  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:10.598287  7733 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.585178 16346 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:10.585331 16328 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 12:29:10.586688 16336 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:29:10.586225 16307 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:29:10.587613 16334 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:29:10.591095  7730 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 12:29:10.605250 16265 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.002s
I20260709 12:29:10.606653 15838 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.606940 16210 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:10.607987 16341 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:29:10.608911 15852 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.609043 16051 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.611055 15932 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.612480 16209 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.445008 16344 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.575603 16343 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:10.481666 15891 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.630440  8407 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:10.668115  7810 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:10.472900 16235 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.619410 15882 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:10.619798 16094 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:10.642319 16208 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.581s	user 0.005s	sys 0.009s
W20260709 12:29:10.691802  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 215033us
I20260709 12:29:10.476892 16347 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.700589  7804 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:10.639901 16064 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:10.701105  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:10.704089  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 239953us
I20260709 12:29:10.713829  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:10.731287  7733 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:10.730911 16338 consensus_queue.cc:1243] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 22.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:29:10.739419  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:10.740396  8406 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:10.702968 15838 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:10.744326 15838 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:10.727514 16125 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.004s
I20260709 12:29:10.640790 16346 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.749444  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:10.750226  7810 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.712049 16027 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:10.710247 15882 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:10.681092 16265 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.722102 16343 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:10.756937  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:10.701968 16210 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:29:10.345697  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:10.757513  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:10.795637  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 3019) took 1076 ms. Trace:
I20260709 12:29:10.791908  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.796589 16343 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.797276 16032 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:10.799072  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3670) took 1064 ms. Trace:
I20260709 12:29:10.799263  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.799590  7804 rpcz_store.cc:276] 0709 12:29:09.734453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.813869 (+ 79416us) service_pool.cc:224] Handling call
0709 12:29:09.822185 (+  8316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:10.720049 (+897864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:10.798818 (+ 78769us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54000681d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:10.799003 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2019463936}
I20260709 12:29:10.800253 16032 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:10.800422 16347 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.801394  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.801990  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1783) took 1196 ms. Trace:
I20260709 12:29:10.802402  7736 rpcz_store.cc:276] 0709 12:29:09.605270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.605485 (+   215us) service_pool.cc:224] Handling call
0709 12:29:09.606080 (+   595us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:09.606552 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:09.606588 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:09.617435 (+ 10847us) log.cc:844] Serialized 52 byte log entry
0709 12:29:09.761026 (+143591us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b18003eb9c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:10.140698 (+379672us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:10.252880 (+112182us) log.cc:844] Serialized 14 byte log entry
0709 12:29:10.417516 (+164636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:10.417606 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:10.801610 (+384004us) raft_consensus.cc:1685] finished
0709 12:29:10.801731 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:10.801929 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2215,"prepare.queue_time_us":171595,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":121414,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":67859,"spinlock_wait_cycles":72206080,"thread_start_us":628247,"threads_started":3,"wal-append.queue_time_us":630728}
I20260709 12:29:10.802165  7732 raft_consensus.cc:1217] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Deduplicated request from leader. Original: 16.14->[18.15-18.15]   Dedup: 18.15->[]
I20260709 12:29:10.710719 15891 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:10.803870  7653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45918 (request call id 3435) took 1244 ms (client timeout 1533 ms). Trace:
W20260709 12:29:10.828662  7653 rpcz_store.cc:269] 0709 12:29:09.559358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.559541 (+   183us) service_pool.cc:224] Handling call
0709 12:29:10.803662 (+1244121us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b540033d6d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:10.803799 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1726648192}
I20260709 12:29:10.807173  7732 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:29:10.796234  7811 rpcz_store.cc:276] 0709 12:29:09.719006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.719212 (+   206us) service_pool.cc:224] Handling call
0709 12:29:09.719541 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:10.099753 (+380212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:10.795288 (+695535us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54003ea8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:10.795540 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847223552}
I20260709 12:29:10.803202  7804 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:10.790135  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:10.832818  7730 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:10.820468  7733 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:10.729058 16312 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 12:29:10.723964 16094 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.723937 16208 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.842484  7806 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:10.844880  7810 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
W20260709 12:29:10.774248 16202 consensus_meta.cc:287] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
I20260709 12:29:10.849634  7811 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:10.808804 16205 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:29:10.864045  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.238s (SENT)
I20260709 12:29:10.808269 16267 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:10.866568  7733 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:10.760115 16125 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.868484  7805 raft_consensus.cc:2393] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:10.871666 16349 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:29:10.881038  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1220) took 1182 ms. Trace:
I20260709 12:29:10.873678 16344 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.856312 16311 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:10.872241 16208 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.803560 16027 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:29:10.903364  8406 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:10.891976  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.875368 16064 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.874935 16094 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.902738  8407 rpcz_store.cc:276] 0709 12:29:09.698464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.698884 (+   420us) service_pool.cc:224] Handling call
0709 12:29:10.880852 (+1181968us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400587854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:10.880967 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442773376}
W20260709 12:29:10.915246  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 186272us
I20260709 12:29:10.869716 16267 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:10.867134 16205 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:10.931355 16235 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:10.931715 16205 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.933341 16205 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:10.934247 16173 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.835s	user 0.011s	sys 0.003s
W20260709 12:29:10.935638  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71629.5us
W20260709 12:29:10.936512  7627 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 3.253s (SENT)
I20260709 12:29:10.882887 15838 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.861536 16202 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.940640  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.946189  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.939002  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:10.941970  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:10.942714  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:10.950415  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:10.912955 16357 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.956923  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.938123 16227 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.931474 16265 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:10.941777 16027 consensus_queue.cc:1243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:29:10.941592 16346 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:10.947970 16351 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:10.954073 16318 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:10.975250  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.957567 15838 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:10.977391  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:10.979936  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3674) took 1244 ms. Trace:
W20260709 12:29:10.985316 16342 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.002s
I20260709 12:29:10.985634 15882 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.976733 16357 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:10.978479 16227 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:10.987833  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3207) took 1168 ms. Trace:
I20260709 12:29:10.989064  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:11.007925  7659 rpcz_store.cc:276] 0709 12:29:09.819691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.822710 (+  3019us) service_pool.cc:224] Handling call
0709 12:29:09.823136 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:10.182957 (+359821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:10.987467 (+804510us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54001a3bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:10.987742 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825470336}
I20260709 12:29:10.992026 16213 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:10.992367 16141 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:10.996892  7807 rpcz_store.cc:276] 0709 12:29:09.735107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:10.149022 (+413915us) service_pool.cc:224] Handling call
0709 12:29:10.149534 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:10.263255 (+113721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:10.979466 (+716211us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003c9e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:10.979842 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259546496}
I20260709 12:29:10.999404  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.013320  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.014760  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.009328  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3583) took 1288 ms. Trace:
I20260709 12:29:11.018851  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.015565  7732 rpcz_store.cc:276] 0709 12:29:09.721145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.748803 (+ 27658us) service_pool.cc:224] Handling call
0709 12:29:10.801999 (+1053196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:10.830313 (+ 28314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:11.009077 (+178764us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540037c150. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:11.009252 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2421152640}
I20260709 12:29:11.015714 16141 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:11.015960 15838 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:11.029426 16338 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:11.029546 16141 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.031003 16141 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:11.033555 16360 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.039348 16360 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:11.041752  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.041805 16360 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:11.043516  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:11.044281  7812 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:11.031131  7809 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:29:11.026716 16247 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.004s
I20260709 12:29:11.087805 16247 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.090278 16247 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:11.032701 15882 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.003441 16048 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:29:11.006386 16342 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.094802 16342 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.043970  7550 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:11.060873 16231 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.001s
W20260709 12:29:11.061208  7551 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 271) took 52174 ms (client timeout 15000 ms). Trace:
I20260709 12:29:11.095908  7662 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:29:11.081570 16357 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:11.019539 16094 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:11.046118 16318 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:29:11.020296 16303 raft_consensus.cc:2764] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:29:11.096614  7550 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 6 to 16, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 16 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.101575  8407 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 24. Current term is 25. Ops: [24.20-24.20]
I20260709 12:29:11.101634  7550 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.105274  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.104743  7550 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.107337  7550 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 13 to 18, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 18 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:11.109798  7550 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 24, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 24 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:11.111912  7550 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 15 to 21, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.112998 16338 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.111933 16094 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.113620 16338 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.114477 16094 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.114158  7550 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 13 to 15, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 15 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.117297  7550 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.119575  7550 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 14 to 16, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:11.121965  7550 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 15, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 15 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.124135  7550 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 9 to 16, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 16 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.126242  7550 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 16 to 21, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:11.127568  8405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1212) took 1719 ms (client timeout 1973 ms). Trace:
W20260709 12:29:11.128136  8405 rpcz_store.cc:269] 0709 12:29:09.408343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.408491 (+   148us) service_pool.cc:224] Handling call
0709 12:29:11.127302 (+1718811us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400691ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:11.127479 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485020544}
I20260709 12:29:11.083855  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1563) took 1211 ms. Trace:
I20260709 12:29:11.084964 16202 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.002905 16267 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.133466 16267 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:11.133625 16266 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:11.002867 16027 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 12:29:11.096344  7551 rpcz_store.cc:269] 0709 12:28:18.886716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.887140 (+   424us) service_pool.cc:224] Handling call
0709 12:29:11.060847 (+52173707us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:53.545497 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:53.546343 (+   846us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:53.546412 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:28:53.546450 (+    38us) tablet.cc:662] Decoding operations
0709 12:28:53.753721 (+207271us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:53.753848 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:53.753898 (+    50us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:28:53.980829 (+226931us) tablet.cc:701] Row locks acquired
0709 12:28:53.980890 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:28:53.981125 (+   235us) write_op.cc:270] Start()
0709 12:28:53.981346 (+   221us) write_op.cc:276] Timestamp: P: 1783600133981033 usec, L: 0
0709 12:28:53.981372 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:28:54.069235 (+ 87863us) log.cc:844] Serialized 9334 byte log entry
0709 12:28:54.696205 (+626970us) op_driver.cc:464] REPLICATION: finished
0709 12:28:55.900166 (+1203961us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc005958ef0
0709 12:28:56.397012 (+496846us) write_op.cc:301] APPLY: starting
0709 12:28:56.397130 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:57.253309 (+856179us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:57.253356 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:59.077674 (+1824318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:59.078284 (+   610us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 12:28:59.078340 (+    56us) write_op.cc:312] APPLY: finished
0709 12:28:59.079035 (+   695us) log.cc:844] Serialized 212 byte log entry
0709 12:28:59.594536 (+515501us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.636656 (+ 42120us) write_op.cc:454] Released schema lock
0709 12:28:59.752329 (+115673us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:59.753095 (+   766us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.753145 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":396588,"reported_tablets":49,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":1676411,"apply.run_cpu_time_us":125364,"apply.run_wall_time_us":3356007,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104447,"num_ops":24,"prepare.queue_time_us":284513,"prepare.run_cpu_time_us":59106,"prepare.run_wall_time_us":835324,"raft.queue_time_us":301321,"raft.run_cpu_time_us":29884,"raft.run_wall_time_us":1794971,"replication_time_us":714557,"spinlock_wait_cycles":46243456,"thread_start_us":2965718,"threads_started":5,"wal-append.queue_time_us":841815,"wal-append.run_cpu_time_us":30290,"wal-append.run_wall_time_us":2469310}]]}
I20260709 12:29:11.020834 16351 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:11.135996 16351 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.136535 16351 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:11.091972 16209 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:11.138084 16209 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:11.139143 16209 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.140175  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3683) took 1239 ms. Trace:
I20260709 12:29:11.140661  7805 rpcz_store.cc:276] 0709 12:29:09.901241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:10.157773 (+256532us) service_pool.cc:224] Handling call
0709 12:29:11.139949 (+982176us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54003e9ed4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:11.140101 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600701696}
I20260709 12:29:11.141822  7729 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:11.143697  7737 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:29:11.059414 16125 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:11.067790 16327 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:11.168385 16327 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:11.171204  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.172128  7737 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:29:11.173646  7737 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:11.177142 16352 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:11.177798 16352 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:11.147109 16209 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:11.178953 15882 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.179651 15882 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.178974 16352 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.184134 16338 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.185105  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.185683 16324 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:11.186803  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.205971  7655 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:11.058195 15891 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.131363  7736 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:29:11.131459  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.138341  7806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 3020) took 1418 ms. Trace:
I20260709 12:29:11.146515  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.147770  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.131589  7657 rpcz_store.cc:276] 0709 12:29:09.924550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:10.237617 (+313067us) service_pool.cc:224] Handling call
0709 12:29:10.238298 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:10.458227 (+219929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:11.083479 (+625252us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54003980d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:11.083753 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489963264}
I20260709 12:29:11.150452  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.152199  7656 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:11.213806  7806 rpcz_store.cc:276] 0709 12:29:09.719462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.719638 (+   176us) service_pool.cc:224] Handling call
0709 12:29:09.742085 (+ 22447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:10.853688 (+1111603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:11.138064 (+284376us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003f5054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:11.138263 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2530172288}
I20260709 12:29:11.155081  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:11.218974  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59673.9us
W20260709 12:29:11.212024  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55533.7us
I20260709 12:29:11.220134  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:11.187557  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:11.220980  8380 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:11.221460  8405 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:11.222380  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.224395  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.188158  7732 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:11.189661  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.225665  7661 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:11.191635  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.226815  7654 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:11.182176 15882 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:11.198863 16308 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:11.096961  7662 pending_rounds.cc:85] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 10
I20260709 12:29:11.242038  7662 pending_rounds.cc:107] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:29:11.242375  7662 raft_consensus.cc:2889] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:11.166518 16094 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.228103  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:11.147451  7550 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 13 to 17, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.232282  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.234028 16228 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:11.236189 16353 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:29:11.238517  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92098.7us
I20260709 12:29:11.249331  7657 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:29:11.251781  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.232350  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.252734  7658 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:29:11.197432  7733 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
W20260709 12:29:11.254943  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 3022) took 1534 ms (client timeout 1662 ms). Trace:
I20260709 12:29:11.255266 16352 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:11.254703  7550 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 18 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:11.255525  7809 rpcz_store.cc:269] 0709 12:29:09.720355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.720656 (+   301us) service_pool.cc:224] Handling call
0709 12:29:11.254711 (+1534055us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003f4b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:11.254839 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2864763776}
I20260709 12:29:11.256994  7550 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.259763  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.260552  7653 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:11.245313 16226 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:29:11.249248 16208 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:11.263737 16208 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:11.264207 16208 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:11.268271  7731 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:29:11.240630  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83966.1us
I20260709 12:29:11.269948  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.270380  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.270782  7735 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:11.240790  7732 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:11.271940  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:11.249943  7812 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:11.253000  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:11.267660  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:11.282315  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:11.284246  7661 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:11.272225  7732 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.286506  7732 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:11.285724  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.290285  7661 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:11.218254 16210 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:29:11.246089 15891 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:11.295361  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.245705 16346 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.299454  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.290515  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.303460  7729 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:11.268637 16353 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:11.309687  7808 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:11.305624  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.307256  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:11.310551  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:11.310566  7809 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:11.239557 16366 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:29:11.313709  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:11.315385  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:11.319850  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 164997us
I20260709 12:29:11.321149  7806 raft_consensus.cc:2393] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:11.285877 16251 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.285930 16094 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.331085 16350 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:11.287832 15838 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:11.288915 16338 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:11.309084 16346 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:11.314538  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61634.1us
I20260709 12:29:11.319834  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.323292  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.328603  7810 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:11.330564 16158 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:11.310276 16257 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.339023  7807 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:29:11.332007 15838 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:11.321528  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.336050  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:11.338271  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.347329  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:11.312371 16353 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:11.339607  7653 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:11.340869 16257 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.297741  7550 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 18 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.319376 16342 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.342938 16344 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.324772 16267 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.350230  7809 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:11.359818  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
W20260709 12:29:11.339469  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52681.8us
I20260709 12:29:11.366279  7807 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:11.367264  7732 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:29:11.373675  7732 pending_rounds.cc:85] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:29:11.374221  7732 pending_rounds.cc:107] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:29:11.374742  7732 raft_consensus.cc:2889] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:11.369032  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:11.367300  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53719us
I20260709 12:29:11.376286  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:11.333352 16213 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:11.379504 16158 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:11.380091  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.379742 16351 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:11.445811  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72706.6us
W20260709 12:29:11.380527 16347 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:29:11.380645 16032 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:11.382642 16346 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.352341 16366 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:29:11.343000 16364 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.351466 16064 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:29:11.481129 16351 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.483112 16257 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:11.387459 16170 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:11.387717 16267 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.389748 16360 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:11.391891  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:11.394336  7808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 3023) took 1673 ms (client timeout 1985 ms). Trace:
W20260709 12:29:11.395782  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48617.7us
I20260709 12:29:11.509213  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:11.509440  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.467296 16032 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:11.397451 15838 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.398008 16344 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.362989 16210 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:29:11.398636  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 77258.9us
I20260709 12:29:11.318131 16141 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 12:29:11.400806  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.406241  7655 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:11.396377  7550 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:11.448276  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.449442  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.449759  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.518874 16351 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:11.532473  8415 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:11.449103  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.507485 16360 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:11.468199  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92053.8us
I20260709 12:29:11.520268 16032 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:11.372031 16227 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:11.472698 16364 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.457007 16318 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:29:11.549373 16364 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.473603 16226 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:11.485116  7659 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:11.447476 16213 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:11.552268 16213 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:11.552441 16267 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.478582 16303 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:11.567500 16303 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:11.444970 16361 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.511301  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:11.569651  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58475.6us
I20260709 12:29:11.397359 16375 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:11.533840 16210 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.571579 16375 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:11.571789 16210 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.572297 16375 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:11.572853 16353 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.574378 16210 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:11.512657  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 120876us
I20260709 12:29:11.478701 16308 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:29:11.397830 16342 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.514667  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:11.529176  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.577178  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:11.583556  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68962.7us
I20260709 12:29:11.533457  8407 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:11.586299  7732 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:11.586755  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.575070 16210 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:11.585083  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:11.386885 16350 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:11.593860  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.585700  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:11.596169  7661 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:29:11.597221  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.599259  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.600317  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.519696 16343 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.365521 16357 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:11.537844  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:11.609702  7660 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:29:11.610154  7812 raft_consensus.cc:606] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.610670  7812 raft_consensus.cc:614] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:11.506150  7805 raft_consensus.cc:1217] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Deduplicated request from leader. Original: 15.10->[17.11-19.12]   Dedup: 19.12->[]
I20260709 12:29:11.611316  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.611603  7805 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:29:11.612239  7734 raft_consensus.cc:2393] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:11.544072 16351 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:11.554768 16226 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:11.554925  8415 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:11.542237 16158 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.561316 16327 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.566618  7657 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:29:11.507898  7808 rpcz_store.cc:269] 0709 12:29:09.720664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:09.721864 (+  1200us) service_pool.cc:224] Handling call
0709 12:29:11.394069 (+1672205us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400295c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:11.394250 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3514636672}
I20260709 12:29:11.602422  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.604348  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:11.604677  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.605158  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.550957 16318 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:11.606278  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:11.608287  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.608412  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.612289  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:11.609017  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3450) took 1212 ms. Trace:
I20260709 12:29:11.618651  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.619941  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.620306  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:11.621038  7653 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:29:11.637485  7731 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:11.631126  7662 rpcz_store.cc:276] 0709 12:29:10.397029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:10.881786 (+484757us) service_pool.cc:224] Handling call
0709 12:29:10.882117 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:11.242910 (+360793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:11.608711 (+365801us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:11.608929 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490135296}
I20260709 12:29:11.642783  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.644322  7735 raft_consensus.cc:2410] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 25 because replica has last-logged OpId of term: 24 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 19.
I20260709 12:29:11.588145 16231 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:11.653978  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:11.654567  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 253883us
I20260709 12:29:11.602843 16361 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:29:11.655936  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.657670 16257 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:11.659145  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.608956 16156 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:11.613853 16353 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:11.609514 16267 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:11.661737  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:11.614473 16210 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:11.680917  7735 raft_consensus.cc:2393] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:11.683821  7812 raft_consensus.cc:606] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.697633  7806 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:29:11.695699  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 114241us
I20260709 12:29:11.697633  7809 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:29:11.622166 15916 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:11.703648  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:11.620637 16352 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:11.669855  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.717561  7729 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:29:11.707146  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.661695 16231 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:11.719017  7735 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:29:11.684298  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.711207  7662 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:11.720706  7737 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:11.721401  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.722587  7729 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:29:11.670459  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.716270  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.724103  7734 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:11.671613 16327 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:11.728044  8415 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:11.659830 16226 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.735368  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.690174 16361 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:11.736268  8407 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:11.700639 16267 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:11.702070 16257 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:11.747889 16344 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.699257 16324 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:29:11.755952  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.749234 16210 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.757372  7735 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:11.725320 16231 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:11.730381 15916 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:11.771365  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:11.776479  7809 raft_consensus.cc:606] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.734477 16156 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:11.741621 16361 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:11.779062  7729 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:11.642561 16360 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.745941 16267 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:11.783998  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.794490  7662 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:11.725404 16208 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.690172 16351 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.808581  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.810348  7729 raft_consensus.cc:2376] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25. Re-sending same reply.
I20260709 12:29:11.737197 16350 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.862569  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.766366 16251 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:29:11.863819  7735 raft_consensus.cc:2368] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:29:11.865496  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "629e345c827545598250d02a9f39e2fb"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:11.867455  7809 raft_consensus.cc:606] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:11.873880 16156 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:11.883561 16257 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.885115  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.885497  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:11.886970  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.887465 15916 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:11.880668 16338 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.005s
I20260709 12:29:11.888453  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.889253 16338 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.890698 16208 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:11.890321 16338 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:11.891554 16338 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:11.892195  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:11.892920  7729 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:11.892529 16338 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.893990  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:11.895680 16344 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:29:11.896420 16344 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:11.897890  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.898599  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:11.902931  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 298674us
W20260709 12:29:11.903956 16376 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.427s	user 0.006s	sys 0.000s
W20260709 12:29:11.906739  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.907505  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:11.916436  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:11.929752  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:11.929332  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:11.935276  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 161829us
I20260709 12:29:11.925099  7729 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:11.938899  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.939997  7662 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:11.943907  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:11.944957  8405 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:11.945312  7658 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:11.943897  7662 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:29:11.913852 16364 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:11.948086  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 214261us
W20260709 12:29:11.948236  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 230354us
I20260709 12:29:11.950282  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:11.958125  7662 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:11.969175  8405 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:29:11.924796 16377 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:11.937594 16343 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:11.925625 16342 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:11.973589 16375 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.990444  7812 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:11.997460  7809 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:11.969782 16356 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:11.993268  7812 raft_consensus.cc:606] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:12.006866  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:12.007797 16141 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:12.008930 15882 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:12.009459 15882 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:12.010426 15882 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:11.904211 16361 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.024897 16364 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:29:12.005831 16342 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:29:12.031394 16393 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.028s
W20260709 12:29:12.044691 16381 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.295s	user 0.000s	sys 0.003s
I20260709 12:29:12.043445 16344 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:12.063416  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 220045us
W20260709 12:29:12.047740 16385 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:29:12.094213  7658 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:11.971438 16032 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.053028 16356 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.115622  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:12.129292 16396 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:29:12.131705 16208 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:29:12.135130 16032 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.139853 16387 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:29:12.142445  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 169734us
W20260709 12:29:12.147531  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 161616us
I20260709 12:29:12.125701 16356 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:29:12.157886  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 201213us
W20260709 12:29:12.150542 16213 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.000s
I20260709 12:29:12.126803 16337 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:29:12.155256 16366 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.175453  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:12.180012 16366 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.188122 16360 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:12.190129 16366 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.179369 16352 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:12.195592 16375 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.173674 16032 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.197391 16032 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.194721 16231 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.199939 16032 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.200330  7661 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:12.200719  7659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3453) took 1776 ms. Trace:
I20260709 12:29:12.196626 16352 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.201148  7659 rpcz_store.cc:276] 0709 12:29:10.424003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:11.023563 (+599560us) service_pool.cc:224] Handling call
0709 12:29:11.025908 (+  2345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:11.551856 (+525948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:12.200425 (+648569us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54003999d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:12.200649 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055495424}
I20260709 12:29:12.203784 16231 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.200053 16356 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:12.211158  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52331.9us
I20260709 12:29:12.213275  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:12.228327  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.230911  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.162084 16274 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:29:12.233587  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:12.235050  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:12.243417  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.243654  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:12.244056  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:12.246474  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.236235 16027 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:12.236387 16404 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.257566 16404 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.258620 16356 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.236496 16347 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
W20260709 12:29:12.261984 16395 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.134s	user 0.000s	sys 0.002s
I20260709 12:29:12.224253 16375 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:29:12.234001 16202 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:12.268819 16338 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:12.271646  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:12.279314  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.292930  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:12.164973 16213 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.301364  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.234207 16231 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.310094 16231 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.301146 16094 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:12.316097 16327 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.005s
I20260709 12:29:12.271507 16398 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:12.320992 16407 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:12.332796 16361 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:12.333482 16213 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.341001 16303 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.342779 16303 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:12.343427  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:12.344216  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:12.348631 16208 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:12.349201  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:12.349315 16208 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.353152 16364 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.353824 16364 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.357625 16208 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.354601 16094 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:12.363983  7728 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:12.364413 16364 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.369114  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:12.379424 16344 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.382966  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:12.383900  7733 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:12.386888  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:12.396577  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 95309.8us
I20260709 12:29:12.398084  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:12.403155  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 244326us
W20260709 12:29:12.412345  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 168654us
I20260709 12:29:12.414003  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.391939  7777 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:12.431267  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:12.437373  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.439561  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.440163  7733 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
W20260709 12:29:12.442072  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44119.9us
I20260709 12:29:12.442617  7659 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:12.443125  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.444945  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.446092  7653 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 16.
I20260709 12:29:12.447842  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:12.454864  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:12.456398  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64580.6us
I20260709 12:29:12.479952  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.467550  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:12.469247  7777 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:12.414387 16094 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:12.403932 16347 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.487154  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.490301  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.380086 16344 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.417214 16375 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.452826 16398 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.505105  7654 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:12.506412 16404 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.478775 16383 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 12:29:12.501420 16343 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:12.535564 16383 consensus_queue.cc:1243] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer be9c043c6d424ba1ab8def55dc9bb3e2 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:29:12.464949 16032 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:12.492687  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:12.536180 16383 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:12.513621 16231 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.460673 15838 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:12.502642 16406 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:29:12.460822 16202 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.537817 16406 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:12.538342 16406 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:12.538152 16202 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.493527 16386 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:12.464826 16318 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.478721 16392 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.501695 16377 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:12.479447 16382 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:29:12.494169 16350 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:12.536738 16383 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:12.541707  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49156.4us
I20260709 12:29:12.414827 16327 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:29:12.466418 16405 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:12.545342  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 55175.2us
I20260709 12:29:12.605769  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:12.607631  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.608580  7653 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:12.536289 16032 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:12.613425  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 170406us
I20260709 12:29:12.543617 16094 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:12.554392 16210 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:12.485328 16215 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
I20260709 12:29:12.556396 16274 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:12.548628 16386 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:12.504588 16347 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.612064 16027 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.610213 16356 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.620649  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.598564 16318 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.621064  7659 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:12.628077  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 140601us
W20260709 12:29:12.703944  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83365.5us
I20260709 12:29:12.705188  7627 leader_election.cc:400] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:29:12.705754  7627 leader_election.cc:403] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:29:12.658424  7654 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:12.694499  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.712598  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.620260 16032 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:12.656224 16274 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:12.647866 16386 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:12.616573 16382 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:12.707813 16267 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.876s	user 0.011s	sys 0.004s
I20260709 12:29:12.724911  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:12.623641 16404 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.720968  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:12.619917 16399 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:29:12.548120 16375 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.762887  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.555372 16231 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:12.757120  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:12.648844 16313 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:29:12.715767 16352 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:12.664999 16413 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:29:12.656621 16356 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.657070 16027 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.548520 16398 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.738272  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.756044 16251 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:12.713521 16422 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 17. Current term is 18.
I20260709 12:29:12.751127 16347 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.721849 16213 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:12.778956  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.783856  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:12.615680 16392 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.780341 15993 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.797062 16210 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:12.798144  7728 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:29:12.799829  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:12.805017  7662 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 12:29:12.785702 16411 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 12:29:12.709393 16383 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.780342 16405 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:12.782799 16364 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.813882  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:12.801187 16338 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.004s
I20260709 12:29:12.829340 16431 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:12.790299 16158 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:12.849879 16158 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:12.851266 16158 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.853018 16158 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:12.831838  7653 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:12.800411 16347 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:12.857318 16027 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.857824 16318 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.858078 16274 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.798058 16356 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.785949 16209 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:12.826093 16251 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:12.823026 16257 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.778908 16231 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:12.843587 16416 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:29:12.769719 16404 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.849311 16173 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:12.807435 15993 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.835193 15838 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.811537 16202 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.766235 16267 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:12.895630 16419 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:29:12.801543 16213 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.866189 16344 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.872543 16383 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:12.896785 16392 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:12.869284  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85507.8us
I20260709 12:29:12.873597 16431 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:12.907466 16420 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:29:12.846995 16405 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.908820 16404 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.911137  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3901) took 1297 ms. Trace:
I20260709 12:29:12.911681  7730 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:12.912874  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:12.913074 16267 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:12.913509 16361 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.000s
I20260709 12:29:12.957978 16318 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:12.958652 16324 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:12.958729 16318 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.961274 16318 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.915841 15993 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.862485 16364 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.899890 15838 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:12.977030 16406 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.921521 16213 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.978206 16226 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:12.863555 16382 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.924744 16398 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.925503 15882 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:12.933485 16415 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
W20260709 12:29:12.933851 16407 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.612s	user 0.012s	sys 0.000s
I20260709 12:29:12.937557 16375 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.943696  7732 rpcz_store.cc:276] 0709 12:29:11.613675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:11.614213 (+   538us) service_pool.cc:224] Handling call
0709 12:29:11.614618 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:11.623920 (+  9302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:11.623976 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:12.203414 (+579438us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b2c00001158. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:29:12.204108 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 12:29:12.433921 (+229813us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:12.607306 (+173385us) log.cc:844] Serialized 14 byte log entry
0709 12:29:12.618633 (+ 11327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:12.618718 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:12.910786 (+292068us) raft_consensus.cc:1685] finished
0709 12:29:12.910896 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:12.911068 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19733,"prepare.queue_time_us":315165,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":21978,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":26009,"spinlock_wait_cycles":20928768,"thread_start_us":371275,"threads_started":2,"wal-append.queue_time_us":421047}
I20260709 12:29:12.947141  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:12.941255  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:12.959271 16324 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:12.914340  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:12.966085  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:13.030912  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 116675us
I20260709 12:29:12.967756  7811 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:29:12.969364  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:12.971730  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:12.962742 16141 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:12.973832  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:12.975083 16364 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:12.885636 16257 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:12.951481 16431 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.037282  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:12.982159  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 550818us
W20260709 12:29:13.038245  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 97084.9us
I20260709 12:29:12.988598 15838 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.989048 16406 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:12.988889  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:12.989862  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.046175  8411 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:13.011906 16226 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.003458 16405 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.004431  8407 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:29:13.005798  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.008189 16231 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:12.999712  8379 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:12.904877 16437 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.042519  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.052945  7660 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:13.043125  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.043942  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:12.994582  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.083376  7655 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:13.053838  7809 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:13.034971 16407 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:13.053615  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.055406  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.056252  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:13.056602  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.083550  7657 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:13.086014 16405 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:13.086710  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.088131  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.088275  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.047458  7624 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:13.089509 16361 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:29:13.090329 16361 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:13.094846  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.041306 16342 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:29:13.096972  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49622.6us
I20260709 12:29:13.097407  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.047499 16274 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:13.101125  7729 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:13.098755  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.103467  7655 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:29:13.104172  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:13.103693  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 104138us
I20260709 12:29:13.111253  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.112993  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.113598  7654 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:13.059123 16364 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.116710  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:13.121320  7657 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 20: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:29:13.100713 16156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.007s	sys 0.011s
I20260709 12:29:13.122637  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.099845 16437 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.126469 16437 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.059006 16213 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:29:13.117241  7735 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:29:13.041663 16141 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.118250  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.112490  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:13.106854 16407 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:13.100003 16342 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:13.106976 16274 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.140504  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.148696  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:13.133742 16156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.008s	sys 0.012s
I20260709 12:29:13.152918 16344 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:13.121846 16094 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:13.134552 16361 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.156692  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.165557  7702 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:13.147123 16064 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:13.140580 16357 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.139739 16407 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.150305 16274 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:13.180586 16347 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:13.150018 16437 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.146117 16342 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:13.203744  7657 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:13.206367  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:13.263803  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57542.8us
I20260709 12:29:13.208513 16141 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.266381 16350 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.266592 16210 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.267095 16350 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:13.208823 16439 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:29:13.266206 16377 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:13.268723 16356 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.269428 16356 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.270327 16377 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.209566 16375 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:13.271476 16375 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.271962 16375 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:13.272854 16350 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.210029 16386 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.216742 16226 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:13.216840 16438 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:29:13.130661 16364 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.218439  8406 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:29:13.277881  8406 pending_rounds.cc:85] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 15
I20260709 12:29:13.278225  8406 pending_rounds.cc:107] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:29:13.278510  8406 raft_consensus.cc:2889] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:13.279646 16364 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:13.216070 16441 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:29:13.282402  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.218231 15882 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:13.284382 15882 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:13.285760 15882 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.218686 16361 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:13.224040  7730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3615) took 1616 ms (client timeout 1611 ms). Trace:
I20260709 12:29:13.224797 16170 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:13.287774 16170 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:13.287649  7730 rpcz_store.cc:269] 0709 12:29:11.607951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:11.608087 (+   136us) service_pool.cc:224] Handling call
0709 12:29:13.223689 (+1615602us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002aba54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:13.223923 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2945682688}
I20260709 12:29:13.267324 16210 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.289377 16361 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.219380 16094 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.288466 16170 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:13.291111 16210 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.270973 16377 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.292258 16094 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:13.220127  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:13.293715 16377 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:13.294267  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:13.293844 16342 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.295975  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:13.251904  7662 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:29:13.297097  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.297636  8412 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:13.298240  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.298573  8415 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:29:13.299012  7662 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:13.300143  7806 raft_consensus.cc:2368] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:13.308439  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.313591  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.316445  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.317610  7657 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 16. Current term is 17. Ops: []
I20260709 12:29:13.320075  7803 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:13.327174  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.240636 16407 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:13.244220 15838 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.130183 16324 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.355443  7657 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:13.304163 16435 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.306092 16141 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.268088 16344 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:13.303289 16064 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:13.244638 16357 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.385486  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:13.350140 15882 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:13.350812 16347 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.337185 16428 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:13.336689 16387 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.362922 15971 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:13.304116 16437 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:13.399354  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 286949us
W20260709 12:29:13.387964 16408 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:29:13.406746  8379 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:13.414929  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.389436 16324 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:13.419925  7730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9"
dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:44388
I20260709 12:29:13.422641  7730 raft_consensus.cc:606] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:13.423063  7730 raft_consensus.cc:614] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:13.395193 16257 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.368713 15838 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.379330 16141 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.381748 16344 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.434593  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:13.437669  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 392621us
I20260709 12:29:13.438325  7807 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:13.426280 16366 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:29:13.429168 16158 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.576s	user 0.015s	sys 0.003s
I20260709 12:29:13.442221  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.422658 16356 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.427714 16431 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.377121 16435 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.399923 16347 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.451962  7812 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:29:13.405740 16387 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.407655 16428 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.462508  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.462675  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.469414  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:13.467152  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:13.409160 15971 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.473390 16428 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.473934 15971 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.475121  7728 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:29:13.476629 16437 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.477826 16344 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.479907 16435 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.448307 16342 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.484481  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.485170 16357 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.486163  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.486465  7808 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:29:13.449414 16064 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:13.491904 16443 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:29:13.492408  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.454149 16366 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:13.495114 16342 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:13.496255 16366 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.505182  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.510066  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.511528  7654 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:13.512493  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 27.
I20260709 12:29:13.508023  7812 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:29:13.516223  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:13.524005  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.524560  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.479101 16170 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.532408 16170 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.533277 16170 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.533824 16170 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.535645  8411 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:13.536496 16444 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
I20260709 12:29:13.536711 16141 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.539009 15838 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.544421 16125 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:13.545059 16125 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:13.546451 16347 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.546173 16125 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.547663 16125 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:13.525838 16453 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.549002  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:13.517603 16342 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:13.527114 16451 log.cc:927] Time spent T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:29:13.548825 16453 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.553678 16453 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.526012 16356 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:13.554430 16356 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.554908 16356 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:13.556156  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.558463  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.559437  7808 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:13.562314 16375 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.564993 16375 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:13.569065  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:13.523301  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:13.576129  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52929.6us
I20260709 12:29:13.577975  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.578578  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.579205  7812 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 28: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:13.581401  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.581856  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:13.582181  7659 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:13.582433  7656 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:13.590070  7657 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:29:13.592270  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:13.592774  7807 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:13.592880  7700 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.593644  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.629514  8413 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:13.630584  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.595458  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.631904  7657 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:13.615844  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:13.599833  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.636117  8415 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:29:13.597452  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:13.637691  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 44922.3us
I20260709 12:29:13.639644  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.640300  7728 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 17. Current term is 18. Ops: []
W20260709 12:29:13.688174  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 53797.8us
I20260709 12:29:13.688884  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.647754  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.665439  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.648908  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.692730  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.693681  7809 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:13.689963  7805 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
W20260709 12:29:13.592491 16032 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.813s	user 0.011s	sys 0.003s
I20260709 12:29:13.589632 16327 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.627521 16342 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:13.700475  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.591986 16453 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:13.621454 16375 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:13.520955 16435 consensus_queue.cc:1059] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.167s
I20260709 12:29:13.628847 16343 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:13.726352  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 236211us
I20260709 12:29:13.728526  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:13.721889 16375 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.738833 16375 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:13.718894 16032 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.742108 16032 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:13.719398 16342 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.745044 16170 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.745644 16342 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.722159 16327 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.722453 16021 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:29:13.748742 16327 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:13.749598 16421 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 1.096s	user 0.004s	sys 0.000s
I20260709 12:29:13.750854 16347 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:29:13.751417 16347 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:13.752396 16347 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.753829 16347 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:29:13.756613  7810 raft_consensus.cc:2368] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
W20260709 12:29:13.758013 16274 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 12:29:13.758337 16407 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.762782 16386 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:13.646736 16210 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:13.647637 16344 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:13.764606 16210 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:13.765288 16407 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.758548 16274 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.765502 16210 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:13.766691  7808 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:13.769043 16274 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.730347  7804 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:13.690115 16424 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:13.735653 16343 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.760159 16387 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:13.727387 16453 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:13.762725  7803 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:13.785300  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.787549  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:13.787096  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.771818  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.789172  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.773080  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.776350  7809 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: [23.17-26.18]
I20260709 12:29:13.720820  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.777765  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.779045  7806 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 27. Current term is 28. Ops: []
W20260709 12:29:13.737259  7811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.003s	sys 0.012s
W20260709 12:29:13.793054  7811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.003s	sys 0.012s
I20260709 12:29:13.769906  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.794377  8407 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:13.794721  8412 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:13.806094  8408 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:29:13.777386 16424 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.810974  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.810575  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:13.811966  8413 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:13.814278  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.813202  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:13.810463  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 82078.6us
I20260709 12:29:13.823824  8409 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:29:13.751121 16356 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:13.813877 16424 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:13.836908  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.833356  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.828529  8413 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:13.838939  8409 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:13.840938  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:13.844707  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:13.850293  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:13.827816 16469 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:29:13.838939 16360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.009s	sys 0.015s
I20260709 12:29:13.854451 16469 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:13.849105 16318 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
W20260709 12:29:13.860709 16360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.009s	sys 0.015s
I20260709 12:29:13.871414 16343 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.868789 16469 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:13.882958 16453 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:13.924330  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:29:13.934528  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 338839us
I20260709 12:29:13.935957  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.938256  7655 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:13.941222  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:13.942417  7655 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:13.945606  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:13.951853  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:13.955377  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:13.641781  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:13.960722  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 318901us
I20260709 12:29:13.962617  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:13.966463  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:13.957489  7660 raft_consensus.cc:2368] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26.
I20260709 12:29:13.973647  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:13.975471  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 258522us
I20260709 12:29:13.946825  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:13.985126  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45923.9us
I20260709 12:29:13.989710  7736 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:13.988698  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:14.002751  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:14.004784  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.009560  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:13.850355  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:14.014156  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 163722us
I20260709 12:29:14.015712  7730 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:14.019420  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:14.021453  8379 leader_election.cc:400] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26.
I20260709 12:29:13.997946  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:14.022403  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:14.024260  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:14.026247  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:14.008325  7736 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:14.028193  8379 leader_election.cc:403] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:29:14.021776  7702 leader_election.cc:400] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:14.057082  7702 leader_election.cc:403] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:29:14.032258  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:14.029327  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1229) took 1083 ms. Trace:
W20260709 12:29:14.079656  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56608.3us
I20260709 12:29:14.030395  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:14.031343  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:14.085208 16346 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:14.085914 16267 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:14.087628  7728 raft_consensus.cc:3060] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:14.090771  8415 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:14.091887 16156 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:29:14.032734 15838 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:14.038316 15882 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.099606 16343 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.103667  7732 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:14.049463 15971 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:29:14.104893 16356 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.105862 16344 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:14.107846 16366 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.046330 15916 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:29:14.117297  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.119333 16170 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:14.110836 16382 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:14.085958 16346 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.112615 15838 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.133745 16449 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.145045 15838 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:14.145500 16449 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.129055 16382 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.078514  8405 rpcz_store.cc:276] 0709 12:29:12.945243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:12.945419 (+   176us) service_pool.cc:224] Handling call
0709 12:29:12.945886 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:12.963518 (+ 17632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:12.963556 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:12.964085 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:29:13.554027 (+589942us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800463cc0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:13.770802 (+216775us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:13.770859 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:13.770933 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:14.028613 (+257680us) raft_consensus.cc:1685] finished
0709 12:29:14.028651 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.029000 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1353,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":21513,"spinlock_wait_cycles":24954240,"thread_start_us":216539,"threads_started":1,"wal-append.queue_time_us":852702}
I20260709 12:29:14.136561 16346 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:14.137535 16366 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.137610 15882 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:14.139269  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 118212us
I20260709 12:29:14.039479 15891 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.114s
I20260709 12:29:14.161293  7737 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:14.154807 16382 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:14.161357  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63478.3us
I20260709 12:29:14.168884 16448 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.172724 16448 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:14.140847  7627 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.611s (SENT)
I20260709 12:29:14.142074 16170 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:14.142976 16208 raft_consensus.cc:993] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:29:14.143503 16343 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.152240  7728 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:14.199383  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:14.208189  7657 consensus_meta.cc:287] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.011s
I20260709 12:29:14.208868  7657 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:14.203938 16267 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:14.212918 16343 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.200657  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:14.216909 15882 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.214483  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:14.227276 16357 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:29:14.229223  7626 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:14.229883 16357 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:29:14.224148  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 226420us
I20260709 12:29:14.160207 16469 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.213881 16231 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.214910 16448 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.244807  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:14.245338  8415 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:14.250936 16226 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:14.251530 16437 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:14.253216 16351 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:14.253573 16437 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.255720 15891 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:14.259130 16125 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.259541 16208 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:14.260169 16125 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.261317 16469 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:14.263185 16449 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.264353 16021 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:14.265607 16366 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.266057 15916 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:14.266494  7730 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:14.266682 15916 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:14.267580  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3912) took 1436 ms. Trace:
W20260709 12:29:14.268956  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 113374us
I20260709 12:29:14.252425 16407 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:14.285326  7736 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:14.290572  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 150040us
W20260709 12:29:14.291712  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.292806  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:14.296128  7728 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:14.268168  7734 rpcz_store.cc:276] 0709 12:29:12.730446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:12.730613 (+   167us) service_pool.cc:224] Handling call
0709 12:29:12.731075 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:12.785601 (+ 54526us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:29:12.879159 (+ 93558us) log.cc:844] Serialized 14 byte log entry
0709 12:29:13.365334 (+486175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:13.365398 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:13.365455 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.166053 (+800598us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b1800450700. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419
0709 12:29:14.166757 (+   704us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54000726d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:14.166808 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.167109 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14764,"prepare.queue_time_us":11486,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":8088,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":6572,"spinlock_wait_cycles":129237632,"thread_start_us":563808,"threads_started":2,"wal-append.queue_time_us":527573}
W20260709 12:29:14.298518  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 277031us
W20260709 12:29:14.259037 16375 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.005s
I20260709 12:29:14.300410  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:14.300079 16375 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.276995 16158 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:29:14.303287 16375 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.251729 16226 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:14.305213 16226 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:14.305814  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1686) took 1010 ms. Trace:
I20260709 12:29:14.306336  8411 rpcz_store.cc:276] 0709 12:29:13.295714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.296804 (+  1090us) service_pool.cc:224] Handling call
0709 12:29:14.305348 (+1008544us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54007a0154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:14.305543 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546735360}
I20260709 12:29:14.269186 16231 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:14.307289  7661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.010s	sys 0.020s
W20260709 12:29:14.307873  7661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.010s	sys 0.021s
I20260709 12:29:14.309799 16231 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.311962 16125 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.298684  8405 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:29:14.258844 16227 raft_consensus.cc:1081] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:29:14.268188 15916 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.319497 15916 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:14.304450  7735 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:14.322722  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:14.324467  7737 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:29:14.328084  7728 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:29:14.330624  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 3050) took 1275 ms. Trace:
I20260709 12:29:14.331215  7810 rpcz_store.cc:276] 0709 12:29:13.055440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.055966 (+   526us) service_pool.cc:224] Handling call
0709 12:29:14.330136 (+1274170us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54003f4b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:14.330403 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542527360}
I20260709 12:29:14.335855  7662 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:29:14.339303 16437 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.341151  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:14.341670  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:14.343492  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:14.347759  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3910) took 1622 ms. Trace:
W20260709 12:29:14.350630 16210 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.012s
I20260709 12:29:14.351786  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:14.341238 16347 consensus_meta.cc:287] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.587s	user 0.016s	sys 0.008s
W20260709 12:29:14.353518  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60895.5us
I20260709 12:29:14.348693  7731 rpcz_store.cc:276] 0709 12:29:12.725170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:12.728261 (+  3091us) service_pool.cc:224] Handling call
0709 12:29:12.728866 (+   605us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:13.225300 (+496434us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:29:13.225660 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:29:13.556349 (+330689us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b18003f55c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:13.589171 (+ 32822us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:13.590007 (+   836us) log.cc:844] Serialized 52 byte log entry
0709 12:29:13.590230 (+   223us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:13.590540 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 12:29:13.596819 (+  6279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.346797 (+749978us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400211e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:14.346906 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:14.347015 (+   109us) raft_consensus.cc:1685] finished
0709 12:29:14.347036 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.347476 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5834,"prepare.queue_time_us":15357,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":4919,"raft.queue_time_us":8387,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":10684,"spinlock_wait_cycles":1148038784,"thread_start_us":363117,"threads_started":1,"wal-append.queue_time_us":510018}
I20260709 12:29:14.356041 15882 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:14.358477 16407 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:14.359639 16383 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.360093  7658 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:29:14.361049  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.361964  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:14.362763 16357 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:14.366868  7735 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:29:14.368124  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:14.324672  8406 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:14.310886  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1580) took 1870 ms. Trace:
I20260709 12:29:14.374178  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:14.374643 16375 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:14.378118 16424 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.004s
I20260709 12:29:14.379207  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:14.390641  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.390103  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.412115 16407 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.401535 16210 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.412173 16383 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.415728 16125 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:14.422111 16231 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:14.404076  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:14.422751  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60956.1us
I20260709 12:29:14.388060  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:14.386458  7653 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:29:14.416043  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64505.1us
I20260709 12:29:14.384528 15838 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.402616  7661 rpcz_store.cc:276] 0709 12:29:12.440168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:12.440340 (+   172us) service_pool.cc:224] Handling call
0709 12:29:12.961230 (+520890us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b5800160898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:29:12.961479 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:13.129442 (+167963us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:29:13.156533 (+ 27091us) log.cc:844] Serialized 14 byte log entry
0709 12:29:14.308685 (+1152152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:14.308814 (+   129us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:14.308953 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.310478 (+  1525us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54004353d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:14.310527 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.310771 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34459,"prepare.queue_time_us":27659,"prepare.run_cpu_time_us":382,"prepare.run_wall_time_us":2933,"raft.queue_time_us":2343,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":188,"spinlock_wait_cycles":404387968,"thread_start_us":1151312,"threads_started":1,"wal-append.queue_time_us":1090947}
W20260709 12:29:14.462106  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.452947  7734 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:14.416425 16375 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:14.444809 16125 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:14.445773  7655 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:14.398017 15882 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:14.489979  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86082.5us
I20260709 12:29:14.417083 16170 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26.
W20260709 12:29:14.493009  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 66692.5us
I20260709 12:29:14.476660 16231 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.395771 16448 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.093s
I20260709 12:29:14.491246 16375 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:14.447158 16210 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.510841  7732 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:14.463958 16324 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.399572 16437 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:14.489789 16125 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:14.400077 16346 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.512923  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:14.495337 15882 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:14.528532  7728 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:14.540304  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:14.446075 16382 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.496631 16366 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.513186 16231 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:14.476862 16424 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.549377  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:14.445008 16407 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.512445 16267 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.513887 15891 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:14.571708 15891 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.573764 16382 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:14.568322  7656 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.926s	user 0.023s	sys 0.007s
I20260709 12:29:14.578668  7735 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 16. (term mismatch)
W20260709 12:29:14.562155  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48278.1us
I20260709 12:29:14.568511 16461 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:14.569317 16360 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:14.557133 16351 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:14.604808 16346 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.579473  7735 pending_rounds.cc:85] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 15
I20260709 12:29:14.605378 16351 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.592415 15838 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:14.595295  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:14.605677  7735 pending_rounds.cc:107] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 16.16
I20260709 12:29:14.596019 16398 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:14.606468  7735 raft_consensus.cc:2889] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:14.598119  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 99595.8us
I20260709 12:29:14.598786 16468 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 12:29:14.600544 16383 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.599587 16226 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.603468 16486 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:29:14.604159  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91363.9us
I20260709 12:29:14.558729 16366 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.578058  7737 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:29:14.565008 16437 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.605939 16351 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:14.597580 16407 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.586876 16324 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.614620  8415 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:14.584755 16360 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.512727 16449 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 12:29:14.619978  8411 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:14.605664  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:14.617986  7656 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:14.633313  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:14.615705 16398 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.635165 16324 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.636677 16461 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.637394 16226 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:14.650743  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45413.4us
I20260709 12:29:14.638947  7778 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:14.616258 16382 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
W20260709 12:29:14.665445  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 159481us
I20260709 12:29:14.638475 16360 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:14.649971 16398 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:14.615978 16267 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.664327 16170 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:14.651239 16437 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.682803  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3247) took 1092 ms. Trace:
I20260709 12:29:14.648523 16407 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:29:14.691967  7656 rpcz_store.cc:276] 0709 12:29:13.581337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.581597 (+   260us) service_pool.cc:224] Handling call
0709 12:29:14.665127 (+1083530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.685600  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:14.688149  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:14.695972  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74174.9us
I20260709 12:29:14.628159 15916 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:29:14.670432 16486 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.703181  7728 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
W20260709 12:29:14.704296  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 71169.6us
I20260709 12:29:14.705816  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:14.711315  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.675812 16346 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:14.707585  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.669631 16324 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:29:14.684998 16170 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:14.725837  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:14.729744  7661 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:14.703918 15838 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.705062 16267 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.728116 16170 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:14.741696  7655 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:29:14.733191 16227 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:14.745200 16377 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:14.761052  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55347.8us
I20260709 12:29:14.732569 15916 consensus_queue.cc:1243] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:29:14.737241 16226 raft_consensus.cc:2804] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:14.745414 16210 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:14.766949  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:14.786413  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48375.5us
I20260709 12:29:14.754912 16366 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.789618  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:14.775367 16357 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.763516 16424 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:14.786149  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:14.805115  8411 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:29:14.807384 16377 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.810227 16377 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.811444 16357 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.815604 16461 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:14.817674  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.817986  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:14.818652  7805 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:14.819869  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.802343  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.822302 15838 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.822779  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:14.787397 16226 raft_consensus.cc:697] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:14.823975  7811 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:14.824730 16226 consensus_queue.cc:237] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.825766  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:14.827735  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 76956.8us
I20260709 12:29:14.828946  7700 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:14.830808 16491 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.197s	user 0.000s	sys 0.003s
I20260709 12:29:14.831126  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:14.849282  7735 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 17.
I20260709 12:29:14.856272  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.860697  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:14.819897  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:14.865200  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102881us
I20260709 12:29:14.874547  7657 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:14.835328 16485 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260709 12:29:14.900873  7661 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:14.909106  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:14.910184  7660 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:14.852896 16210 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:14.856420 16437 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:14.801198 16366 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 12:29:14.938827  7660 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:14.926281  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:14.942760 16404 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.943660 16404 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:14.948560 16493 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:29:14.949838  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:14.951445  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 120470us
I20260709 12:29:14.953348  7700 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:14.926452 16210 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.901522 16377 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.955610 16357 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.956291 16377 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.957441  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:14.958144  7656 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:14.932160 16267 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:14.960614 16267 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.961030 16267 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:14.963379 16208 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:14.965094 16208 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:14.965636 16208 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:14.966140  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3918) took 1326 ms. Trace:
I20260709 12:29:14.969236 16227 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.970113 16227 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.970527  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.971628  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:14.972437 16346 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.973150  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.973170 16346 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.976397  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:14.957441  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 190573us
W20260709 12:29:14.978745 16488 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.344s	user 0.001s	sys 0.001s
I20260709 12:29:14.979172 16461 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:14.980074 16461 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.980559 16461 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:14.983829  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:14.982872  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 144904us
I20260709 12:29:14.984702  7810 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:14.923336 16407 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:14.787345 16486 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:14.995409 16407 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:14.982784 16413 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:15.003146  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:14.980742  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.946866 16404 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:14.966709  7729 rpcz_store.cc:276] 0709 12:29:13.639253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.640527 (+  1274us) service_pool.cc:224] Handling call
0709 12:29:13.640931 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:13.643129 (+  2198us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:29:13.643168 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:13.643817 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 12:29:14.345179 (+701362us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b180046a620. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:14.705141 (+359962us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:14.705245 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.705409 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:14.965671 (+260262us) raft_consensus.cc:1685] finished
0709 12:29:14.965693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.965943 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1678,"raft.queue_time_us":6572,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":32272,"spinlock_wait_cycles":3999488,"thread_start_us":274582,"threads_started":1,"wal-append.queue_time_us":1082926}
I20260709 12:29:14.958765 16377 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.000373 16210 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:15.013170 16413 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:15.040791  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.018011 16357 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.047894  7811 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:15.040900  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:14.966688 16208 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.054221 16492 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.021412 16404 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.036018 16377 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:15.050606 15838 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.069721 16227 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:15.087822 16490 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.312s	user 0.002s	sys 0.000s
I20260709 12:29:15.072875 16398 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:15.084018 15882 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.009s
I20260709 12:29:15.089545 15882 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:15.090287 16375 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:29:15.090878 16398 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.092042  7653 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
W20260709 12:29:15.092024 16125 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.002s
I20260709 12:29:15.055382 16492 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.093107 16125 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.095449 16125 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.096876  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:15.098533 16468 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:15.077754 16448 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:29:15.100126  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:15.100340 16407 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.102305 16486 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.104067 16398 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.104326  8406 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:15.106132 16437 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.106940 16437 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.104823 16398 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.115708  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.121646  7657 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:15.110824 15838 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.081040 16346 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.166630  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.090803 16375 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.164004  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.092355 15882 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.176903  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.071091 16357 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.199338  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 431426us
I20260709 12:29:15.149976 16448 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:29:15.216791  7661 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:15.218786  7658 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:15.153537 16486 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.082142 16404 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.154418 16407 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:15.228031  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59221.5us
I20260709 12:29:15.229966  7731 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:15.231170 16021 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.231424 16448 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:15.232054  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.256047  7659 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:15.257239  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.235342  7657 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:15.259656  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.259492 16021 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.262048 16021 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:15.262286 16499 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.262938 16499 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.265322 16499 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.250468  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:15.236817  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.218120 15838 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.218372 16413 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.245754 16231 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.666s	user 0.012s	sys 0.003s
I20260709 12:29:15.200399 16346 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.247995 16375 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:15.296494  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46112.5us
I20260709 12:29:15.270373  7653 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:15.283803  7626 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.854s (SENT) [suppressed 7 similar messages]
I20260709 12:29:15.288273  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.271624  8408 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:15.284636  7659 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:15.297704  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:15.308295  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.309290  7657 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:15.313478 16448 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:15.314291 16448 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.314739 16448 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:15.315706 16413 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:15.311563 16324 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:15.343158 16404 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.344017 16404 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.344240 15838 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.290418 16486 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.348379  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 319327us
I20260709 12:29:15.290125 16231 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.306874 16346 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.362779  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:15.326361 16375 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:15.352566 16351 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.003s
W20260709 12:29:15.375173  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:15.337826 16492 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:15.360149 16347 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.015s	sys 0.016s
I20260709 12:29:15.330358 16021 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
W20260709 12:29:15.378471  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 425183us
I20260709 12:29:15.325709 16407 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.357453 16324 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:15.336125 16461 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.361698 16377 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:15.344707 16404 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.432040  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60539.7us
I20260709 12:29:15.416234  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1839) took 1019 ms. Trace:
I20260709 12:29:15.361456 16125 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:15.431401  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:15.450340  7731 rpcz_store.cc:276] 0709 12:29:14.396700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.508496 (+111796us) service_pool.cc:224] Handling call
0709 12:29:14.541545 (+ 33049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:15.255747 (+714202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:15.415659 (+159912us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54001c48d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:15.416059 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582514560}
W20260709 12:29:15.380985 16476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.009s	sys 0.021s
W20260709 12:29:15.451280  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:15.381985 16347 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.016s	sys 0.016s
W20260709 12:29:15.389756 16170 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.003s
I20260709 12:29:15.463166  7737 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:29:15.381572 15916 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:15.458256 16476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.010s	sys 0.021s
I20260709 12:29:15.451772 16469 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:15.382330 16375 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.379668 16351 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.472927 16173 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 12:29:15.473229 15916 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.474429  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:15.473253 16125 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.474535  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.478021 16469 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:15.480922  7775 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:15.482128 16383 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:15.482887 16375 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.485643 16351 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.485827 16508 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:15.486547  7656 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:29:15.470348  7777 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:15.488488 16324 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.489509 16486 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.489684 16125 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.490512 16125 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:15.366216  7658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "629e345c827545598250d02a9f39e2fb"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:29:15.492640 16461 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:15.490972  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:15.495971  7729 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:29:15.494278 16469 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.501909 16407 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.502152 16500 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.505112 16503 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.505885 16503 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.503832 16021 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:15.508442 16346 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.503610 16383 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:15.510604  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.511482  7731 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:15.511845  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.512600 16231 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.512948  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.513600  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.516224  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.518669  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.519717 16156 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:15.521478 16156 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:15.521920 16156 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:15.501912 16267 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.001s
I20260709 12:29:15.502979 16500 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.471884 16170 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.523196 16267 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.527609  8412 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:29:15.528532 16469 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:15.529209 16170 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.529460 16267 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.524127  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:15.526348  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51998us
I20260709 12:29:15.525115  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.479733  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:15.519754  7728 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:15.532392  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:15.485915 16424 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:15.542577  7729 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:15.491074 16125 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:15.546118 16503 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.550715  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.552047  7737 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:15.525899 16500 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.523072 16156 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.542855 16170 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:15.551136 16125 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:15.559814  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.575176  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:15.648542  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 448107us
I20260709 12:29:15.571614  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.649672  7728 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:15.650511  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:15.565680  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.610684  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:15.615861  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 332122us
I20260709 12:29:15.567862  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.564970  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.655544  7729 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:15.575973  8411 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:29:15.577689  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.657437  8406 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:15.553699  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 74202.3us
I20260709 12:29:15.657738  7662 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:15.660912  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.661715  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.654482  7737 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:15.578298  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:15.583998 16213 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:15.550004 16267 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.612164 16208 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:15.553517 16424 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:15.580412 16357 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.572080 16377 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:15.564692 15916 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.688735  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.554255 16346 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.576671 16398 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.639901 16170 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:15.718106 15882 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:15.677716 16213 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:15.732921  7662 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:15.734598  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:15.740984  8408 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:15.693418 16500 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.745083  7658 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:15.745726  7658 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:15.714722 16156 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:15.715037 16404 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.748646  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:15.770185  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:15.750583  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.703434 16503 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.748483  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.759122  7728 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:15.721624 16170 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:15.693337 16227 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:29:15.722981 16437 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:15.762995  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 284626us
I20260709 12:29:15.720001 16375 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.786727  7775 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:15.780278  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49126.9us
W20260709 12:29:15.749763 16210 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.004s
I20260709 12:29:15.713049 16267 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.720945 16514 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:29:15.820210  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:15.816843  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.783001 16357 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:15.835420  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 48818.5us
I20260709 12:29:15.837028  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:15.822472  7735 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:15.841751  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:15.824447  8406 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:15.831907  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:15.847959  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 109599us
W20260709 12:29:15.860538  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 52948.1us
I20260709 12:29:15.863695  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:15.730053 16346 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.875521  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:15.862596 16508 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:15.881855 16504 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:15.882786 16504 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:15.884047 16504 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.886729  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:15.887809  7736 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:15.873284  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:15.877995  7731 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:15.823961 16514 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.894081  7735 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:15.785272 16449 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.087s
I20260709 12:29:15.859393 16347 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:15.915715  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:15.866098 16351 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:15.899812 16413 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.003s
I20260709 12:29:15.919970  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:15.920485  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 47307.5us
I20260709 12:29:15.920739  7736 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:15.922896  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:15.923298 16508 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:15.924011  8407 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:15.925037  7807 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26. Ops: [21.13-25.15]
I20260709 12:29:15.927363  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.929937 16346 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.932191 16267 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.933979  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.937100  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:15.938330  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.886350 15838 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.061s
I20260709 12:29:15.939756  7733 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:15.942011  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.826264 16210 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.900974 16514 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:15.949671  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.950268 16210 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:15.913965 16424 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.952839  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:15.939792 15838 consensus_queue.cc:1243] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 26.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:29:15.799374 16500 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.814419 16375 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:15.819818 16503 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:15.976512  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:15.864756 16357 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:15.922194 16360 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.208s	user 0.017s	sys 0.001s
I20260709 12:29:15.985458  7736 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 12:29:15.989183  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 265939us
W20260709 12:29:16.005053  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:16.031369  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:16.032294  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54486.5us
I20260709 12:29:15.966398 16231 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:16.040550  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.043120  7736 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:16.038936  7803 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:15.961519 16377 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:16.062942  7653 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.003s
I20260709 12:29:16.063433  7653 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:16.037973  7729 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 17.
I20260709 12:29:15.927484 16226 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:29:16.070457  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:16.071802 16512 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:29:16.048354 16404 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.075661 16404 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:16.077005 16530 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.100s	user 0.000s	sys 0.002s
I20260709 12:29:16.078276 16404 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.072868 16461 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:16.082727 16357 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.090212  7659 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: [22.15-24.16]
I20260709 12:29:16.092724  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.093706  7736 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 17.
I20260709 12:29:16.093748  7659 raft_consensus.cc:1275] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:29:15.971338 16347 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.104542  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:16.105221 16469 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.002s
I20260709 12:29:16.106546 16503 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:15.976555 16351 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.115913  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:16.030534 16520 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.230s
I20260709 12:29:16.128878  7731 raft_consensus.cc:2393] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:16.099161 16448 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.784s	user 0.013s	sys 0.003s
I20260709 12:29:16.142876  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.143736  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:15.988713 16360 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:16.115383 16524 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:29:16.157701  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:16.102221 16231 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:16.184432  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 68456.1us
I20260709 12:29:16.188956  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:16.189365  7810 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:29:16.189909  7736 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:16.199144  8414 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:16.199915  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.201540  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.163604 16173 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:16.199359  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:16.233088  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:16.240149  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51179.3us
I20260709 12:29:16.150005 16448 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.263048  7731 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:29:16.228634  7733 raft_consensus.cc:2410] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:29:16.265094  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.209263 16468 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:16.203294 16532 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:29:16.282558 16529 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:29:16.210398 16173 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:16.291266  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.293643  8415 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:16.295439  7731 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:16.298599  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.296772  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:16.278666 16125 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
I20260709 12:29:16.309154  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:16.310243 16324 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.310966 16324 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.311386 16545 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:16.312168 16545 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:16.313145  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:16.313545 16324 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:16.314960 16536 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:29:16.321913  8406 pending_rounds.cc:85] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 17
I20260709 12:29:16.322376  8406 pending_rounds.cc:107] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:29:16.322647  8406 raft_consensus.cc:2889] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:16.293721 16522 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.198s	user 0.003s	sys 0.000s
I20260709 12:29:16.276041 16468 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:16.311604  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:16.285820 16532 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:16.360937  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 511347us
I20260709 12:29:16.361938  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:16.369499  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.370518  7736 raft_consensus.cc:2410] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:29:16.373716  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:16.375731  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 74858.8us
I20260709 12:29:16.305655 16413 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.297734 15916 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:16.375126  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:16.377876  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.390249  7731 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:16.328162 16545 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.307135 16503 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:16.388844  7655 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:16.376717  7654 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:16.393759 16508 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:16.396420  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 84916.9us
I20260709 12:29:16.409731  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:16.338958 15838 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:16.410521  8409 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:16.398805  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.378495 16324 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:16.412945  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.266901 16377 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.378784 16375 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:16.413903  7731 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:16.422713 16532 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.424038  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.424178 16532 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:16.428094  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:16.321681 16125 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.434201  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.380329 16538 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:16.380522 15891 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:16.390756 16500 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.448204  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:16.452059  7731 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:16.458348  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 113602us
I20260709 12:29:16.408525 16503 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.409837 16413 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.484793  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.489900  7809 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:16.492093  7806 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:16.504812  7658 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.752s	user 0.013s	sys 0.000s
I20260709 12:29:16.438705 16538 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:16.437371 15838 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.454684 16347 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.464193 16324 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:16.517961  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:16.420742 16508 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:16.504103 16539 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:29:16.505455  7658 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.381570 16231 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.540259  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:16.532589  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80612.4us
I20260709 12:29:16.532102  7810 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:29:16.494325 16503 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:16.484681 15891 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.506465 15916 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:16.568862  8412 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:16.566429  8411 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
W20260709 12:29:16.531673 16547 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:29:16.541545 16360 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.515698 16538 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:16.632057 16508 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:16.530606 16526 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.326s	user 0.002s	sys 0.000s
I20260709 12:29:16.641006  7806 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:16.667593  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:16.631212  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51835.4us
W20260709 12:29:16.630432 16551 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:29:16.667676 16156 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.689664  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:16.629155 16541 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:29:16.513656 16351 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.692696  7728 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:16.521952 16500 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.694841  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:16.536158 15838 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:16.704427  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:16.632071 16413 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:16.707376 16535 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.009s
I20260709 12:29:16.522433 16173 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.708436 16531 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:16.719329 16173 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:16.720063  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:16.720288 16125 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:16.723543 15916 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.724550 16516 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:16.726135  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1549) took 1151 ms. Trace:
I20260709 12:29:16.726625  8415 rpcz_store.cc:276] 0709 12:29:15.574474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.577945 (+  3471us) service_pool.cc:224] Handling call
0709 12:29:16.725865 (+1147920us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b540059ecd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:16.726037 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435889536}
I20260709 12:29:16.728044  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:16.643956 16558 consensus_queue.cc:1048] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:29:16.729555  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:16.729808  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:16.730993 15838 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:16.709414 16375 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:16.731494 15838 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:16.732095 15838 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:16.706664 16500 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.698634 16213 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.734827 16213 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:16.707631 16156 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:16.743484 16553 consensus_queue.cc:1175] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" responder_term: 25 status { last_received { term: 21 index: 14 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: [22.15-24.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:29:16.749097  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1610) took 1482 ms. Trace:
I20260709 12:29:16.749670  7656 rpcz_store.cc:276] 0709 12:29:15.266367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.613209 (+346842us) service_pool.cc:224] Handling call
0709 12:29:15.613796 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:15.614401 (+   605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:15.614447 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:15.615153 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 12:29:15.712052 (+ 96899us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:15.712118 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:15.712226 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:16.748745 (+1036519us) raft_consensus.cc:1685] finished
0709 12:29:16.748778 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:16.749006 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":911,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"thread_start_us":84194,"threads_started":1,"wal-append.queue_time_us":289656}
W20260709 12:29:16.752140 16560 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:29:16.754892  8406 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:29:16.654724  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:16.773294  7803 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:16.765126  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:16.776549  8409 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:16.777575  8406 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:29:16.193800  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:16.786405  8407 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 12:29:16.786859  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 247338us
I20260709 12:29:16.795806  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:16.798329  8415 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:16.801162 16543 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.801789 15891 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.801987 16543 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.806964  8415 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:29:16.788231 16448 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:16.797887 16170 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.005s
W20260709 12:29:16.847437  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 138887us
W20260709 12:29:16.852378 16514 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.904s	user 0.017s	sys 0.001s
W20260709 12:29:16.853061 16549 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.314s	user 0.003s	sys 0.000s
I20260709 12:29:16.853634  7702 leader_election.cc:400] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:16.854341  7702 leader_election.cc:403] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:29:16.911891  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:16.912169  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:16.945631  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 180569us
I20260709 12:29:16.910962 16227 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:16.956439 16156 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:16.833750 16170 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.962203 16170 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:16.868515 16531 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:16.963202  8408 raft_consensus.cc:1240] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:16.964318 16531 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.789723 16324 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.833671 16413 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.118s
I20260709 12:29:16.994128  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:16.956488 16227 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:16.970968 16208 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.273s	user 0.018s	sys 0.002s
I20260709 12:29:16.853116 16514 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:16.999993  7659 raft_consensus.cc:2376] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21. Re-sending same reply.
I20260709 12:29:17.025646 16413 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:17.025940 16227 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:17.026393 16413 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.027429 16227 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:17.029277 16413 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.030557 15838 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:16.972811 16375 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:17.045132  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.046224  7660 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:17.068526  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:17.069653  7657 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24.
I20260709 12:29:17.000011 16231 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:17.076226 16231 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.087821 16516 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:17.088516 16516 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.089043 16516 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:17.092918  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.093562  7730 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:17.099717  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.116925  8411 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:17.120797  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.122505  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.123695  7661 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:17.080828 16504 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.125823 16504 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:17.126248 16558 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:17.128237 16351 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:17.128715 16351 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:17.129729 16351 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:17.132019  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:17.140311 16548 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.608s	user 0.000s	sys 0.002s
I20260709 12:29:17.141640 16375 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:17.145184 16226 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:17.145956  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1552) took 1366 ms. Trace:
I20260709 12:29:17.146755  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3724) took 1214 ms. Trace:
I20260709 12:29:17.151859  7732 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:17.152180 15838 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:17.153388 15838 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:17.146435  8408 rpcz_store.cc:276] 0709 12:29:15.779455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.806605 (+ 27150us) service_pool.cc:224] Handling call
0709 12:29:15.819027 (+ 12422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:16.963851 (+1144824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.145663 (+181812us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540046d7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:17.145891 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630761344}
I20260709 12:29:17.158694  8414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1701) took 2353 ms. Trace:
I20260709 12:29:17.159248  8414 rpcz_store.cc:276] 0709 12:29:14.805288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.807108 (+  1820us) service_pool.cc:224] Handling call
0709 12:29:14.807523 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:16.204662 (+1397139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.158255 (+953593us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54006c25d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:17.158587 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3164531456}
I20260709 12:29:17.147169  7807 rpcz_store.cc:276] 0709 12:29:15.932142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.954709 (+ 22567us) service_pool.cc:224] Handling call
0709 12:29:15.977916 (+ 23207us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:16.060303 (+ 82387us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:29:16.261952 (+201649us) log.cc:844] Serialized 14 byte log entry
0709 12:29:16.429778 (+167826us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:16.431061 (+  1283us) log.cc:844] Serialized 52 byte log entry
0709 12:29:16.431380 (+   319us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:16.431435 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:16.431560 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:17.139655 (+708095us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:17.146394 (+  6739us) raft_consensus.cc:1685] finished
0709 12:29:17.146514 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.146663 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104320,"prepare.queue_time_us":344487,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":16858,"raft.queue_time_us":118479,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":52551,"spinlock_wait_cycles":22456576,"thread_start_us":285589,"threads_started":3,"wal-append.queue_time_us":199024}
I20260709 12:29:17.152037 16538 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.098187 16500 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.120500 16469 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 12:29:17.175698 16557 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:17.176631 16557 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.179776 16550 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:17.180823  7735 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:29:17.180603 16550 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.184847  7655 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:29:17.186452  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.187583  7806 raft_consensus.cc:2393] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:17.188939  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.190737  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.129460 16503 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.191628  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:17.194618 16503 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.196687  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.198520 16550 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.199390 16557 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.205036  7777 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24.
I20260709 12:29:17.205811  7777 leader_election.cc:403] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:29:17.209092  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.214494  7735 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 12:29:17.201347  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 435017us
I20260709 12:29:17.165709  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:17.222497  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:17.235113  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.257856  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:17.258975  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 93406.5us
I20260709 12:29:17.259675  7728 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:17.218694  7624 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:17.260615  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:17.260725  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 51795.6us
W20260709 12:29:17.265007  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60367.2us
I20260709 12:29:17.265705  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3653) took 1549 ms. Trace:
I20260709 12:29:17.118422 16570 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:17.266636  7804 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:29:17.266916 16568 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.253877 16351 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:17.268555  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.269865  7806 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:17.274983  7729 rpcz_store.cc:276] 0709 12:29:15.713989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.724986 (+ 10997us) service_pool.cc:224] Handling call
0709 12:29:17.263527 (+1538541us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b540007f8d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:17.263694 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510865152}
I20260709 12:29:17.265935  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:17.274389  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 55799.8us
I20260709 12:29:17.267748  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.280968  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:17.290912  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:17.291011  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:17.305202  8411 raft_consensus.cc:2368] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24.
I20260709 12:29:17.306162  7805 raft_consensus.cc:2410] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:29:17.310549  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:17.313228  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.317559  7810 raft_consensus.cc:2410] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:29:17.232479 16532 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:29:17.319006  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.311033  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:17.329836  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72100.4us
I20260709 12:29:17.304272  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.330209  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3966) took 1056 ms. Trace:
I20260709 12:29:17.324440  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:17.335855  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.337847  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.334344  7733 rpcz_store.cc:276] 0709 12:29:16.273492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.272997 (+999505us) service_pool.cc:224] Handling call
0709 12:29:17.330133 (+ 57136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.328446  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.338325  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:17.337844  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 484453us
I20260709 12:29:17.325835 16578 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:17.292708 16449 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:17.348306  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.349326  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.350891  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:17.350812  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.355556  8411 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:17.350430  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.358537  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:17.345042  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.365109  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:17.372437  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.551s (SENT)
I20260709 12:29:17.329862 16521 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24.
W20260709 12:29:17.379195  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75111us
I20260709 12:29:17.373193  7627 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.354526 15916 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:17.354555 16578 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:17.366083 16520 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:17.309382 16568 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:17.402980  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44624.1us
I20260709 12:29:17.404670  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.407136  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:17.388674 16213 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.008s
I20260709 12:29:17.411821  7775 leader_election.cc:400] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24.
I20260709 12:29:17.360828 16226 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:29:17.372910 16576 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:17.443027  8405 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:29:17.392504 16557 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:17.361338 16449 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.453029  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:17.454107  8411 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:17.395339 16578 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:17.462893 16538 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.463265 16580 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:17.463919 16552 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:17.464250 16580 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:17.464859 16580 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:17.470667  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57358.9us
W20260709 12:29:17.470976  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 124160us
I20260709 12:29:17.474387  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:17.475749  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71132.5us
I20260709 12:29:17.479827  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.479955  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.464790 16562 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:17.481443  8414 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:17.450707 16554 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:17.491717  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.458930 16534 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:17.402412 16125 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.070s
I20260709 12:29:17.447480 16520 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:17.496299  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 115320us
I20260709 12:29:17.496042  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.467144 16375 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:17.508581  7805 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:17.458729 16531 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:17.440253 16550 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:29:17.441609 16521 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:29:17.482628 16562 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:17.500259 16554 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:17.512617  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 117585us
I20260709 12:29:17.503016 16534 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.502388 16538 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:17.544930  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.545490  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.549896  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.544755  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.550998  8415 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:17.511283 16231 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:17.551424  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:17.547574  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 175085us
I20260709 12:29:17.555910  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.557785  7806 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:17.558207  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.550673  7808 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:17.545607 16554 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:29:17.553504 16571 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
I20260709 12:29:17.552613 16534 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:17.577778  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.582573  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.597858  7810 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
W20260709 12:29:17.599129 16582 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:29:17.599285  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.600070  7810 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:17.600346 16568 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:17.601150 16575 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
I20260709 12:29:17.601219  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.602619 16504 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:17.584101 16581 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:17.585281 16567 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:29:17.586337  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.530692 16531 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.605490  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.587905  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.607436 16531 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:17.607720 16588 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:29:17.608445  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.609935  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.588572  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1247) took 1816 ms. Trace:
I20260709 12:29:17.593947  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.587775  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.566095 16231 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:17.572314 16413 consensus_queue.cc:1059] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
W20260709 12:29:17.613389  8412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1242) took 1953 ms (client timeout 2228 ms). Trace:
I20260709 12:29:17.607805  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.592211 16579 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:17.606841  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:17.602818  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:17.598517 16520 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.628048  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:17.624331  8412 rpcz_store.cc:269] 0709 12:29:15.660443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.660595 (+   152us) service_pool.cc:224] Handling call
0709 12:29:17.613080 (+1952485us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:17.613282 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079181696}
I20260709 12:29:17.727944  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.727650 16579 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:17.729516  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.730305 16579 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:17.615897  8413 rpcz_store.cc:276] 0709 12:29:15.704974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.720035 (+ 15061us) service_pool.cc:224] Handling call
0709 12:29:15.722563 (+  2528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:15.722833 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:29:16.209065 (+486232us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b18003a9c60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9
0709 12:29:17.137440 (+928375us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b18003a9c60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:29:17.343254 (+205814us) log.cc:844] Serialized 14 byte log entry
0709 12:29:17.519045 (+175791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.520581 (+  1536us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:29:17.520637 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.520719 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.520871 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25513,"prepare.queue_time_us":43264,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":7875,"raft.queue_time_us":31451,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":15258,"spinlock_wait_cycles":49356032,"thread_start_us":309578,"threads_started":2,"wal-append.queue_time_us":195004}
I20260709 12:29:17.731405 16573 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:17.732929  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.733697  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:17.733758  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:17.735706  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.736708  7736 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:17.738224  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:17.739859  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 121321us
I20260709 12:29:17.741719  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.743471  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.730373  7732 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 18: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:17.744104  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:17.747574  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.748096  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.739054  7736 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:17.750416  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.753499  7733 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:17.739965  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.755753  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:17.758230  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 170573us
W20260709 12:29:17.761701  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:17.763247  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.767997  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.769011  7736 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:17.770318  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.742311  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:17.757324  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:17.774459  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 365745us
I20260709 12:29:17.777765  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.778838  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.781679  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.783342  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.781018  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:17.792297  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.784411  7736 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:17.795526  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.796991  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:17.800040  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.808887  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.804486  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.810906  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.811965  7662 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:17.813555  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.815551 16469 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:17.818099  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 182117us
I20260709 12:29:17.772361 16579 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:17.773522 16581 raft_consensus.cc:2764] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:29:17.797924 16500 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:17.820013  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.834379  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:17.773039 16125 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:17.796720 16538 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:17.828425  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.824120  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.839553  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.840552  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.841639  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.844395  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.845126  7659 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:17.846045  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.846572  7659 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:17.847903  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:17.849732  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.849725  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.821475 16156 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.790234 16568 consensus_queue.cc:1059] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:29:17.875325  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:17.850689  7659 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:17.877301  7732 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:29:17.875876  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:17.873220  7700 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.859483  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:17.830153 16579 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.851747  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.884917  8411 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:17.838974 16449 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:17.895349  7730 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:17.843036 16538 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.868322 16213 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:17.637919 16562 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.928310 16366 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.848067 16469 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:17.898736  7700 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:17.915540 16555 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:17.916072 16599 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:17.918447 16608 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:17.909240 16538 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:17.893844 16461 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:17.919523 16600 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:17.920858  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73050.8us
I20260709 12:29:17.876392 16170 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:17.912995  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:17.958793 16351 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.007s
I20260709 12:29:17.959471  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:17.904167 16579 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:17.965510  8412 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:17.906941 16531 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:17.903744 16324 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:17.966492  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 224287us
I20260709 12:29:17.980115  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:17.928411 16586 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:17.886997 16156 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.988142  7730 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
W20260709 12:29:17.996259 16584 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:29:17.964370 16366 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:17.964474 16599 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:17.902014 16609 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:29:18.012204  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3954) took 2125 ms. Trace:
I20260709 12:29:17.967573 16608 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:18.014557  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:18.044883  8380 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.975170 16461 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:18.045404  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65636.5us
I20260709 12:29:18.007731  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.003559 16561 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:18.048530  7737 rpcz_store.cc:276] 0709 12:29:15.886402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.010999 (+2124597us) service_pool.cc:224] Handling call
0709 12:29:18.011489 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.011733 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:18.011767 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:18.011822 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:18.011860 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:18.011941 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:18.012108 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:17.999063 16565 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.053s	user 0.001s	sys 0.003s
I20260709 12:29:18.052580  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:18.013172 16605 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:29:18.006503  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1852) took 1705 ms. Trace:
W20260709 12:29:18.000800 16607 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:29:18.062394  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:18.053418  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:18.049410  7662 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:18.072420  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:18.063140  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:18.106911  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:18.111971  7730 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:29:18.112125  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49113.8us
I20260709 12:29:18.060276  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.108595  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.115047  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:18.051474 16600 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:18.102151  7734 rpcz_store.cc:276] 0709 12:29:16.301464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.551896 (+250432us) service_pool.cc:224] Handling call
0709 12:29:17.263948 (+712052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:17.264384 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.264415 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:17.286714 (+ 22299us) log.cc:844] Serialized 26 byte log entry
0709 12:29:17.450137 (+163423us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:17.450212 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.450339 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:17.612073 (+161734us) raft_consensus.cc:1685] finished
0709 12:29:17.612100 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:18.006144 (+394044us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b540007f8d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:18.006418 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3139,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":16424,"spinlock_wait_cycles":1469869824,"thread_start_us":109027,"threads_started":1,"wal-append.queue_time_us":165804}
W20260709 12:29:18.124256  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 225718us
I20260709 12:29:18.124707  7737 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
W20260709 12:29:18.121780  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 59486.7us
I20260709 12:29:18.048120 16599 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:17.981488 16578 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.127885  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:18.133248  7732 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:18.054010 16604 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.134511  7737 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:29:18.126444  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:18.055696 16608 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:18.097723 16581 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:18.060252 16531 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:18.136454 16606 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:29:18.138000  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77813.7us
I20260709 12:29:18.150743  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.141448  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.096441 15916 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:18.096379 16561 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:18.047617 16324 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.121455 16226 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:18.137712 16580 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.629s	user 0.014s	sys 0.001s
I20260709 12:29:18.129947 16600 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.156819  7655 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:18.137089 16231 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:18.182085  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.164347  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.158219 16461 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.186288  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.186460 16461 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:18.166965 16596 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
W20260709 12:29:18.167109 16603 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.120s	user 0.001s	sys 0.002s
I20260709 12:29:18.192515  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.166350 16351 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:18.200690  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.165768 16595 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:18.173061 16156 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:18.169708 16613 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.213905  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.186177 15916 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.182085 16231 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:18.215991  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.217247  7732 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:18.164194 16226 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:18.182854 16600 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:18.210976 16516 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.121s	user 0.013s	sys 0.003s
I20260709 12:29:18.242554  7658 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.209283 16351 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.207966 16606 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:18.172461 16468 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.211288 16595 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.183497 16580 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.148331 16604 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.257192  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:18.249599 15882 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.269002 15882 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.236536 16231 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:18.231436 16125 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.265810  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.230594  8406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:29:18.276547  8406 raft_consensus.cc:606] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:18.262342  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45918 (request call id 3525) took 3061 ms. Trace:
I20260709 12:29:18.241524 16226 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:18.268651  7659 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:18.166436 16531 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.285440 16610 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:18.288287  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3557) took 1257 ms. Trace:
I20260709 12:29:18.275790 16581 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:18.278789  7658 rpcz_store.cc:276] 0709 12:29:15.182163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.346182 (+164019us) service_pool.cc:224] Handling call
0709 12:29:18.207970 (+2861788us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54004358d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f19a3d 00007fc006ee9cd0 00007fc00046e254 00007fc00046e1e2 00007fc00046e111 00007fc00046e08c 00007fc00046d342 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:18.243242 (+ 35272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582790016}
I20260709 12:29:18.248334 16594 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:18.259107 16351 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:18.282064 15882 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:18.354065  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 47994.6us
W20260709 12:29:18.293486  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:18.296202 16579 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.297221 16612 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:18.297211  7731 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:18.306078  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1861) took 1553 ms. Trace:
I20260709 12:29:18.307554  7662 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:29:18.307689 16366 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.312650 16614 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:18.284335 16125 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:18.313804  7803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3740) took 2021 ms (client timeout 1870 ms). Trace:
I20260709 12:29:18.249781 16516 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.317211  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:18.319897 16620 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.320582  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:18.321496 16534 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.001s
I20260709 12:29:18.397209 16534 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.267904 16545 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:29:18.322132 16618 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.324189 16604 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.233158 16613 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:18.329628 16569 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.063s	user 0.005s	sys 0.015s
I20260709 12:29:18.167162 16561 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.266917 16595 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:18.286731 16580 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.289059  7657 rpcz_store.cc:276] 0709 12:29:17.031138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.148898 (+117760us) service_pool.cc:224] Handling call
0709 12:29:17.166277 (+ 17379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:17.176739 (+ 10462us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:29:17.470686 (+293947us) log.cc:844] Serialized 14 byte log entry
0709 12:29:17.868054 (+397368us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:17.886583 (+ 18529us) log.cc:844] Serialized 26 byte log entry
0709 12:29:17.886789 (+   206us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:17.886841 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.886951 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:18.012313 (+125362us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:18.287699 (+275386us) raft_consensus.cc:1685] finished
0709 12:29:18.287881 (+   182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:18.288106 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203183,"prepare.queue_time_us":365237,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1661,"raft.queue_time_us":360737,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":249,"spinlock_wait_cycles":54402432,"thread_start_us":384571,"threads_started":3,"wal-append.queue_time_us":349215}
W20260709 12:29:18.332676 16611 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:29:18.275177 16413 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:29:18.340378  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:18.344031  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:18.346226  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:18.407045 16573 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.003s
W20260709 12:29:18.334966 16554 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.001s
I20260709 12:29:18.363583  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1855) took 2057 ms. Trace:
I20260709 12:29:18.426543  7731 rpcz_store.cc:276] 0709 12:29:16.306590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.570021 (+263431us) service_pool.cc:224] Handling call
0709 12:29:18.296332 (+1726311us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:29:18.296708 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.360296 (+ 63588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:18.363511 (+  3215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69603072}
I20260709 12:29:18.428123  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:18.360700  7728 rpcz_store.cc:276] 0709 12:29:16.752545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.752798 (+   253us) service_pool.cc:224] Handling call
0709 12:29:16.753255 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.280860 (+527605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:18.305649 (+1024789us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54002481d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:18.305961 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164139264}
W20260709 12:29:18.366251  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 102948us
I20260709 12:29:18.366406  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:18.367427  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:18.434966  7810 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:18.429917  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:18.436014  7728 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
W20260709 12:29:18.373333  7803 rpcz_store.cc:269] 0709 12:29:16.292671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.292830 (+   159us) service_pool.cc:224] Handling call
0709 12:29:18.313392 (+2020562us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400133dd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:18.313620 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474347776}
I20260709 12:29:18.437754  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:18.381671  7729 raft_consensus.cc:2368] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:29:18.439785  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.377810  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:18.385815  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65311.9us
I20260709 12:29:18.439383  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:18.441247  7655 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:18.441418  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.387308 16623 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:18.365366 16614 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.447438 16366 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.447878 16614 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:18.394253 16516 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.396198 16324 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.396541  7653 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:18.408754  8411 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:18.363186 16612 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.384315  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.453083  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.443195  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.421962 16578 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:18.365877 16538 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.361759  7730 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:18.466779  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:29:18.467851  8413 raft_consensus.cc:606] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Received request to transfer leadership to TS fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.468499  8413 raft_consensus.cc:614] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:18.481094  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:18.467125  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:18.589152  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:18.602967  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 218776us
W20260709 12:29:18.605360  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 138309us
I20260709 12:29:18.450829 16213 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.158s
I20260709 12:29:18.444060 16617 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.166s
I20260709 12:29:18.428264 16573 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:18.611831 15916 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:18.460911 16366 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.612674 16227 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:18.613524 16538 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:18.613633 16227 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.614396 16227 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:18.612888 16366 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.629690  7736 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:18.633942  8405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1713) took 1315 ms. Trace:
I20260709 12:29:18.634631  8405 rpcz_store.cc:276] 0709 12:29:17.318498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.318684 (+   186us) service_pool.cc:224] Handling call
0709 12:29:18.633507 (+1314823us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400183b54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:18.633825 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249475456}
W20260709 12:29:18.467069 16552 consensus_meta.cc:287] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.002s	user 0.015s	sys 0.002s
I20260709 12:29:18.639173 16552 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:18.644292  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 163297us
I20260709 12:29:18.606859  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:18.665763  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 60150.5us
I20260709 12:29:18.447345 16580 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:18.760759  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 152554us
I20260709 12:29:18.751675  7728 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:29:18.756999  7653 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:18.764354  7656 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:18.407655 16620 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.465169 16609 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:29:18.746558 16611 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.313s	user 0.000s	sys 0.000s
I20260709 12:29:18.804046  8405 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:29:18.604110 16377 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:18.641043 16558 raft_consensus.cc:3055] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:18.607396 16610 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.423307 16600 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:18.875895  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:18.469002 16612 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:18.412298 16618 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:18.872972  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 266176us
I20260709 12:29:18.758324 16534 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:18.749148 16619 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:18.756287 16604 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:18.900878  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.442834 16554 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.941694  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:18.903277  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 235641us
I20260709 12:29:18.801602 16170 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.782583 15916 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.143s
W20260709 12:29:18.878630  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 425636us
I20260709 12:29:18.795279 16538 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:18.802307 16324 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:29:18.893555 16565 log.cc:927] Time spent T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:29:18.813912 16557 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.814090 16516 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:18.874679 16558 raft_consensus.cc:740] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:18.800340 16609 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.746503 16581 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.953900 16604 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:18.959856  7812 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:18.959383  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:18.960990  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:18.994944  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:18.900717 16383 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:18.974644  8405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:29:18.949190 16531 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:18.965709 16538 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.996441 15891 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:19.000916  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:18.995162 16599 consensus_meta.cc:287] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.837s	user 0.014s	sys 0.000s
I20260709 12:29:19.003448  8405 raft_consensus.cc:606] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:18.965446 16324 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:19.014662  8405 raft_consensus.cc:614] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:18.979393 16125 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.016588  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:18.961824 16170 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:18.992918 16609 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.025136  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:19.001950 16579 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.005363 16623 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:19.004477 16377 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:18.948319 16610 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.010639 16578 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.010886 16538 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:19.011020 16606 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.012356 16594 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:19.031850 16351 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.002s
I20260709 12:29:18.989755 16557 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.023255 16619 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:19.030501 16531 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:19.018903 16552 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 12:29:19.062228 16461 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.868s	user 0.013s	sys 0.002s
I20260709 12:29:19.036550 16623 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:19.068167 16600 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.068578 16623 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.070168 16557 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:19.076103 16554 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.054910 16558 consensus_queue.cc:260] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.081787 16610 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.082199 16618 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.046891 16578 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:19.091194 16630 log.cc:927] Time spent T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:29:19.046741 16226 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.004s
I20260709 12:29:19.092547 16226 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.098011 16573 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:19.063166 16613 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.009212 16383 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:19.061870  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72316.5us
I20260709 12:29:19.086853 16623 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:19.121259  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 148052us
I20260709 12:29:19.130277 16552 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:19.130993 16552 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:19.132283 16552 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.132866 16543 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.133967 16552 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:19.135819 16625 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:29:19.137918 16383 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:19.138384 16556 log.cc:927] Time spent T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 2.507s	user 0.007s	sys 0.011s
I20260709 12:29:19.053681 16351 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.140096 16170 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.142073 16351 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.054134 16594 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.143970 16554 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:19.144929 16594 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.148321  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:19.149181 16227 consensus_meta.cc:287] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.534s	user 0.013s	sys 0.000s
I20260709 12:29:19.149307  7653 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:19.149775 16227 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.150983  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.098752 16573 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.151949  7653 raft_consensus.cc:2393] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:19.153859 16573 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.156288  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.157932 16550 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:19.158473 16550 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:19.160234 16628 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:29:19.023797 16599 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.160739 16554 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.161480  7654 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:19.162806 16599 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.163381 16604 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.166646 16324 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.090176 16578 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:19.173653 16608 consensus_meta.cc:287] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.018s	user 0.013s	sys 0.000s
I20260709 12:29:19.108523 16377 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:19.053825 16606 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:19.176226 16231 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.892s	user 0.005s	sys 0.005s
I20260709 12:29:19.162631 16554 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.187206 16226 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:19.195196 16633 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:29:19.201745  7812 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1410) took 2033 ms (client timeout 1811 ms). Trace:
W20260709 12:29:19.204162  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:19.200727 16598 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.290s	user 0.008s	sys 0.014s
W20260709 12:29:19.200727 16366 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.001s
W20260709 12:29:19.217533  7812 rpcz_store.cc:269] 0709 12:29:17.220094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.232211 (+ 12117us) service_pool.cc:224] Handling call
0709 12:29:19.201487 (+1969276us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003ea3d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.201645 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3827560704}
I20260709 12:29:19.221405  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:19.224481  8413 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:19.223181  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 198102us
I20260709 12:29:19.225703  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:19.195822 16608 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.229312  7731 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:19.230120 16504 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:19.197211 16377 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.244946  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:19.255980  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:19.277611  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.279019  7659 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.276170 16504 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.288832  7653 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:19.219136 16620 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.291774  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.236734 16324 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:19.294669  7653 raft_consensus.cc:2368] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24.
I20260709 12:29:19.278462 16377 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:19.230543 16227 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:19.306597  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.277875 16606 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.302027  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.288954 16504 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:19.279158 16578 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:19.216946 16231 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.223750 16538 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.316888  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:19.320396  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.322000  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:19.290416 16637 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:19.276692 16366 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.327629  7627 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:19.327667  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:19.279525 16581 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 12:29:19.281795 16594 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:29:19.333807  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:19.332173  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.335654  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.344811  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:19.314977 16620 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:19.348465  7662 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:19.313227 16613 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:19.327667 16614 consensus_meta.cc:287] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.006s
I20260709 12:29:19.343696 16637 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:19.352173 16637 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.355595  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:19.356195 16623 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.357167 16573 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:19.357708 16573 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:19.358080 16573 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:19.359335 16595 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.910s	user 0.005s	sys 0.008s
I20260709 12:29:19.359817 16595 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.362308 16595 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:19.330375 16618 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:19.325758 16606 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:19.366237 16613 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.319358 15838 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:19.368203 15838 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.368718 16613 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.370810 15838 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.371821 16579 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:19.377681  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3994) took 1018 ms. Trace:
I20260709 12:29:19.379570 16620 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.384217  7735 rpcz_store.cc:276] 0709 12:29:18.359619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.064902 (+705283us) service_pool.cc:224] Handling call
0709 12:29:19.065336 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:19.177207 (+111871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.177241 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:19.177304 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:19.177339 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.377371 (+200032us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540037ee54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.377424 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.377596 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256686976}
I20260709 12:29:19.397400  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.333906 16610 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:19.330163 16531 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:19.400648  7702 leader_election.cc:400] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24.
I20260709 12:29:19.411093  7702 leader_election.cc:403] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:29:19.344676 16500 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:29:19.383412 16640 log.cc:927] Time spent T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:29:19.412612  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.341673 16608 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.328495 16578 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.420302  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:19.495085  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 75386.4us
W20260709 12:29:19.470597  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:19.498966  7653 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:19.474120  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3990) took 1202 ms. Trace:
I20260709 12:29:19.501039  7731 rpcz_store.cc:276] 0709 12:29:18.271756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.622018 (+350262us) service_pool.cc:224] Handling call
0709 12:29:18.746242 (+124224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:19.229925 (+483683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.473700 (+243775us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400248bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.474013 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110351104}
I20260709 12:29:19.382954 16634 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:19.405539 16613 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:29:19.399835 16581 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:19.394577 16614 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.508401 16208 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.512156 16468 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:19.516050 16208 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.535493 16208 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:19.535653  7658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1626) took 1132 ms. Trace:
I20260709 12:29:19.536521  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.531409 16554 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:19.537437  7735 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:19.530802 16568 raft_consensus.cc:3055] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:19.538414 16568 raft_consensus.cc:740] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:19.539541 16562 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:19.540277 16568 consensus_queue.cc:260] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.548344 16568 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.543707 16495 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.549916 16495 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.509927 16500 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:19.545110 16538 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.546121  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.552287  7657 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:19.552145 16500 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.553730  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.554524  7657 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 18.
I20260709 12:29:19.532487 16366 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.556169  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.542219 16578 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.556916  7657 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:19.536319  7658 rpcz_store.cc:276] 0709 12:29:18.403695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.403860 (+   165us) service_pool.cc:224] Handling call
0709 12:29:18.416763 (+ 12903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:18.417244 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:18.417296 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:19.177331 (+760035us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b2c0001c748. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc00081ba11 00007fc00082b372 00007fc001f5da01 00007fc001f2243a 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2
0709 12:29:19.191838 (+ 14507us) log.cc:844] Serialized 26 byte log entry
0709 12:29:19.297892 (+106054us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:19.297951 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.298059 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:19.519065 (+221006us) raft_consensus.cc:1685] finished
0709 12:29:19.519108 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.535533 (+ 16425us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1424,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6033,"spinlock_wait_cycles":12472192,"thread_start_us":67762,"threads_started":1,"wal-append.queue_time_us":106458}
I20260709 12:29:19.539073  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.560389  7659 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:19.496361 16618 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:19.567621  7659 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:19.500424  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.569496  7659 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
W20260709 12:29:19.565969 16612 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.012s
I20260709 12:29:19.576551 16612 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:19.580894 16626 log.cc:927] Time spent T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.554s	user 0.005s	sys 0.001s
I20260709 12:29:19.583082 16538 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.584039 16538 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.571240  7653 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:19.579032 16612 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.592680  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:19.597575  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.599152  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.591275  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.608441  7658 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:19.510986 16609 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.511061 16634 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:19.617551  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122765us
I20260709 12:29:19.620052  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:19.630147  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90596.4us
I20260709 12:29:19.636467  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.632716 16599 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:19.639775 16599 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:19.640362 16599 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.657968  7653 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:19.666980  7658 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:19.669670  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.670207  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.673151  7735 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26.
I20260709 12:29:19.674365  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:19.674500  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:19.644908 16231 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.675711  8407 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:19.677253  8405 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:19.688112  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.676769 16593 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:19.691157  8408 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:19.692838  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:19.693363 16609 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.681775 16514 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.695923 16449 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.696707 16449 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.697599 16573 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.642696 16504 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.701896 16504 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.705291 16620 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.682912 16495 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.706642  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.709010  7660 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:19.712090  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3674) took 2066 ms (client timeout 1945 ms). Trace:
W20260709 12:29:19.712833  7729 rpcz_store.cc:269] 0709 12:29:17.645260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.755752 (+110492us) service_pool.cc:224] Handling call
0709 12:29:19.711701 (+1955949us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400419454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.711961 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341804288}
I20260709 12:29:19.694350  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:19.715114  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.754575  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3277) took 1449 ms. Trace:
I20260709 12:29:19.755075  7662 rpcz_store.cc:276] 0709 12:29:18.304820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.364049 (+ 59229us) service_pool.cc:224] Handling call
0709 12:29:18.364363 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:19.350274 (+985911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.754290 (+404016us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b540028df54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.754505 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258802560}
I20260709 12:29:19.756376  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:19.716902 16634 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.726500  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3981) took 1521 ms. Trace:
I20260709 12:29:19.737542  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.737697  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1627) took 1333 ms. Trace:
W20260709 12:29:19.768513  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:19.768285  7656 rpcz_store.cc:276] 0709 12:29:18.407275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.407466 (+   191us) service_pool.cc:224] Handling call
0709 12:29:18.764061 (+356595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.765019 (+   958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.737433 (+972414us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400116550. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.737616 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819110784}
I20260709 12:29:19.725472  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1255) took 1599 ms. Trace:
I20260709 12:29:19.770468  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.771358  7656 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 17.
I20260709 12:29:19.770843  8411 rpcz_store.cc:276] 0709 12:29:18.125997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.126229 (+   232us) service_pool.cc:224] Handling call
0709 12:29:18.137015 (+ 10786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.462626 (+325611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.725052 (+1262426us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54007b2fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.725376 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609068416}
I20260709 12:29:19.772740  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.774396  7656 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:19.779572  8405 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:19.782891  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.784085  7729 raft_consensus.cc:2368] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 18. Current term is 19.
I20260709 12:29:19.786653  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.784454  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:19.793089  7627 leader_election.cc:400] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26.
I20260709 12:29:19.793684  7627 leader_election.cc:403] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:29:19.766389  7732 rpcz_store.cc:276] 0709 12:29:18.205485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.248521 (+ 43036us) service_pool.cc:224] Handling call
0709 12:29:18.273980 (+ 25459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:18.434915 (+160935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:18.434954 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:18.461337 (+ 26383us) log.cc:844] Serialized 26 byte log entry
0709 12:29:18.461552 (+   215us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:18.461592 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.725888 (+1264296us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002495d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.725963 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:19.726042 (+    79us) raft_consensus.cc:1685] finished
0709 12:29:19.726065 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.726412 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369340160}
I20260709 12:29:19.796116  7660 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:19.801120  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:19.816926  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3972) took 2050 ms. Trace:
I20260709 12:29:19.821319 16616 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:19.827678  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.827867 16557 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.828588  7807 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:19.828785 16557 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.831557 16557 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.832523  7812 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:19.832790  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.833863  7806 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:19.802587 16593 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26.
I20260709 12:29:19.737949 16538 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.838587  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.839884  7656 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:19.840843 16504 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:19.841997 16021 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.841696 16504 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.801167 16609 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:19.843521  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3992) took 1485 ms. Trace:
I20260709 12:29:19.843736 16609 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:19.844398  8414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1714) took 2525 ms (client timeout 1910 ms). Trace:
I20260709 12:29:19.845199 16609 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:19.846385  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:19.846944  8415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1256) took 1714 ms. Trace:
I20260709 12:29:19.749260 16573 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.847374  8415 rpcz_store.cc:276] 0709 12:29:18.132593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.132751 (+   158us) service_pool.cc:224] Handling call
0709 12:29:19.845316 (+1712565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:19.845568 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.845597 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:19.845642 (+    45us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:19.845704 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.845761 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.846742 (+   981us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400587850. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.846886 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3900977920}
I20260709 12:29:19.848515  7811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "629e345c827545598250d02a9f39e2fb"
dest_uuid: "d72176ccaed049958ac68f706d4f052d"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:50788
I20260709 12:29:19.849494  7811 raft_consensus.cc:606] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:19.849890 16508 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.844055  7737 rpcz_store.cc:276] 0709 12:29:18.358644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.872333 (+513689us) service_pool.cc:224] Handling call
0709 12:29:18.943736 (+ 71403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.955088 (+ 11352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:18.955128 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:18.955174 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:18.955214 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.843288 (+888074us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400212854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:19.843337 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.843448 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25558272}
I20260709 12:29:19.742353 16550 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.852185  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.853142  7737 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:19.818975 16377 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:19.844882  8414 rpcz_store.cc:269] 0709 12:29:17.319035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.328000 (+  8965us) service_pool.cc:224] Handling call
0709 12:29:19.844164 (+2516164us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400587854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.844294 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319960320}
I20260709 12:29:19.856325  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.875286  7658 raft_consensus.cc:1240] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:19.850879  7655 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
W20260709 12:29:19.860397  7654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3268) took 1876 ms (client timeout 1619 ms). Trace:
I20260709 12:29:19.831738  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.864269  7659 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:19.855556  7733 raft_consensus.cc:1240] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:19.853991  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.849924  7811 raft_consensus.cc:614] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:19.860909  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:19.876849  7654 rpcz_store.cc:269] 0709 12:29:17.984175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.984514 (+   339us) service_pool.cc:224] Handling call
0709 12:29:19.860070 (+1875556us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400116054. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:19.860279 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534118784}
I20260709 12:29:19.874220  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.842641 16021 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.894825  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:19.897583  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 209788us
W20260709 12:29:19.900992  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:19.828482 16554 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.896255  7729 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:19.898881  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.902434  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.906244  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.904218  7656 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:19.904135  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:19.815687 16613 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.913777  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.901975  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.880702 16351 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:19.916999  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.903113  7803 raft_consensus.cc:2376] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24. Re-sending same reply.
I20260709 12:29:19.912467  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:19.881340 16561 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:19.919632  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.907692  7660 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: []
I20260709 12:29:19.915043  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.922840  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.924564  7658 raft_consensus.cc:2393] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:19.917955  7732 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:19.924661  7775 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:19.858155  7553 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 21 to 22, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 22 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.926282  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:19.897329 16324 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.897558 16620 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.935104  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.936061  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.937388  7808 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:19.927461  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.928155  7775 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:19.930069  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:19.916378  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.939831  7656 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:29:19.920718  7806 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:19.850613 16508 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.940250  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.943121  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.944207  7806 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26.
I20260709 12:29:19.946229  7658 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:19.929754  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.943986  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.913117 16554 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.918010 16581 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:19.953086  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:19.951776  7811 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:19.934363 16635 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:19.953997  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.941828 16500 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:29:19.954654 16449 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:19.955050 16324 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.955653  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:19.922667 16613 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:20.025846  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:19.955997  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:19.956173 16021 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.934710 16461 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:19.934959 16125 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:29:20.048301 16125 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:20.048800 16125 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:20.049827 16125 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.957442 16504 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.957553  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.956907 16632 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:19.958185 16637 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:29:20.064046 16637 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:20.064652 16637 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:19.905733 16531 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:20.067601 16531 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.926795 16351 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:19.928176 16579 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:19.960909  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:19.962122  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:19.970573 16581 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:19.976029  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:19.978578 16554 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:20.019277  7809 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:19.915413 16377 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:19.968572 16620 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:19.941618 16545 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:19.962762 16562 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:19.979696 16599 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:20.041841  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 108247us
I20260709 12:29:20.044070 16635 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:19.917860 16538 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:29:20.061280  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:20.070096 16627 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 1.123s	user 0.002s	sys 0.003s
I20260709 12:29:20.034302 16553 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:20.110502  7737 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:20.111128  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:20.035044  7732 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:20.249414  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:20.663538  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:20.106592  7656 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:20.664819  7658 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:20.660343  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.660617  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1568) took 1078 ms. Trace:
W20260709 12:29:20.669562  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 713607us
W20260709 12:29:20.683048  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.992s (SENT) [suppressed 36 similar messages]
I20260709 12:29:20.668179  8413 rpcz_store.cc:276] 0709 12:29:19.587018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.594271 (+  7253us) service_pool.cc:224] Handling call
0709 12:29:19.594760 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:19.595324 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.595355 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:19.595928 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:29:19.797830 (+201902us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:19.797876 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.797949 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:19.960632 (+162683us) raft_consensus.cc:1685] finished
0709 12:29:19.960655 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.660528 (+699873us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1814,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":13718,"thread_start_us":172039,"threads_started":1,"wal-append.queue_time_us":314441}
I20260709 12:29:20.753134  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3690) took 1440 ms. Trace:
I20260709 12:29:20.111440  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: true dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.697229  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:20.623641  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.012s	sys 0.015s
W20260709 12:29:20.757128  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.013s	sys 0.015s
W20260709 12:29:20.757807  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 827790us
I20260709 12:29:20.754904  7808 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:20.762845  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:20.702560  7736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3980) took 2898 ms (client timeout 1779 ms). Trace:
W20260709 12:29:20.763923  7736 rpcz_store.cc:269] 0709 12:29:17.788657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.794965 (+  6308us) service_pool.cc:224] Handling call
0709 12:29:20.686967 (+2892002us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54000d2ad4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:20.687213 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915848576}
I20260709 12:29:20.765413  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.766412  7736 raft_consensus.cc:2410] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:29:20.766901  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3685) took 1456 ms. Trace:
I20260709 12:29:20.767262  7736 rpcz_store.cc:276] 0709 12:29:19.310926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.765126 (+1454200us) service_pool.cc:224] Handling call
0709 12:29:20.766834 (+  1708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.768337  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.769131  7736 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:20.769713  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1883) took 1047 ms. Trace:
I20260709 12:29:20.770066  7736 rpcz_store.cc:276] 0709 12:29:19.722378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.768041 (+1045663us) service_pool.cc:224] Handling call
0709 12:29:20.769639 (+  1598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.071269 16618 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24.
I20260709 12:29:20.707029  7657 raft_consensus.cc:1240] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:20.707341  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:20.707511  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1719) took 2497 ms. Trace:
I20260709 12:29:20.708276 16599 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:20.708707  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.777524  7809 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:20.685094 16125 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:20.709353 16620 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:20.711359 16657 log.cc:927] Time spent T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:29:20.694263 16632 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:20.467743  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:20.691466 16613 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:20.756006  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.699349  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.807721  7807 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:20.111555 16581 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:20.752841 16377 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:20.671500 16545 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.814896 16545 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:20.773471  7735 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:20.771193  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.685540 16635 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:20.772212  8409 rpcz_store.cc:276] 0709 12:29:18.212769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.230080 (+ 17311us) service_pool.cc:224] Handling call
0709 12:29:18.244303 (+ 14223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:18.255683 (+ 11380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:18.255750 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:18.276347 (+ 20597us) log.cc:844] Serialized 26 byte log entry
0709 12:29:18.424230 (+147883us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:18.424288 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:18.424373 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:20.069192 (+1644819us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:20.707127 (+637935us) raft_consensus.cc:1685] finished
0709 12:29:20.707154 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.707436 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10673,"raft.queue_time_us":11286,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":19746,"spinlock_wait_cycles":10014464,"thread_start_us":117722,"threads_started":1,"wal-append.queue_time_us":358071}
I20260709 12:29:20.798287  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.800076  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.753913  7737 rpcz_store.cc:276] 0709 12:29:19.312430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.918623 (+606193us) service_pool.cc:224] Handling call
0709 12:29:20.753014 (+834391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.805168 16632 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:20.813918 16377 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:20.772063  7728 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:29:20.815672  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120130us
I20260709 12:29:20.815974  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.824837  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:20.829286  7737 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:20.824829  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.822623  7736 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:20.831378  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3701) took 1206 ms. Trace:
I20260709 12:29:20.831877  7736 rpcz_store.cc:276] 0709 12:29:19.624554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.770891 (+1146337us) service_pool.cc:224] Handling call
0709 12:29:20.831178 (+ 60287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.830368  7809 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:20.836658  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.838162  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3700) took 1214 ms. Trace:
I20260709 12:29:20.838821  7737 rpcz_store.cc:276] 0709 12:29:19.624153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.824457 (+1200304us) service_pool.cc:224] Handling call
0709 12:29:20.838055 (+ 13598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.840265  7807 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:20.842432  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.843524  7807 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:20.759330 16553 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:20.851732  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.854084  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.858489  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.859674  7808 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:20.863131  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:20.826128 16660 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:20.827006 16635 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:20.828464 16324 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:20.871644  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.827865 16538 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:29:20.876549  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:20.880043  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1880) took 1158 ms. Trace:
I20260709 12:29:20.874639  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3688) took 1562 ms. Trace:
I20260709 12:29:20.880532  7737 rpcz_store.cc:276] 0709 12:29:19.721299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.839878 (+1118579us) service_pool.cc:224] Handling call
0709 12:29:20.879945 (+ 40067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.850490 16531 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:20.872059 16660 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:20.885933 16351 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:20.896287  7736 rpcz_store.cc:276] 0709 12:29:19.311726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.862705 (+1550979us) service_pool.cc:224] Handling call
0709 12:29:20.874521 (+ 11816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.910152  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.912111  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:20.912894  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3683) took 1602 ms. Trace:
I20260709 12:29:20.914587  7737 rpcz_store.cc:276] 0709 12:29:19.310557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.909723 (+1599166us) service_pool.cc:224] Handling call
0709 12:29:20.912804 (+  3081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.918438  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3681) took 1608 ms. Trace:
I20260709 12:29:20.919169  7736 rpcz_store.cc:276] 0709 12:29:19.310141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.911779 (+1601638us) service_pool.cc:224] Handling call
0709 12:29:20.918333 (+  6554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.908649  7728 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 29.
I20260709 12:29:20.926681  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3997) took 1713 ms. Trace:
I20260709 12:29:20.930224  7737 rpcz_store.cc:276] 0709 12:29:19.212865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.915716 (+1702851us) service_pool.cc:224] Handling call
0709 12:29:20.925572 (+  9856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:20.925846 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:20.926037 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:20.926128 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:20.926192 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:20.926300 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.926573 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.893913 16324 consensus_queue.cc:1059] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:29:20.938125 16553 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:20.946852  7812 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:20.947746  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3759) took 1084 ms. Trace:
I20260709 12:29:20.948134  7812 rpcz_store.cc:276] 0709 12:29:19.862972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.110995 (+248023us) service_pool.cc:224] Handling call
0709 12:29:20.947671 (+836676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.949576  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.951527  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:20.952332  7804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.087s	sys 0.059s
W20260709 12:29:20.952736  7804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.087s	sys 0.059s
I20260709 12:29:20.928660  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1882) took 1206 ms. Trace:
I20260709 12:29:20.954084  7728 rpcz_store.cc:276] 0709 12:29:19.722028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.702045 (+980017us) service_pool.cc:224] Handling call
0709 12:29:20.928573 (+226528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.954831  7732 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
W20260709 12:29:20.955525  7732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3686) took 1644 ms (client timeout 2177 ms). Trace:
I20260709 12:29:20.955811  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4004) took 1662 ms. Trace:
W20260709 12:29:20.956027  7732 rpcz_store.cc:269] 0709 12:29:19.311093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.929295 (+618202us) service_pool.cc:224] Handling call
0709 12:29:20.955462 (+1026167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.956285  7728 rpcz_store.cc:276] 0709 12:29:19.293793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.954868 (+1661075us) service_pool.cc:224] Handling call
0709 12:29:20.955307 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:20.955499 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:20.955526 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:20.955569 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:20.955603 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:20.955661 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.955759 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.961974  7807 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 29.
I20260709 12:29:20.968510  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4002) took 1747 ms. Trace:
I20260709 12:29:20.969152  7737 rpcz_store.cc:276] 0709 12:29:19.220782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.932009 (+1711227us) service_pool.cc:224] Handling call
0709 12:29:20.967716 (+ 35707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:20.968000 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:20.968043 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:20.968109 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:29:20.968170 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:20.968248 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.968424 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.974236  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.975677  7812 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:20.977497  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.984416  7807 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 19: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:20.987073  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:20.983350  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:20.988979  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:20.990077 16561 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.049s	user 0.012s	sys 0.000s
I20260709 12:29:20.949458 16660 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:20.991006  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.021302  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 3107) took 1103 ms. Trace:
I20260709 12:29:21.021770  7807 rpcz_store.cc:276] 0709 12:29:19.917591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.990720 (+1073129us) service_pool.cc:224] Handling call
0709 12:29:21.021208 (+ 30488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:20.991789  7812 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
W20260709 12:29:20.994375 16637 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.929s	user 0.014s	sys 0.000s
I20260709 12:29:20.995097 16377 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:20.966882 16324 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:21.003863 16632 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:21.014819  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1877) took 1294 ms. Trace:
I20260709 12:29:21.057204  7812 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:29:21.056614  7737 rpcz_store.cc:276] 0709 12:29:19.720289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.970454 (+1250165us) service_pool.cc:224] Handling call
0709 12:29:21.014189 (+ 43735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.014431 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.014455 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.014501 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:21.014543 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.014598 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.014742 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.051651  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.051903  7805 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:21.057794  7812 pending_rounds.cc:85] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 15
I20260709 12:29:21.059679  7812 pending_rounds.cc:107] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:29:21.059988  7812 raft_consensus.cc:2889] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:21.062254  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 378547us
I20260709 12:29:21.065461  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3754) took 1204 ms. Trace:
I20260709 12:29:21.065937  7812 rpcz_store.cc:276] 0709 12:29:19.861455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.056042 (+1194587us) service_pool.cc:224] Handling call
0709 12:29:21.056664 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.065165 (+  8501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.065393 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.069556 16377 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:21.070597 16377 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.071846  7809 raft_consensus.cc:2410] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:29:19.817534  7734 rpcz_store.cc:276] 0709 12:29:17.766592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.124202 (+357610us) service_pool.cc:224] Handling call
0709 12:29:18.142707 (+ 18505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:18.149194 (+  6487us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:29:18.272030 (+122836us) log.cc:844] Serialized 14 byte log entry
0709 12:29:19.580149 (+1308119us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:19.745381 (+165232us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b180030f760. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac223a
0709 12:29:19.816374 (+ 70993us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:19.816468 (+    94us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:19.816551 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.816656 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:19.816844 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34120,"prepare.queue_time_us":255976,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":5431,"raft.queue_time_us":51880,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":13446,"spinlock_wait_cycles":33456384,"thread_start_us":255302,"threads_started":2,"wal-append.queue_time_us":691259}
W20260709 12:29:21.076013 16658 log.cc:927] Time spent T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:29:21.078064  7732 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:29:21.057521 16637 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:21.080374  8408 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1717) took 3736 ms (client timeout 3223 ms). Trace:
W20260709 12:29:21.092458  8408 rpcz_store.cc:269] 0709 12:29:17.344475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:17.344658 (+   183us) service_pool.cc:224] Handling call
0709 12:29:21.080087 (+3735429us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54005e0e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:21.080258 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5344756608}
I20260709 12:29:21.093287  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3753) took 1233 ms. Trace:
I20260709 12:29:21.093864 16637 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.093766  7803 rpcz_store.cc:276] 0709 12:29:19.860287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.860436 (+   149us) service_pool.cc:224] Handling call
0709 12:29:21.093058 (+1232622us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54003c46d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:21.093210 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49718144}
I20260709 12:29:21.080693  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.081336  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45926 (request call id 3290) took 1692 ms. Trace:
W20260709 12:29:21.097747  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3691) took 1785 ms (client timeout 2227 ms). Trace:
W20260709 12:29:21.098263  7729 rpcz_store.cc:269] 0709 12:29:19.312736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.080312 (+1767576us) service_pool.cc:224] Handling call
0709 12:29:21.097668 (+ 17356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.099043  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1878) took 1378 ms. Trace:
I20260709 12:29:21.082798 16558 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:21.079133  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1884) took 1315 ms. Trace:
I20260709 12:29:21.101703  7734 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:21.102797  8408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:29:21.104267  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1879) took 1383 ms. Trace:
I20260709 12:29:21.051621  7808 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:21.095570  7653 rpcz_store.cc:276] 0709 12:29:19.389121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.660074 (+270953us) service_pool.cc:224] Handling call
0709 12:29:19.660436 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:19.664949 (+  4513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.664981 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:19.677239 (+ 12258us) log.cc:844] Serialized 52 byte log entry
0709 12:29:20.026652 (+349413us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:29:20.026703 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:20.026802 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:21.080952 (+1054150us) raft_consensus.cc:1685] finished
0709 12:29:21.080992 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.081247 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5564,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":6763,"thread_start_us":281043,"threads_started":1,"wal-append.queue_time_us":1086013}
I20260709 12:29:21.101473  7732 rpcz_store.cc:276] 0709 12:29:19.763626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.077030 (+1313404us) service_pool.cc:224] Handling call
0709 12:29:21.077546 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.078726 (+  1180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.079039 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.105528  8408 raft_consensus.cc:606] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:21.099782  7737 rpcz_store.cc:276] 0709 12:29:19.720669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.058305 (+1337636us) service_pool.cc:224] Handling call
0709 12:29:21.098214 (+ 39909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.098435 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.098461 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.098609 (+   148us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:21.098702 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.098814 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.098979 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.062069 16554 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.104698  7734 rpcz_store.cc:276] 0709 12:29:19.720983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.074713 (+1353730us) service_pool.cc:224] Handling call
0709 12:29:21.096841 (+ 22128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.103947 (+  7106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.104205 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.106067  8408 raft_consensus.cc:614] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:21.107440  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3761) took 1243 ms. Trace:
I20260709 12:29:21.107928  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.108347  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3756) took 1246 ms. Trace:
W20260709 12:29:21.110131  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 296372us
I20260709 12:29:21.109889  7807 rpcz_store.cc:276] 0709 12:29:19.862125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.096482 (+1234357us) service_pool.cc:224] Handling call
0709 12:29:21.107724 (+ 11242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.107958 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.107986 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.108030 (+    44us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:21.108074 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.108134 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.108287 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.064093 16665 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.074868 16377 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.111886  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.068528 16660 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:21.114617  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.111311  7808 rpcz_store.cc:276] 0709 12:29:19.863612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.858169 (+994557us) service_pool.cc:224] Handling call
0709 12:29:21.107373 (+249204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.113816  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1886) took 1048 ms. Trace:
I20260709 12:29:21.117121  8408 raft_consensus.cc:1240] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 12:29:21.058149  8379 leader_election.cc:336] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.200s (SENT)
I20260709 12:29:21.133721  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.134244  7729 rpcz_store.cc:276] 0709 12:29:20.065669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.099190 (+1033521us) service_pool.cc:224] Handling call
0709 12:29:21.113216 (+ 14026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.113434 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.113462 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.113524 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:21.113557 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.113614 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.113742 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:21.135347  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 77218.4us
W20260709 12:29:21.136274  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 2.233s (SENT)
I20260709 12:29:21.136847  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.149125  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.059891 16324 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.150072  8408 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:21.151914  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3757) took 1289 ms. Trace:
I20260709 12:29:21.148546  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.157709  7807 rpcz_store.cc:276] 0709 12:29:19.862406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.112887 (+1250481us) service_pool.cc:224] Handling call
0709 12:29:21.151063 (+ 38176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.151383 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.151425 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.151496 (+    71us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:21.151546 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.151632 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.151820 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:21.119264 16554 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.159396  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.159639  8411 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:21.160181  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.160930  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.176280  7732 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:21.161087 16324 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.158034  8415 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:21.118638 16637 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.154786  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.090143 16581 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:21.214214  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 59571.4us
W20260709 12:29:21.172116  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.170s (SENT)
I20260709 12:29:21.216859  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.159547  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.217391  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.218075  8409 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
W20260709 12:29:21.230634  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58612.4us
I20260709 12:29:21.241092  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:21.241428  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.243060  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.155923 16670 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.264060  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.274506  8408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce"
dest_uuid: "992d6df95ef74f7f8b12534534a45501"
mode: GRACEFUL
new_leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:59578
I20260709 12:29:21.276068  8408 raft_consensus.cc:606] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Received request to transfer leadership to TS be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:21.174131 16377 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:21.155709 16538 consensus_queue.cc:1059] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.091s
I20260709 12:29:21.283571  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.253293  7807 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:21.209317 16599 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.277789  7731 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:21.253855 16351 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:21.281977  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3752) took 1422 ms. Trace:
I20260709 12:29:21.289469  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.296241  7734 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:21.252722 16324 raft_consensus.cc:3055] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:21.252460 16531 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:21.286479 16514 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:21.270663 16500 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:21.118450 16665 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.315013  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:21.306344 16663 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:29:21.322559  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 218892us
I20260709 12:29:21.221163 16637 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.275993 16613 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:21.254045 16614 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.219626 16620 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:21.298367  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:21.341361  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109673us
I20260709 12:29:21.281301 16671 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:21.302721  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.304046  7807 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:21.305109  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:29:21.295188 16599 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.302961 16610 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.281616 16670 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.302359 16324 raft_consensus.cc:740] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:21.333491  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 197234us
I20260709 12:29:21.309321 16508 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.309041 16538 raft_consensus.cc:3055] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:21.351244  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.350613  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:21.356751  7734 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:21.297629  7804 rpcz_store.cc:276] 0709 12:29:19.859759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.865645 (+  5886us) service_pool.cc:224] Handling call
0709 12:29:19.866228 (+   583us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:29:19.898122 (+ 31894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.898154 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:29:19.917696 (+ 19542us) log.cc:844] Serialized 78 byte log entry
0709 12:29:20.953176 (+1035480us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:21.280116 (+326940us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.281538 (+  1422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.281633 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:21.281692 (+    59us) raft_consensus.cc:1685] finished
0709 12:29:21.281710 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.281901 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31731,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":11624,"thread_start_us":1191046,"threads_started":2,"wal-append.queue_time_us":1082132}
W20260709 12:29:21.340143  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.344348  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.343240  7661 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:21.358887  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:21.320763 16500 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:21.359016  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:21.344337  7627 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.945s (SENT)
I20260709 12:29:21.362440 16599 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.363421 16554 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.365998 16610 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.346055 16602 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.392964  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.405777  7653 raft_consensus.cc:2393] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:21.352820 16538 raft_consensus.cc:740] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:21.408649  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.409467  7732 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:21.377169  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.413280  7807 raft_consensus.cc:2410] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:29:21.425554  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:21.426326  7658 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:21.432015 16637 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.411985  7810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3755) took 1550 ms. Trace:
I20260709 12:29:21.381930  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.379112  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:21.371459  7731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3697) took 2054 ms (client timeout 2221 ms). Trace:
W20260709 12:29:21.437402  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 58399.9us
W20260709 12:29:21.437184  7731 rpcz_store.cc:269] 0709 12:29:19.316477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.694997 (+1378520us) service_pool.cc:224] Handling call
0709 12:29:21.371162 (+676165us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54002477d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:21.371370 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071654784}
W20260709 12:29:21.403553  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 59279.4us
I20260709 12:29:21.415832 16665 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:21.402637 16666 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.403373 16614 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:21.420295 16635 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.006s
I20260709 12:29:21.423573 16581 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.424785  7659 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:29:21.425035  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3998) took 2210 ms. Trace:
I20260709 12:29:21.438416  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:21.439361  7627 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.269s (SENT)
I20260709 12:29:21.419845  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.453840  7627 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.435328  7810 rpcz_store.cc:276] 0709 12:29:19.861811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.089244 (+1227433us) service_pool.cc:224] Handling call
0709 12:29:21.089637 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.089915 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:29:21.287950 (+198035us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.411533 (+123583us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.411607 (+    74us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:21.411663 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.411740 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.411902 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":270781,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":172,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":121,"thread_start_us":320651,"threads_started":2}
I20260709 12:29:21.406010 16671 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:21.458341  7658 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:21.458685  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.382651 16351 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.453300  7736 rpcz_store.cc:276] 0709 12:29:19.214554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.931424 (+1716870us) service_pool.cc:224] Handling call
0709 12:29:20.931965 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:20.936960 (+  4995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:20.937017 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:20.947693 (+ 10676us) log.cc:844] Serialized 26 byte log entry
0709 12:29:21.269775 (+322082us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:21.269819 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.269890 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:21.424784 (+154894us) raft_consensus.cc:1685] finished
0709 12:29:21.424808 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.424968 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":284,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8158,"thread_start_us":288442,"threads_started":1,"wal-append.queue_time_us":411267}
I20260709 12:29:21.455408  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.460325  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.460489  7807 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:21.444288  7804 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:21.442001 16514 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:21.465309  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:21.445266  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3771) took 1531 ms. Trace:
I20260709 12:29:21.463245  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.413084 16602 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.450351 16669 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.476075  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.473171  7811 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:21.480557  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.489617  7661 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 18.
I20260709 12:29:21.450769 16580 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.477087  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 3993) took 3117 ms. Trace:
I20260709 12:29:21.469369  7810 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:29:21.507529  7730 rpcz_store.cc:276] 0709 12:29:18.359196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.950957 (+591761us) service_pool.cc:224] Handling call
0709 12:29:18.968874 (+ 17917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.104470 (+2135596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.104503 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.104550 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:21.104604 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.476671 (+372067us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54002486d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:21.476746 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.476955 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4910834944}
I20260709 12:29:21.509567  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.495282  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.511776  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.512553 16670 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.514113 16669 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.467895 16514 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:21.515322  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.455463 16614 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.516537  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.517050  8413 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:29:21.517791  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.451717 16666 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.517992 16614 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.515899 16514 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:21.518607  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.518821 16614 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:21.519416  8405 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:21.520692  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.521574 16599 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.522516  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.466270  7808 rpcz_store.cc:276] 0709 12:29:19.913652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.131918 (+1218266us) service_pool.cc:224] Handling call
0709 12:29:21.132392 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.132698 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:29:21.343069 (+210371us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.444094 (+101025us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.444778 (+   684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.444840 (+    62us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:21.444903 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.444994 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.445144 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":231906,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":249,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12526,"spinlock_wait_cycles":18295552,"thread_start_us":291930,"threads_started":2}
I20260709 12:29:21.525467  7658 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:21.526258  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.526280  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.529965  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:29:21.530510  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3687) took 2219 ms. Trace:
I20260709 12:29:21.457926 16610 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.529778 16508 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:21.532472  8408 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:21.536991  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.371206 16500 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:21.478778 16545 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.004s
I20260709 12:29:21.538235  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.464272 16665 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:21.539696  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.540098  8409 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:29:21.538761  7807 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 25. Current term is 26. Ops: []
W20260709 12:29:21.540771  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45637.1us
I20260709 12:29:21.469604 16679 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:21.540482  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.467666 16668 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:21.546829  7661 raft_consensus.cc:1240] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:21.540633  8413 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:21.531641 16632 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:21.457229 16635 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.549621  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.553375  7659 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:29:21.553823  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.548588  7658 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:21.554890 16671 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:21.541795 16599 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.542483 16508 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.551796  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:21.563423  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.468559 16351 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.547964 16679 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.558014  7808 raft_consensus.cc:2410] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:29:21.570148  7807 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:29:21.573012  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.462853 16324 consensus_queue.cc:260] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.549101 16632 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.532351 16580 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.558400 16614 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:21.568359 16508 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:21.464112 16538 consensus_queue.cc:260] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.591741 16687 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:21.594273 16635 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.600813 16599 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.575956 16679 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:21.586261 16580 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.616375 16669 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.617952 16666 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.619293  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.619743  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.620157 16602 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.581529 16632 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:21.548586 16545 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.620237  7658 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:21.627699  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.576354 16665 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:29:21.574699 16610 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:21.631606  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.631606 16610 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:21.635499 16610 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:21.581851 16688 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:21.635550  8407 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:21.636672  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:21.636837 16545 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.637441  7660 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:21.639623 16351 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.641327  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.642184  8407 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:21.644209  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:21.645085  7734 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:21.646032  7732 raft_consensus.cc:1240] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:21.646430  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.649907  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:21.651540  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:21.642189  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.655800  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.641996  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.661105  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.662034  8411 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:21.664922  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:21.665519 16558 consensus_queue.cc:1059] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:21.667377  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.668504  8415 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:21.670727  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 122949us
I20260709 12:29:21.670893  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:21.673280  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:21.671414  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.671106 16125 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.005s
I20260709 12:29:21.676265 16691 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:21.675598 16668 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:21.677044  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.678136  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:21.680218 16668 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:21.681661  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.673547 16595 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:21.683779 16595 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:21.684295 16595 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:21.684556 16351 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:21.685050 16351 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:21.685477 16351 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:21.619102 16508 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.635113 16514 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.694116 16595 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.696053 16514 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.696208 16691 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:21.684320  8415 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:21.698300 16351 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.699357  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.701009  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.701488 16351 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.697105 16595 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:21.675419  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:21.649389  7626 connection.cc:256] RPC call timeout handler was delayed by -1.6329231599997911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 12:29:21.706645  7626 connection.cc:256] RPC call timeout handler was delayed by -1.565423159999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 12:29:21.707086  7626 connection.cc:256] RPC call timeout handler was delayed by -1.5439742700000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 12:29:21.715523  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.723333  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:21.732885 16602 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.733794 16602 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.736575 16602 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:21.737130 16599 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:29:21.737437  7626 connection.cc:256] RPC call timeout handler was delayed by -1.3566231599997991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:29:21.737867 16599 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.740525  8415 raft_consensus.cc:1240] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:21.742205 16599 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:21.742584  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:21.742808 16351 consensus_queue.cc:1059] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:21.744997 16351 raft_consensus.cc:3055] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:21.745529 16351 raft_consensus.cc:740] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:21.746683 16351 consensus_queue.cc:260] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.748884 16351 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:21.753532  7626 connection.cc:256] RPC call timeout handler was delayed by -1.3278231599997525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 12:29:21.754472  7626 connection.cc:256] RPC call timeout handler was delayed by -1.1015826600000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 12:29:21.755167  7626 connection.cc:256] RPC call timeout handler was delayed by -1.083117189999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
I20260709 12:29:21.756798  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.761512 16599 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:29:21.663961 16635 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:21.766355  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.770130  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:21.772451  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 107662us
I20260709 12:29:21.663578 16666 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.102s
W20260709 12:29:21.779019  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 107071us
I20260709 12:29:21.775660  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.784418 16666 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:21.784937 16666 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:21.785861 16666 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.787359 16666 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:21.794720  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120325us
I20260709 12:29:21.764714 16688 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:21.770694 16691 raft_consensus.cc:993] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: : Instructing follower be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:29:21.770315 16545 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:21.768215 16635 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:21.808593  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:21.816045  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:21.776600 16627 log.cc:927] Time spent T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:29:21.812714  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:21.817863  7734 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:21.769490 16599 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.804593 16688 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.828879  7626 leader_election.cc:400] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26.
I20260709 12:29:21.830415  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.818648 16635 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:21.822265 16614 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:21.833348  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:21.822515 16602 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:29:21.825290 16609 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:21.827814 16599 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.831290 16695 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:21.812139 16514 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:29:21.836017 16687 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:29:21.844329  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:21.836835  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63258.1us
I20260709 12:29:21.830505 16688 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:21.845912  7734 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
W20260709 12:29:21.855417  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.850939  8380 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:21.865270  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:21.874028  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 209924us
I20260709 12:29:21.850446 16695 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.865449 16668 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:21.867367 16614 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:21.863688 16697 raft_consensus.cc:1081] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Signalling peer be9c043c6d424ba1ab8def55dc9bb3e2 to start an election
I20260709 12:29:21.884359  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.864741 16602 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:21.886987  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:21.888828  8411 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:21.882822 16666 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:21.896627  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:21.880410 16695 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:21.890200 16558 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:21.884047 16614 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.906239 16614 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:21.906592 16703 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:21.877975 16679 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.908604  8413 raft_consensus.cc:2368] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25.
I20260709 12:29:21.909714 16679 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:21.912212  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 3999) took 2696 ms (client timeout 1975 ms). Trace:
I20260709 12:29:21.912418 16599 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:21.912742  7735 rpcz_store.cc:269] 0709 12:29:19.215569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.687704 (+472135us) service_pool.cc:224] Handling call
0709 12:29:21.911992 (+2224288us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540039b7d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:21.912119 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493957376}
I20260709 12:29:21.902351 16125 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:21.913869 16125 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.914261 16125 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:21.914340  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3587) took 2531 ms. Trace:
I20260709 12:29:21.914764  7657 rpcz_store.cc:276] 0709 12:29:19.382733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.612805 (+230072us) service_pool.cc:224] Handling call
0709 12:29:19.633374 (+ 20569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:20.771588 (+1138214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.914040 (+1142452us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540002fb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:21.914239 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468533632}
I20260709 12:29:21.901528 16666 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:21.935950 16705 consensus_queue.cc:1059] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:21.941300  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1875) took 3289 ms. Trace:
I20260709 12:29:21.941875  7733 rpcz_store.cc:276] 0709 12:29:18.652000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.209832 (+557832us) service_pool.cc:224] Handling call
0709 12:29:19.255360 (+ 45528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:19.877930 (+622570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.930645 (+2052715us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b540037da54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:21.941207 (+ 10562us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379581184}
I20260709 12:29:21.530997  7728 rpcz_store.cc:276] 0709 12:29:19.311425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.957384 (+1645959us) service_pool.cc:224] Handling call
0709 12:29:20.983422 (+ 26038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:20.985289 (+  1867us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:29:21.278660 (+293371us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.413898 (+135238us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:21.461012 (+ 47114us) log.cc:844] Serialized 26 byte log entry
0709 12:29:21.461246 (+   234us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:21.463892 (+  2646us) log.cc:844] Serialized 14 byte log entry
0709 12:29:21.469095 (+  5203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:21.469224 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:21.529414 (+ 60190us) raft_consensus.cc:1685] finished
0709 12:29:21.529440 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:21.530431 (+   991us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17200,"prepare.queue_time_us":360611,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4806,"raft.queue_time_us":1192,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":10794,"spinlock_wait_cycles":31646848,"thread_start_us":384070,"threads_started":2,"wal-append.queue_time_us":149872}
W20260709 12:29:21.947570  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46882.1us
I20260709 12:29:21.950806  8411 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
W20260709 12:29:21.952155  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118949us
I20260709 12:29:21.954799  7626 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:21.951443  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 142953us
I20260709 12:29:21.900211 16602 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:21.948005 16699 log.cc:927] Time spent T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:29:21.946466 16616 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:21.964635 16602 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:21.967891 16692 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:29:21.960256  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:21.970558 16554 raft_consensus.cc:3055] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:21.971148 16554 raft_consensus.cc:740] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:21.972225 16554 consensus_queue.cc:260] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:21.973490 16554 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:21.962668 16616 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:21.977519 16610 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.981184 16670 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.981909 16670 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:21.958356 16558 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:21.988694 16670 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:21.991130  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.005841  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:22.007081  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52399.2us
I20260709 12:29:22.008790  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:22.017194  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:22.019410  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:21.991415 16666 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.022643 16704 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:22.023595 16704 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.025905 16550 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:22.026537 16704 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:22.028704  7809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1441) took 1195 ms (client timeout 1586 ms). Trace:
W20260709 12:29:22.029251  7809 rpcz_store.cc:269] 0709 12:29:20.833481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.836246 (+  2765us) service_pool.cc:224] Handling call
0709 12:29:22.028454 (+1192208us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54003732d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:22.028635 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537984768}
I20260709 12:29:22.030926  7804 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:29:22.031446  7804 pending_rounds.cc:85] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 15
I20260709 12:29:22.031699  7804 pending_rounds.cc:107] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:29:22.031948  7804 raft_consensus.cc:2889] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:22.000973 16671 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.035380 16697 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:22.036381 16593 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:22.039610 16671 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:21.991554 16558 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.045306  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:21.960989  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 60751.3us
W20260709 12:29:22.047127  7778 connection.cc:256] RPC call timeout handler was delayed by -1.0210324979998404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 12:29:22.102970  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80970.7us
W20260709 12:29:22.059777  7778 connection.cc:256] RPC call timeout handler was delayed by -1.0129353289999017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:29:22.115749  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:22.117611  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:22.160692  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 152161us
W20260709 12:29:22.157514 16632 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.000s
I20260709 12:29:22.161852  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:22.164139  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 213889us
I20260709 12:29:22.173141  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:22.046803 16610 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.038834 16635 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.178408  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:22.179090  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:22.165571 16579 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.387s	user 0.027s	sys 0.030s
I20260709 12:29:22.179200  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:22.180179  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:22.176643  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.188117  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:22.190649  7807 raft_consensus.cc:2393] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:22.192698  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce"
dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
 from {username='slave'} at 127.0.0.1:40408
I20260709 12:29:22.192667  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.204654  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:22.206087  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92225.6us
I20260709 12:29:22.196799  7658 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:22.194247  7660 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:22.209606  7660 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:22.166404 16602 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:29:22.190191 16579 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.437s	user 0.028s	sys 0.031s
I20260709 12:29:22.220348  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:22.253326  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:22.190411 16687 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:29:22.354650  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:22.209025 16679 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:22.373824  8411 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:29:22.214134 16665 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:22.380620  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:22.383848 16514 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:22.383565 16707 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:22.385836 16514 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:22.216836 16602 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:22.402844 16602 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:22.408089 16710 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.409240 16704 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:22.410207 16704 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.222144 16610 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:22.186667 16599 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.169s
I20260709 12:29:22.409091 16710 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.430459  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:22.433524 16704 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.219753 16579 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:22.184187 16632 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:22.418654 16351 consensus_meta.cc:287] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.669s	user 0.015s	sys 0.000s
I20260709 12:29:22.460093 16351 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:22.458318 16668 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:22.465445 16695 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.556s	user 0.003s	sys 0.009s
I20260709 12:29:22.466003 16695 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.374078 16679 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.418666 16514 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:22.377425 16665 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.494923  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4010) took 1368 ms. Trace:
I20260709 12:29:22.471459  7775 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:22.471029  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.481878 16579 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.482928 16632 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:22.483395 16125 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.001s
I20260709 12:29:22.483671 16715 consensus_queue.cc:1059] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:29:22.474362  7660 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.523422  7658 raft_consensus.cc:1240] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:22.523730  7657 raft_consensus.cc:2468] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:22.495828  7729 rpcz_store.cc:276] 0709 12:29:21.126055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.158898 (+ 32843us) service_pool.cc:224] Handling call
0709 12:29:21.159213 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.349885 (+190672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:21.349926 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:21.349978 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:21.350029 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:22.494385 (+1144356us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000726d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:22.494477 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:22.494800 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437739392}
W20260709 12:29:22.467736 16688 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.000s
W20260709 12:29:22.521737  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 50393.5us
I20260709 12:29:22.530371  7775 leader_election.cc:400] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25.
I20260709 12:29:22.531600  7775 leader_election.cc:403] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:22.487437 16679 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:22.485945 16562 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.501125 16665 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:22.538595  7660 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:22.539309  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 326201us
I20260709 12:29:22.512643 16691 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:22.544287 16324 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:22.544946 16609 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:22.545702 16609 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.528403 16351 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.551537  8413 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: [27.21-27.21]
I20260709 12:29:22.488343 16668 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:29:22.527909 16635 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:22.552137  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 191420us
I20260709 12:29:22.557282  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:22.525141 16714 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.557994  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.558604  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:22.536422 16689 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:29:22.558270 16714 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.560297  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.560484  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:22.561224  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:22.561671 16714 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.534616 16695 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.541946 16550 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:22.564354  7804 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:22.562186  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:22.569901  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 354201us
I20260709 12:29:22.570255 16718 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:22.570531  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.577349  8413 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:22.572163  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.573387 16710 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.547928 16632 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:22.524116 16125 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.552433 16709 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:29:22.578303  8415 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:22.582113 16717 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:22.583511 16666 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:22.584342 16718 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:22.562884 16514 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.585884 16695 consensus_queue.cc:1059] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:29:22.588666  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:22.591447 16554 consensus_meta.cc:287] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.617s	user 0.015s	sys 0.002s
I20260709 12:29:22.594882 16550 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.599586  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.551834 16687 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:22.537282 16688 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.606127  8409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1259) took 1092 ms. Trace:
I20260709 12:29:22.541960 16562 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.609915  8413 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:22.575167 16715 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.117s
I20260709 12:29:22.633293  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: true dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.739878  8414 raft_consensus.cc:3055] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:22.635262  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.741285  8407 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:22.637223  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:22.644531 16595 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:22.657289  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4014) took 1530 ms. Trace:
I20260709 12:29:22.743157  7732 rpcz_store.cc:276] 0709 12:29:21.127139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.410610 (+283471us) service_pool.cc:224] Handling call
0709 12:29:21.645826 (+235216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.646457 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:22.656773 (+1010316us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400247cd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:22.657151 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465327360}
I20260709 12:29:22.661324 16538 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:22.662499 16693 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:29:22.668442 16593 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:29:22.751756  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 170848us
W20260709 12:29:22.752956  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 153544us
I20260709 12:29:22.672469 16558 raft_consensus.cc:3055] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:21.970178  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:22.740458  8414 raft_consensus.cc:740] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:22.686846  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:22.683434 16666 consensus_queue.cc:1243] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:29:22.753261  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:22.689437 16514 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:22.690876 16719 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:29:22.698848  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 121282us
I20260709 12:29:22.556667 16351 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:22.712018  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 149923us
I20260709 12:29:22.767902  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:22.768811  7661 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:22.769371  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:22.770023  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:22.770781  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:22.771126  7653 raft_consensus.cc:2376] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22. Re-sending same reply.
I20260709 12:29:22.614714  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:22.726096  7775 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:22.691133 16717 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:22.772905 16717 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.584381 16609 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.773581  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:22.716343  8409 rpcz_store.cc:276] 0709 12:29:21.513821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.529547 (+ 15726us) service_pool.cc:224] Handling call
0709 12:29:21.530002 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:21.545565 (+ 15563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:22.605802 (+1060237us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b54003228d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:22.606046 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10215808}
I20260709 12:29:22.770323  7662 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:22.774312  7656 raft_consensus.cc:2376] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26. Re-sending same reply.
I20260709 12:29:22.716979  8415 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:22.772900  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:22.735914 16614 consensus_meta.cc:287] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.829s	user 0.016s	sys 0.000s
I20260709 12:29:22.781512  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:22.783692  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:22.785069  8413 raft_consensus.cc:2393] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:22.761972 16719 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.788762 16719 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.790431 16719 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:22.791646 16687 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:22.793151  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 67221.1us
I20260709 12:29:22.795043  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:22.795818 16713 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:22.796607 16713 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.796710  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:22.752069 16693 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:22.797763 16713 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.799135 16693 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.800710 16693 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:22.802028  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:22.803683  7805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45836 (request call id 3091) took 4458 ms (client timeout 1992 ms). Trace:
W20260709 12:29:22.804220  7805 rpcz_store.cc:269] 0709 12:29:18.345691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.345859 (+   168us) service_pool.cc:224] Handling call
0709 12:29:22.803472 (+4457613us) spinlock_profiling.cc:243] Waited 2.63 s on lock 0x7b54001329d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:22.803585 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6046296448}
I20260709 12:29:22.752344 16595 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.773787  7806 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:22.809437  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:22.809998  7775 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:22.801769 16602 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.814291  8414 consensus_queue.cc:260] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.747961 16610 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:22.817086 16610 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.821601  8414 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:29:22.824687 16671 consensus_meta.cc:287] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.002s
I20260709 12:29:22.825280 16671 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.756337 16558 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.759178 16715 consensus_queue.cc:1243] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 28.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:29:22.835866  7659 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:22.835866  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:22.838163  8413 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:22.842610  8414 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:22.845470  7624 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:22.853936 16723 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:22.807175 16614 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.829089 16700 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:22.857226 16554 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:22.840150 16324 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.858378 16554 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.859318 16550 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.861543 16554 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.866163 16351 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.867084 16558 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:22.868940 16558 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:22.821671 16125 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:22.871747  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:22.872270 16706 raft_consensus.cc:2764] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:29:22.872869  8414 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:22.847280 16704 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:22.873773 16704 raft_consensus.cc:697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.855350 16602 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:22.877104  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:22.895490  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 142112us
I20260709 12:29:22.900427 16728 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:22.905671  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55277.2us
I20260709 12:29:22.858493 16324 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:22.906510  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 61151.6us
W20260709 12:29:22.911702  7806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3760) took 3041 ms (client timeout 1927 ms). Trace:
I20260709 12:29:22.911027  8379 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:22.916172  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:22.912240  7806 rpcz_store.cc:269] 0709 12:29:19.863303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.957231 (+ 93928us) service_pool.cc:224] Handling call
0709 12:29:22.904549 (+2947318us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:22.904704 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6236209920}
I20260709 12:29:22.920903  7806 pending_rounds.cc:85] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 16
I20260709 12:29:22.918941 16691 raft_consensus.cc:2804] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:22.921748  7806 pending_rounds.cc:107] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:29:22.885814 16688 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:22.928541  7806 raft_consensus.cc:2889] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:22.919140 16694 raft_consensus.cc:3055] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:29:22.927951  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 1.39095e+06us
I20260709 12:29:22.910367 16715 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:22.915760 16602 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:29:22.850687 16668 consensus_queue.cc:1059] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
W20260709 12:29:22.946979  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.31408e+06us
I20260709 12:29:22.930404 16637 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:22.931669 16581 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:22.875008 16704 consensus_queue.cc:237] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:22.932015 16694 raft_consensus.cc:740] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:22.941169 16715 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:22.970794 16700 consensus_queue.cc:1059] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:22.969748  8407 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 28.
W20260709 12:29:22.984726  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 73649.4us
W20260709 12:29:22.987468  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53752.9us
I20260709 12:29:22.953759 16614 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:22.988531  7702 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:22.954349 16691 raft_consensus.cc:697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:22.957098 16671 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:22.957333 16602 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:22.992264  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.010926  8415 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:22.995826  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:22.999971  7775 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:23.016601  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:22.986259 16715 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:23.027813  7729 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 17. Current term is 18. Ops: []
I20260709 12:29:22.998324  7700 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:22.985986 16562 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.032461  7731 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
W20260709 12:29:23.034246  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45815.2us
I20260709 12:29:23.023182 16691 consensus_queue.cc:237] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.024647 16725 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:23.025462  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:22.918828 16728 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.006122  7733 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:22.989573 16620 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:23.037487  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.035423  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:23.001550 16618 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
W20260709 12:29:23.050606 16703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.009s	sys 0.024s
I20260709 12:29:23.050879 16508 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.007424 16613 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.065656 16613 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.067533 16663 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:23.068279 16613 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:22.976408 16581 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.070632  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:23.075574 16581 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.076599 16377 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:23.077818 16377 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:23.079968  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:23.042843 16579 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:23.085490  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:23.084619  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:23.054297 16550 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:23.088354  7733 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:29:23.064785 16703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.010s	sys 0.024s
I20260709 12:29:23.091935  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:23.095836  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178405us
W20260709 12:29:23.082998 16665 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 12:29:23.055253 16711 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:23.059422 16500 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.073295 16637 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:23.102077  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.142542  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.109314  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.110461  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.143463  7657 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:23.092645 16579 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.131762  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:23.112984  7811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45832 (request call id 3763) took 3248 ms (client timeout 2161 ms). Trace:
I20260709 12:29:23.103814  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.099043  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:23.110216  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.110229  7662 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:23.086844 16669 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:23.083542 16725 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:23.138660  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 173877us
I20260709 12:29:23.185258 16580 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:23.186172 16735 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:29:23.186113 16580 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.167372 16711 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:23.187942 16632 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 12:29:23.188498 16632 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25.
I20260709 12:29:23.188922 16580 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.189591 16580 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:23.190081 16580 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:23.191717  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1893) took 1379 ms. Trace:
I20260709 12:29:23.192198  7734 rpcz_store.cc:276] 0709 12:29:21.812091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.851540 (+ 39449us) service_pool.cc:224] Handling call
0709 12:29:21.851904 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:22.954952 (+1103048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:22.954995 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:22.955797 (+   802us) log.cc:844] Serialized 26 byte log entry
0709 12:29:22.987110 (+ 31313us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:22.987157 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:22.987267 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:23.191351 (+204084us) raft_consensus.cc:1685] finished
0709 12:29:23.191379 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:23.191631 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1135910,"raft.queue_time_us":1104438,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2154,"thread_start_us":30656,"threads_started":1,"wal-append.queue_time_us":85282}
I20260709 12:29:23.141620 16637 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.194832 16637 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:23.139205 16581 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:23.195660 16581 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.196072 16581 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:23.199859  8415 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:23.140774 16500 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.201917  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.202669  8407 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:23.203347 16500 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.204464 16718 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:23.205117 16717 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.206416 16717 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:23.109740 16665 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.211899 16665 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:23.213888 16734 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:29:23.213482  7553 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 19, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 19 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:23.216241 16679 consensus_meta.cc:287] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.673s	user 0.015s	sys 0.000s
I20260709 12:29:23.216044  7553 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:23.216763 16679 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.218016  7553 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:23.179121 16660 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.223152 16660 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.225659 16660 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.179984 16687 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.227909 16687 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.230239  7657 raft_consensus.cc:1240] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:23.230553 16687 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.231412 16613 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.231755  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.232053 16613 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.232975  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:23.188181 16711 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.233772  7655 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:23.235800 16663 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.235888 16711 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.236670 16711 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.236502 16663 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.237522  7737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4007) took 2113 ms (client timeout 1624 ms). Trace:
I20260709 12:29:23.093863 16618 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:29:23.094313 16550 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.130815  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:23.238023  7737 rpcz_store.cc:269] 0709 12:29:21.124605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.124757 (+   152us) service_pool.cc:224] Handling call
0709 12:29:23.237297 (+2112540us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400419454. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:23.237447 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3613966592}
I20260709 12:29:23.238802  7661 raft_consensus.cc:2393] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:23.059706 16671 consensus_queue.cc:1059] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:29:23.142153 16688 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.240144 16671 raft_consensus.cc:3055] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:23.097975 16727 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.108587 16602 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.240876 16671 raft_consensus.cc:740] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:23.241160 16727 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.241602  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.242252  7659 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:23.241905 16671 consensus_queue.cc:260] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.148356 16324 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:29:23.096661 16703 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:23.243433 16671 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:23.243646 16727 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.150476 16714 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.244283 16703 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.244477 16714 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.143638 16666 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
W20260709 12:29:23.146632 16730 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:29:23.096390 16689 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:29:23.154093  7733 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:23.159446 16614 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.248646 16698 raft_consensus.cc:3055] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:23.063043 16620 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.249261 16698 raft_consensus.cc:740] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:23.249543 16550 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:23.251616 16620 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.251802 16709 raft_consensus.cc:3055] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:23.043357 16694 consensus_queue.cc:260] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.252388 16709 raft_consensus.cc:740] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:23.253126 16694 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:23.253759 16709 consensus_queue.cc:260] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.154284  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:23.165704 16702 raft_consensus.cc:3055] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:23.246986 16714 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.250404 16698 consensus_queue.cc:260] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.255797 16702 raft_consensus.cc:740] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:23.257085 16698 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:23.257298 16702 consensus_queue.cc:260] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.238924 16663 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.261183 16550 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:23.174777  7811 rpcz_store.cc:269] 0709 12:29:19.864179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.060881 (+196702us) service_pool.cc:224] Handling call
0709 12:29:23.112667 (+3051786us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b5400373cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:23.112875 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863096576}
W20260709 12:29:23.180369  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48711.2us
I20260709 12:29:23.240069 16688 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.240530 16377 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.205142 16725 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.244910  8409 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:23.249120 16614 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.161190 16728 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.171010  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.053778 16531 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:29:23.252434 16620 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.176916  7653 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:23.294405  7736 raft_consensus.cc:1240] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
W20260709 12:29:23.285835  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.270607  8411 raft_consensus.cc:1240] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: [24.20-26.21]
I20260709 12:29:23.265501  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:23.269416  8415 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:29:23.339495  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 63991.2us
I20260709 12:29:23.282536  7777 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:23.363294  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 97889.4us
I20260709 12:29:23.179266 16579 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:23.277982  7662 raft_consensus.cc:1240] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:23.237247 16711 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.366707  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:23.371754  7656 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:23.294130 16709 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:23.335170  8407 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:23.258982 16702 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:23.238075 16538 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:23.267405 16580 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.269723 16679 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:23.259727 16663 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.367756  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.284842  7812 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:23.388972  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:23.405836  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.406366 16550 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.407513 16688 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.412882  7655 raft_consensus.cc:2468] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:23.241087 16618 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:23.366895  8379 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:23.420946  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54149.9us
I20260709 12:29:23.421422 16618 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.392827  7653 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 28.
I20260709 12:29:23.423846  8379 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:23.425793 16514 raft_consensus.cc:2804] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:23.426398 16514 raft_consensus.cc:697] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:23.427316 16538 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.427506 16514 consensus_queue.cc:237] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.430186 16538 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.431309 16704 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:23.432286 16704 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:23.433768 16704 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.399952  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 119167us
I20260709 12:29:23.407317  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.372907  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:23.503163  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.504266  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.510401  7730 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:23.511271  8407 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 18: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:23.508137  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.391981 16599 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:29:23.512462  7732 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:23.375142 16689 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:23.387248 16728 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.522401  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:23.520473  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:23.515043  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 416074us
I20260709 12:29:23.358860 16377 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.530454  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 157620us
I20260709 12:29:23.532084  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.534415  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.394272 16324 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:23.541162  7627 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.727s (SENT)
I20260709 12:29:23.551926  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:23.424515 16665 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:23.560129  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.560765  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:23.510453  7777 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:23.569100  7656 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:23.571188  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.572064  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.572933  7805 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:23.577061  7658 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:29:23.577260  7655 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:23.512858 16351 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.576337  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:23.580873  7627 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.745s (SENT)
W20260709 12:29:23.539631 16693 consensus_meta.cc:287] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.727s	user 0.008s	sys 0.008s
I20260709 12:29:23.388334 16620 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.587242  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76834.3us
I20260709 12:29:23.592193  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:23.552318 16738 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:29:23.540940 16610 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:23.424696 16701 consensus_queue.cc:1059] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.154s
I20260709 12:29:23.611742  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.612219  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.623680  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.624779  7661 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:23.616638  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.626333  7657 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:23.550809 16706 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:23.617076  7662 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:23.621171  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.631718  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.563954 16689 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:23.627066  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.636790  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.637617  7661 raft_consensus.cc:2393] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:23.623718  7655 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:23.638329  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:23.638659  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.576205 16665 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.638329  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.639781 16665 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:23.639945  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:23.582314 16719 consensus_meta.cc:287] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.791s	user 0.013s	sys 0.003s
I20260709 12:29:23.624019  7659 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:23.520473 16660 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:29:23.525262 16531 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 12:29:23.632588  7660 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
W20260709 12:29:23.632932  8379 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.582556 16718 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:23.582177 16550 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:23.514816 16663 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.632664  7658 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:29:23.567965 16715 consensus_meta.cc:287] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.000s
I20260709 12:29:23.647143 16715 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.650106 16715 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.582521 16668 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.651209 16668 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.579867 16324 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.607851 16695 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:23.551208 16728 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.613340 16722 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:29:23.663335  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.654390  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:23.551656 16637 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.588922 16714 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:23.698464 16711 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.699962  7653 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:23.701184 16620 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.635675 16538 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.702273 16538 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.703262  8408 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1746) took 1134 ms. Trace:
I20260709 12:29:23.703964 16714 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.704584 16714 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.705544 16538 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.707424 16695 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:23.707265 16595 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.707834  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.708559 16595 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:23.709179  8409 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:23.676528 16377 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.631060 16706 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.677860 16660 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:23.613691 16351 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.685839 16581 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.688241 16728 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.691910 16689 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.582979 16669 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.078s
I20260709 12:29:23.914789 16351 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.915841 16669 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:23.916353 16669 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:23.917675 16669 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.695947  7732 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:23.697135 16637 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.698073  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:23.703718  8408 rpcz_store.cc:276] 0709 12:29:22.568637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:22.568808 (+   171us) service_pool.cc:224] Handling call
0709 12:29:22.636954 (+ 68146us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b58001e4898. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:29:23.703046 (+1066092us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400631d54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:23.703167 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951234304}
I20260709 12:29:23.919689  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.920229  8409 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:23.921836  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:23.717810  7811 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:23.923525  8405 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:23.923796  8406 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:23.718456  7812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1448) took 1158 ms. Trace:
I20260709 12:29:23.924522  7812 rpcz_store.cc:276] 0709 12:29:22.560246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:22.560888 (+   642us) service_pool.cc:224] Handling call
0709 12:29:23.718217 (+1157329us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54003f3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:23.718381 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455120768}
I20260709 12:29:23.720577  7810 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:23.925527  7735 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:23.926527  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.919191 16669 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:23.711599  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50746us
I20260709 12:29:23.662663 16610 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.708565 16695 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:23.738611  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.740487 16668 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.706971 16714 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:23.690987 16324 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:23.904835 16538 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:23.909173 16618 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.908495  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:23.739549  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:23.799516 16703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.026s	sys 0.034s
I20260709 12:29:23.741247 16706 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:23.787873  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 196518us
I20260709 12:29:23.927141 16613 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:23.929405  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:23.899912 16709 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.509s	user 0.018s	sys 0.003s
I20260709 12:29:23.930089 16728 consensus_queue.cc:1059] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:29:23.907418 16747 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.910126 16710 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:29:23.962987 16747 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.964243 16728 raft_consensus.cc:3055] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:23.966190 16747 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:23.970829  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:23.997095  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:23.998917  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.998682 16715 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:24.000283 16715 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:24.002890 16715 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:23.964963 16728 raft_consensus.cc:740] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:24.009387 16720 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:24.010150 16720 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:23.742390 16660 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:23.910611 16697 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:24.013077 16728 consensus_queue.cc:260] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.912388  7803 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 21. (term mismatch)
I20260709 12:29:24.016937  7803 pending_rounds.cc:85] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 20
I20260709 12:29:23.909749  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:24.017470  7803 pending_rounds.cc:107] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 27.21
I20260709 12:29:23.952145  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:23.754045 16579 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:23.954715 16703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.027s	sys 0.043s
I20260709 12:29:23.950938 16618 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.020004  7809 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:23.920959  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:24.020784 16579 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.021822  7736 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:23.931129  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:24.017288 16728 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:23.950664 16538 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.948062  8408 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: [27.21-27.21]
I20260709 12:29:24.011366 16720 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.014035  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:23.951589 16637 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:23.952553 16665 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:23.999050  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 332134us
I20260709 12:29:23.933779 16610 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.032742  8411 raft_consensus.cc:2368] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:24.033100 16720 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:23.955260  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:24.034385  8409 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:23.909853 16753 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.018045  7803 raft_consensus.cc:2889] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:24.036433 16637 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:24.035732 16753 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:23.986441 16714 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:23.905808 16581 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:23.949005 16732 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.907s	user 0.003s	sys 0.000s
I20260709 12:29:23.955588 16614 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:23.800377  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 216694us
I20260709 12:29:23.950199  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:23.950667  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:24.031823  7805 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:23.940434  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:23.936677 16616 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.038002  7803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1450) took 1370 ms. Trace:
I20260709 12:29:24.043799 16610 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.052471  8407 raft_consensus.cc:2368] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:29:24.053092 16616 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:24.052915 16610 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:24.057439  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:24.058764  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.040325  7735 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:24.063421 16703 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.052577  7803 rpcz_store.cc:276] 0709 12:29:22.667166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:22.667350 (+   184us) service_pool.cc:224] Handling call
0709 12:29:22.667895 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:24.035698 (+1367803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:24.037578 (+  1880us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003f4154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:24.037906 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861111552}
W20260709 12:29:23.912688 16717 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.706s	user 0.014s	sys 0.000s
I20260709 12:29:24.064394 16703 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.065847  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:24.067121 16719 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.067313 16714 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:24.067764 16719 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.069031  7807 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:24.071954 16614 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.072953 16614 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.076318 16665 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519)
I20260709 12:29:23.976866  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:24.086280  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.089214 16723 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:24.116840 16660 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:23.902446 16377 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:23.911706 16620 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:24.124675  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.124171 16620 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.017812  7731 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
W20260709 12:29:24.134838  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.142381 16679 raft_consensus.cc:3055] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:24.144886 16679 raft_consensus.cc:740] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:24.145872 16618 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:24.146574  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 169818us
I20260709 12:29:24.146070 16679 consensus_queue.cc:260] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.147042 16753 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.148244 16679 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:24.158474  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:24.159667  7811 raft_consensus.cc:3060] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:24.163996  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.165372  7809 raft_consensus.cc:2410] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 24 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:29:24.167680  7809 raft_consensus.cc:3060] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:24.171516  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 240468us
W20260709 12:29:24.172555  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.173028 16599 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:24.173945 16599 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:24.174331 16599 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:24.174381 16610 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.014866 16697 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:24.176469  7627 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.084s (SENT)
I20260709 12:29:24.177085  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:24.179800 16697 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.179852 16351 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:29:24.181217  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 229160us
I20260709 12:29:24.222637  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.224066  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:24.224368  7653 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:24.145900 16663 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:24.184165  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 150658us
I20260709 12:29:24.224572  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:24.228046  7730 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:24.186448 16614 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.187057 16719 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.187677 16753 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.943576  7553 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:24.119693 16377 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:24.171516  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.240075  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:24.190819 16538 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.193512 16703 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:24.252672  7809 raft_consensus.cc:1275] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:24.253119  8406 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
W20260709 12:29:24.195307 16580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.015s
W20260709 12:29:24.195701  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 129985us
I20260709 12:29:24.195811  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:24.198705  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.001s	sys 0.020s
W20260709 12:29:24.255264  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.001s	sys 0.020s
W20260709 12:29:24.191429  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 290120us
I20260709 12:29:24.255626  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:24.254046  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.257575  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:24.258757  7809 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:29:24.206020  7803 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:24.206226  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1451) took 1503 ms. Trace:
I20260709 12:29:24.259675  8409 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:24.259992  7804 rpcz_store.cc:276] 0709 12:29:22.702374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:22.702558 (+   184us) service_pool.cc:224] Handling call
0709 12:29:22.703010 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:22.753908 (+ 50898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:22.753940 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:22.777137 (+ 23197us) log.cc:844] Serialized 26 byte log entry
0709 12:29:23.709881 (+932744us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b180046d0e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b 00007fc001ac1d23 00007fc001ac20d1 00007fc001edda0f
0709 12:29:23.909747 (+199866us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:23.909801 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:23.909912 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:24.022422 (+112510us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b180046d0e0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:24.205942 (+183520us) raft_consensus.cc:1685] finished
0709 12:29:24.205964 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:24.206119 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75737,"raft.queue_time_us":52654,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":21963,"spinlock_wait_cycles":50445440,"thread_start_us":133805,"threads_started":1,"wal-append.queue_time_us":1107576}
W20260709 12:29:24.186187  7550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 185) took 50769 ms (client timeout 15000 ms). Trace:
W20260709 12:29:24.209332  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.210043  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:24.211679  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:24.212610  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:24.219594  7627 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.177s (SENT)
I20260709 12:29:24.232677  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:24.273661  7812 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:24.274109  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54585.3us
I20260709 12:29:24.275923  7810 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:24.240902  7811 raft_consensus.cc:2468] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:24.244699  7730 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 18. Current term is 19. Ops: []
I20260709 12:29:24.248744  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:24.250800  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.277504  7808 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:24.250285  7553 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:24.258636  8379 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:24.204002  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.263690  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:24.264549  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:24.264897  7700 leader_election.cc:400] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:24.265228  8415 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:29:24.208556  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:24.234093 16660 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:24.258636 16580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.010s	sys 0.015s
I20260709 12:29:24.352857  7804 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:24.275494  7659 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:24.355881  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:24.233400  7731 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:24.299667  8409 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:24.306603  7809 raft_consensus.cc:1240] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:24.308004  7732 raft_consensus.cc:1217] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Deduplicated request from leader. Original: 24.19->[26.20-27.21]   Dedup: 26.20->[27.21-27.21]
I20260709 12:29:24.297137  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.279979  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:24.359900  7734 raft_consensus.cc:3060] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:29:24.279305  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:24.361943  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 113282us
W20260709 12:29:24.360324  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 164633us
I20260709 12:29:24.372284  7732 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: [26.20-27.21]
I20260709 12:29:24.363313  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:24.325610  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 58367.7us
I20260709 12:29:24.298007  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:24.377928  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.378345  8406 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 27.
I20260709 12:29:24.378814  7811 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:24.332263  7700 leader_election.cc:403] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:24.381994  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:24.385444  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.385671  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4031) took 1090 ms. Trace:
I20260709 12:29:24.386296  7811 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:24.386435  7732 rpcz_store.cc:276] 0709 12:29:23.295472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.021505 (+726033us) service_pool.cc:224] Handling call
0709 12:29:24.307810 (+286305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:24.373083 (+ 65273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:24.385309 (+ 12226us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540037cb50. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:24.385542 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657428352}
I20260709 12:29:24.390516  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.391463  7811 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:24.393219  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.266176 16703 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:24.335170  7806 raft_consensus.cc:2368] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25.
I20260709 12:29:24.307883  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1908) took 1054 ms. Trace:
I20260709 12:29:24.268908  7733 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:24.336169 16665 raft_consensus.cc:2804] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:24.407094 16500 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:24.336364 16620 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.337044  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.414995 16500 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:24.415514 16500 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:24.415715  7808 raft_consensus.cc:2393] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:24.173270 16706 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.416554  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4024) took 1150 ms. Trace:
I20260709 12:29:24.417114  7733 rpcz_store.cc:276] 0709 12:29:23.266021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:23.727100 (+461079us) service_pool.cc:224] Handling call
0709 12:29:24.416311 (+689211us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54003b47d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:24.416458 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666296704}
I20260709 12:29:24.392776 16618 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:24.418468 16706 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.418776 16618 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:24.419371 16618 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:24.391283 16754 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:24.421218 16754 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:24.421651 16754 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:24.424474 16377 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.337913  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:24.427335 16545 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:24.295446  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:24.343699  7812 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:24.429214 16618 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:24.345257  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 64040.8us
I20260709 12:29:24.345790 16666 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:24.432153 16618 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.252528 16747 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:24.248384 16753 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.394091  7803 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:24.435381 16753 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:24.395025  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 130285us
I20260709 12:29:24.441721  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:24.397670  7734 raft_consensus.cc:2468] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:24.442574 16500 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.403472  8406 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:24.408691  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:24.449615  8406 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:29:24.403983  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:24.405225  8415 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:24.451397  8413 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:24.390657  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:24.271636 16719 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:24.252771 16637 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.123s
I20260709 12:29:24.411267  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:24.463896  7811 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:24.375658 16695 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:24.462265  8380 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:24.453918 16711 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:24.456555 16595 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:24.459859  7810 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 27.
I20260709 12:29:24.450146  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:24.268424 16351 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.138s
W20260709 12:29:24.474593  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:24.475841  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.476812  8380 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:24.480821  7805 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:24.484032  7775 leader_election.cc:400] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:29:24.485782  7775 leader_election.cc:403] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:29:24.454375 16665 raft_consensus.cc:697] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:24.495258 16714 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:29:24.477463 16595 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:24.453681 16324 consensus_queue.cc:1048] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:29:24.483731 16666 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:24.487718 16719 raft_consensus.cc:2764] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:29:24.438630 16754 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.487319 16697 raft_consensus.cc:2804] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:24.467301 16747 consensus_queue.cc:1059] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:29:24.478112 16500 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.513337 16595 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:24.672116  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1909) took 1009 ms. Trace:
W20260709 12:29:24.275539  7550 rpcz_store.cc:269] 0709 12:28:33.439158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:33.543056 (+103898us) service_pool.cc:224] Handling call
0709 12:29:24.185858 (+50642802us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:12.539718 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:12.769144 (+229426us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:12.780848 (+ 11704us) write_op.cc:435] Acquired schema lock
0709 12:29:12.787272 (+  6424us) tablet.cc:662] Decoding operations
0709 12:29:12.850086 (+ 62814us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:12.850244 (+   158us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:12.850293 (+    49us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:29:12.853125 (+  2832us) tablet.cc:701] Row locks acquired
0709 12:29:12.853194 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:29:12.853483 (+   289us) write_op.cc:270] Start()
0709 12:29:12.853684 (+   201us) write_op.cc:276] Timestamp: P: 1783600152853391 usec, L: 0
0709 12:29:12.853705 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:29:12.854896 (+  1191us) log.cc:844] Serialized 7432 byte log entry
0709 12:29:13.737673 (+882777us) op_driver.cc:464] REPLICATION: finished
0709 12:29:13.846152 (+108479us) write_op.cc:301] APPLY: starting
0709 12:29:13.846301 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:14.100278 (+253977us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:14.100596 (+   318us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:14.458211 (+357615us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:14.458743 (+   532us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:29:14.458810 (+    67us) write_op.cc:312] APPLY: finished
0709 12:29:14.550548 (+ 91738us) log.cc:844] Serialized 172 byte log entry
0709 12:29:14.985278 (+434730us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc001abc20b
0709 12:29:15.476113 (+490835us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:15.514314 (+ 38201us) write_op.cc:454] Released schema lock
0709 12:29:15.514411 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:15.515086 (+   675us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:15.515120 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1176925,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":108110,"apply.run_cpu_time_us":75756,"apply.run_wall_time_us":1668884,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83547,"cfile_init":1,"lbm_read_time_us":1568,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"num_ops":19,"prepare.queue_time_us":353528,"prepare.run_cpu_time_us":124030,"prepare.run_wall_time_us":540721,"raft.queue_time_us":377708,"raft.run_cpu_time_us":18624,"raft.run_wall_time_us":169522,"replication_time_us":869300,"spinlock_wait_cycles":16882560,"thread_start_us":1500050,"threads_started":5,"wal-append.queue_time_us":1178664,"wal-append.run_cpu_time_us":30537,"wal-append.run_wall_time_us":2336316}]]}
I20260709 12:29:24.647115 16754 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:24.678725  7550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 308) took 20415 ms (client timeout 15000 ms). Trace:
I20260709 12:29:24.334414  7656 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
W20260709 12:29:24.687063  7550 rpcz_store.cc:269] 0709 12:29:04.262889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.678169 (+20415280us) service_pool.cc:211] Skipping call since client already timed out
0709 12:29:24.678385 (+   216us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:29:24.411841  7729 rpcz_store.cc:276] 0709 12:29:23.253378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:23.262917 (+  9539us) service_pool.cc:224] Handling call
0709 12:29:23.361301 (+ 98384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:23.374156 (+ 12855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:23.374211 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:23.391836 (+ 17625us) log.cc:844] Serialized 52 byte log entry
0709 12:29:24.255732 (+863896us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:29:24.294552 (+ 38820us) log.cc:844] Serialized 14 byte log entry
0709 12:29:24.307470 (+ 12918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:24.307564 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:24.307608 (+    44us) raft_consensus.cc:1685] finished
0709 12:29:24.307620 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:24.307815 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38305,"prepare.queue_time_us":91448,"raft.queue_time_us":39516,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":10897,"spinlock_wait_cycles":9284992,"thread_start_us":699393,"threads_started":1,"wal-append.queue_time_us":671153}
W20260709 12:29:24.615726  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 131608us
I20260709 12:29:24.618767  7728 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
W20260709 12:29:24.550081 16715 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:29:24.718401 16715 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:24.547482 16728 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.006s
I20260709 12:29:24.525774 16599 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.530843 16665 consensus_queue.cc:237] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.628973 16779 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.675356  7735 rpcz_store.cc:276] 0709 12:29:23.662170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.062570 (+400400us) service_pool.cc:224] Handling call
0709 12:29:24.062904 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:24.063130 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:29:24.465745 (+402615us) log.cc:844] Serialized 14 byte log entry
0709 12:29:24.671070 (+205325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:24.671263 (+   193us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:24.671403 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:24.671536 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:24.671824 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78667,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":682,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":258,"thread_start_us":563717,"threads_started":2,"wal-append.queue_time_us":202527}
I20260709 12:29:24.725257 16779 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.728694 16779 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:24.630455  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 142169us
I20260709 12:29:24.730618  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:24.732470 16599 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.733285 16778 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:24.733973 16778 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:24.640466  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:24.734514 16778 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:24.645920 16697 raft_consensus.cc:697] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:24.594303  7553 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 20, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 20 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:24.648864  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 258142us
I20260709 12:29:24.870824  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:24.649925  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:24.876053  7653 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:24.651429 16377 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:24.879594 16377 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:24.880335 16377 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:24.653378 16581 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.882791 16581 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.657802 16538 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.659184 16747 consensus_queue.cc:1243] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:29:24.886549 16747 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:24.886086 16778 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.612223  7654 raft_consensus.cc:2368] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:24.678964 16602 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:24.888681 16602 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.679700 16613 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:24.699517  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.703081 16351 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:24.894699 16351 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.896553 16579 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:24.897774 16579 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.707564 16720 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:24.708570  7626 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:24.903986  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 195822us
I20260709 12:29:24.905179  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:24.906692  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:24.843169  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46215.3us
W20260709 12:29:24.908581  7728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1898) took 1911 ms (client timeout 2078 ms). Trace:
I20260709 12:29:24.857491  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:24.940218  7730 raft_consensus.cc:3055] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:24.859053  7731 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:25.591668  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.592621  7731 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:25.593999  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.594725  7731 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:24.859980  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.033241  7733 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:29:24.663182  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.601830  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.602008  7734 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:24.858160  7735 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:29:25.605304  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.016045  7553 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 18, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 18 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:24.787853  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:24.723518  7775 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:25.613564 16710 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:25.014782  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 121066us
I20260709 12:29:24.689375  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:24.743549  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:25.057951 16351 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.047516 16538 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.648880 16538 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.013731 16581 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.048834 16602 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.843384 16660 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 12:29:25.027720 16679 consensus_meta.cc:287] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.878s	user 0.013s	sys 0.013s
I20260709 12:29:24.737759 16697 consensus_queue.cc:237] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.918159 16720 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:24.752059 16599 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:25.667610 16599 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:24.826133 16666 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.887111 16747 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:24.842222 16698 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.679984 16698 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.671614 16798 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:25.681849 16798 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.682479 16698 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:24.745260 16715 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.750574 16723 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:25.685222  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:25.685876 16723 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:25.686403 16723 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:24.906764 16728 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:25.688527 16728 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:25.689368 16728 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:24.780071 16754 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.051410 16579 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:24.840559 16595 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:24.648780 16616 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:25.708103 16616 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:25.708601 16616 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:25.087884 16377 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.714450 16377 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.005936 16778 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:25.308722  7660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1666) took 1045 ms. Trace:
W20260709 12:29:25.320277  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 413763us
I20260709 12:29:25.322322 16792 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:24.923091  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3317) took 1285 ms. Trace:
W20260709 12:29:25.411372  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:25.418289  7702 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46548 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 12:29:25.433326 16637 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:25.435846 16793 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.448550 16794 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.449213  7700 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46548 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3710)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3715)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3738)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3717)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4058)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4057)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3746)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3729)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3718)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3722)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4056)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3724)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4054)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4055)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1916)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1917)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3723)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3731)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3744)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3748)
I20260709 12:29:25.482220  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4036) took 2093 ms. Trace:
I20260709 12:29:25.716974  7654 rpcz_store.cc:276] 0709 12:29:23.637867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:23.638011 (+   144us) service_pool.cc:224] Handling call
0709 12:29:24.922550 (+1284539us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54004353d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:24.922921 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235710848}
I20260709 12:29:25.494465 16545 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.497814  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:25.567898 16779 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:25.577894  7728 rpcz_store.cc:269] 0709 12:29:22.997426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:22.997591 (+   165us) service_pool.cc:224] Handling call
0709 12:29:24.908184 (+1910593us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001450d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:24.908465 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3592688384}
I20260709 12:29:25.582984  7730 raft_consensus.cc:740] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:25.728236  7730 consensus_queue.cc:260] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:25.588260  7653 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.711s	user 0.031s	sys 0.007s
I20260709 12:29:25.600009 16753 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.729537  7653 raft_consensus.cc:2468] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:25.729642  7730 raft_consensus.cc:3060] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:25.730238  7653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40408 (request call id 1669) took 1081 ms. Trace:
I20260709 12:29:25.729954 16753 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.730710  7653 rpcz_store.cc:276] 0709 12:29:24.648901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.649215 (+   314us) service_pool.cc:224] Handling call
0709 12:29:25.730170 (+1080955us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:25.731951  7778 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46548 dropped due to backpressure: service queue is full with 50 items [suppressed 35 similar messages]
I20260709 12:29:25.600667  7732 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:25.733196 16753 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.601729  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.734143  7731 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:25.603510  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:24.862243  7737 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:29:25.739194  7737 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:29:25.738739 16545 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.738833  7733 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:25.610561  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.740820  7734 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:25.741417 16545 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.614337 16710 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.744064 16637 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.745918 16710 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.746649 16710 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:25.619756 16709 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.746562 16728 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.747550 16709 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.748682 16715 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:25.749979 16709 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:25.750667 16709 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:29:25.625435  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 936566us
I20260709 12:29:25.746826 16637 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.722681 16377 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.632157 16799 raft_consensus.cc:2749] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:25.756496 16616 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.638808 16324 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:25.758208 16324 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:25.758805 16324 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:25.759771 16616 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.651775  7729 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:25.719452 16792 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:25.760661 16792 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:25.666285 16720 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:25.662724  7735 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:25.677578 16666 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:25.641180  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 923948us
I20260709 12:29:25.763002 16720 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:25.764487  7775 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:25.766777  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.767534  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.767926  7662 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:25.768508  7658 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:25.678532 16747 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:25.692049 16798 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.692437 16754 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.694818 16723 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.778486 16723 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:25.778713 16666 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:25.779493  8414 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:25.722069  7736 rpcz_store.cc:276] 0709 12:29:23.388237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.026030 (+637793us) service_pool.cc:224] Handling call
0709 12:29:24.026416 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:24.126872 (+100456us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:29:24.249947 (+123075us) log.cc:844] Serialized 14 byte log entry
0709 12:29:24.632099 (+382152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:24.632259 (+   160us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:24.632401 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:25.481507 (+849106us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54002aba54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:25.481591 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:25.481923 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56812,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":170140,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":293,"spinlock_wait_cycles":230414592,"thread_start_us":336522,"threads_started":2,"wal-append.queue_time_us":283872,"wal-append.run_cpu_time_us":3933,"wal-append.run_wall_time_us":1006414}
I20260709 12:29:25.739560  7737 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:29:25.781205  7737 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:25.711529 16595 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:25.725634 16793 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:25.735213  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 105359us
I20260709 12:29:25.608318  7553 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:25.628343  8409 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:25.757014  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.628285 16668 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.795747  7624 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:25.796088  7627 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.521s (SENT)
I20260709 12:29:25.796787  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:25.797021 16753 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:25.797578 16753 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:25.797880 16602 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:25.749084 16728 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.759317 16794 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.684561 16599 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.769973  7655 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:25.726719 16779 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.776736  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1602) took 1161 ms. Trace:
I20260709 12:29:25.777026 16754 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:25.776458 16778 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:25.716486  7660 rpcz_store.cc:276] 0709 12:29:24.262937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.265045 (+  2108us) service_pool.cc:224] Handling call
0709 12:29:24.265357 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:24.265708 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:29:24.898635 (+632927us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b1800438a60. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee17e07 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee32455 00007fc00591a3c7 00007fc001f2c9a7 00007fc001f3c0bf 00007fc001f3c038
0709 12:29:25.306666 (+408031us) log.cc:844] Serialized 14 byte log entry
0709 12:29:25.308094 (+  1428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:25.308179 (+    85us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:25.308270 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:25.308366 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:25.308556 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":692990,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":13953,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":27825,"spinlock_wait_cycles":38752640,"thread_start_us":408424,"threads_started":2,"wal-append.queue_time_us":1317}
I20260709 12:29:25.798065 16753 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:25.804600  7656 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:25.785315 16799 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:29:25.807453 16798 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.807885 16793 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:25.808525 16666 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:25.809772  8415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:25.810405  7730 raft_consensus.cc:1275] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:29:25.773813 16377 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.798699 16715 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:29:25.782943 16545 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:25.828329  7553 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:25.816277  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:25.900163  7662 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:25.816903  7659 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:25.817139  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:25.818014  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.790028 16637 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:25.821365 16720 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:25.822785 16602 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:25.717803  7702 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46548 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3738)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3717)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4058)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4057)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3746)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3729)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3718)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3722)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4056)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3724)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4054)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4055)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1916)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1917)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3723)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3731)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3744)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3748)
I20260709 12:29:25.877898 16709 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:25.882478 16800 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
W20260709 12:29:25.879074  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 112225us
W20260709 12:29:25.844882  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48864.7us
I20260709 12:29:25.887416  8413 rpcz_store.cc:276] 0709 12:29:24.616173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.639261 (+ 23088us) service_pool.cc:224] Handling call
0709 12:29:25.776239 (+1136978us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540046cb54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:25.776594 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387177344}
I20260709 12:29:25.894315  7731 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:25.787005 16710 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.903147  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.903990  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:25.924865 16710 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:25.925145  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 52881us
I20260709 12:29:25.925838  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.926262 16754 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:25.926746 16754 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:25.927484 16754 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:25.927616  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.928488  7732 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:25.929929  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.930018 16720 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.930627  7732 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:25.931838  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.932843  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:25.933669  7807 raft_consensus.cc:2376] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25. Re-sending same reply.
I20260709 12:29:25.934012  7656 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:29:25.934219  7735 raft_consensus.cc:2368] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28.
I20260709 12:29:25.935902  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:25.936856  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.935210  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:25.935252  7803 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:29:25.937651  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.941619  7737 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:25.901876 16778 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:25.913201  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:25.931049  7657 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:25.904927  7661 raft_consensus.cc:2368] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:29:25.904664  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.906157  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.905757 16666 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.907661  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:25.910583  8409 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:25.898460  7730 pending_rounds.cc:85] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 15
I20260709 12:29:25.824661  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.922130  8411 raft_consensus.cc:1240] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:29:25.922420 16545 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.795754 16668 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.916664  8415 raft_consensus.cc:2468] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:25.922163  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3712) took 1046 ms. Trace:
I20260709 12:29:26.038268  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:26.294831  8409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40594 (request call id 1606) took 1559 ms (client timeout 1614 ms). Trace:
I20260709 12:29:25.943950  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
W20260709 12:29:26.295367  8409 rpcz_store.cc:269] 0709 12:29:24.735683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.742876 (+  7193us) service_pool.cc:224] Handling call
0709 12:29:26.294759 (+1551883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.296326  7730 pending_rounds.cc:107] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:29:26.296746  7730 raft_consensus.cc:2889] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:26.297701 16663 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:29:26.079893  7803 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:26.298445  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1914) took 1637 ms. Trace:
I20260709 12:29:25.934383  7656 pending_rounds.cc:85] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting all ops after (but not including) 15
I20260709 12:29:25.898329 16324 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:25.949070  7553 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 20, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:25.947969  7655 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:26.302884  7733 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:26.304056  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.305130  7733 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:26.261895  7737 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:26.305831  7655 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:29:25.972826  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.944353  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.197356  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.300002  7656 pending_rounds.cc:107] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:29:26.297466  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.944762  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:25.937651  7735 raft_consensus.cc:3060] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:26.310889  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3717) took 1291 ms. Trace:
I20260709 12:29:26.294147  7732 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:26.020447  7662 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:26.315129  7737 rpcz_store.cc:276] 0709 12:29:25.020096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.917103 (+897007us) service_pool.cc:224] Handling call
0709 12:29:26.310801 (+393698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.317014  7662 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:25.909055 16637 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.318807 16668 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.312152  7729 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:29:26.327253  8414 raft_consensus.cc:1240] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:29:26.311183  7734 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:26.292033  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45499.9us
I20260709 12:29:26.312548  7656 raft_consensus.cc:2889] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:26.222119  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.293574  7731 rpcz_store.cc:276] 0709 12:29:25.014128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.601361 (+587233us) service_pool.cc:224] Handling call
0709 12:29:25.922036 (+320675us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:26.056084  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:25.951367 16709 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.320137 16779 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.320530 16637 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:26.321357  7653 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:26.253996  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:26.313031  8379 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:26.319152  7662 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:29:26.173331  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:26.298861  7730 rpcz_store.cc:276] 0709 12:29:24.661460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.664141 (+  2681us) service_pool.cc:224] Handling call
0709 12:29:24.664763 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:24.939823 (+275060us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400169f20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001e99dc0 00007fc001f241c0 00007fc001f25231 00007fc001f21d8f 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c
0709 12:29:26.297368 (+1357545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:26.298383 (+  1015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557308160}
I20260709 12:29:26.324658  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:25.943742  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.322880  7736 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:26.330811  8405 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:26.331650  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:26.144249  7590 debug-util.cc:398] Leaking SignalData structure 0x7b0800353d00 after lost signal to thread 6454
W20260709 12:29:26.144578  6456 debug-util.cc:398] Leaking SignalData structure 0x7b08002febc0 after lost signal to thread 7775
I20260709 12:29:26.302227 16324 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:25.898759 16798 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.332487  7728 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23. Re-sending same reply.
I20260709 12:29:26.393697  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.416134  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3709) took 1542 ms. Trace:
I20260709 12:29:26.332255  7553 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 18, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 18 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:26.418303  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08001c2600 after lost signal to thread 7593
I20260709 12:29:26.347609  7805 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 25: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:26.371922  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.418689  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.419121  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.416608  7732 raft_consensus.cc:2468] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:26.419800  7730 raft_consensus.cc:2376] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
W20260709 12:29:26.420063  7590 debug-util.cc:398] Leaking SignalData structure 0x7b080011d0a0 after lost signal to thread 7686
I20260709 12:29:26.416622  7733 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
W20260709 12:29:26.420925  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08001034c0 after lost signal to thread 7687
I20260709 12:29:26.416843  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3734) took 1006 ms. Trace:
I20260709 12:29:26.342067  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.146291 16616 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.316691  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.393191  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.081225 16710 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.427707  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3736) took 1009 ms. Trace:
I20260709 12:29:26.123822 16698 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:26.416651  7728 rpcz_store.cc:276] 0709 12:29:24.874219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.904280 (+1030061us) service_pool.cc:224] Handling call
0709 12:29:26.416057 (+511777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.428573  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3737) took 1007 ms. Trace:
I20260709 12:29:26.424201  7737 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:29:26.419353  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:26.428925  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3746) took 1006 ms. Trace:
W20260709 12:29:26.393565  7627 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.541s (SENT)
I20260709 12:29:26.424751  8413 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:26.429445  7730 rpcz_store.cc:276] 0709 12:29:25.415836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.418888 (+1003052us) service_pool.cc:224] Handling call
0709 12:29:26.422231 (+  3343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.422345  7736 rpcz_store.cc:276] 0709 12:29:25.410472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.914354 (+503882us) service_pool.cc:224] Handling call
0709 12:29:26.416762 (+502408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.429445  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3719) took 1023 ms. Trace:
W20260709 12:29:26.395460 16723 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.006s
W20260709 12:29:26.372453 16805 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:29:25.946746 16377 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:26.426239  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5b00 after lost signal to thread 7761
W20260709 12:29:26.453042  7590 debug-util.cc:398] Leaking SignalData structure 0x7b080022e2a0 after lost signal to thread 7762
W20260709 12:29:26.432196  7626 connection.cc:256] RPC call timeout handler was delayed by -1.4561381120001897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:29:26.453801  7626 connection.cc:256] RPC call timeout handler was delayed by -1.33657569099978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
W20260709 12:29:26.454388  7590 debug-util.cc:398] Leaking SignalData structure 0x7b0800376300 after lost signal to thread 7836
W20260709 12:29:26.397773 16792 consensus_meta.cc:287] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.004s
I20260709 12:29:26.429803  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:26.433204  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3739) took 1020 ms. Trace:
I20260709 12:29:26.489454  7731 rpcz_store.cc:276] 0709 12:29:25.413090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.329298 (+916208us) service_pool.cc:224] Handling call
0709 12:29:26.433140 (+103842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.428258  7733 rpcz_store.cc:276] 0709 12:29:25.411980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.306542 (+894562us) service_pool.cc:224] Handling call
0709 12:29:26.421095 (+114553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.432228  7734 rpcz_store.cc:276] 0709 12:29:25.405632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.335712 (+930080us) service_pool.cc:224] Handling call
0709 12:29:26.429372 (+ 93660us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.429075  7732 rpcz_store.cc:276] 0709 12:29:25.412353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.324681 (+912328us) service_pool.cc:224] Handling call
0709 12:29:26.419865 (+ 95184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:26.435441  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122490us
I20260709 12:29:26.435611 16797 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.436489 16697 raft_consensus.cc:2804] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:26.438398  7660 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:26.431128  7659 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:26.373749 16666 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.319097 16545 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.396971 16715 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:26.454298  7626 connection.cc:256] RPC call timeout handler was delayed by -1.3131756909997421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
I20260709 12:29:26.449553 16599 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.432436  7729 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:26.422672 16637 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:26.485383  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:26.496869  7553 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 22, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 22 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:26.462913  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.433259  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4057) took 1026 ms. Trace:
I20260709 12:29:26.490299  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:25.944314 16778 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.459789 16779 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:26.456267  7590 debug-util.cc:398] Leaking SignalData structure 0x7b08003545c0 after lost signal to thread 8439
I20260709 12:29:26.466634 16794 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.516726  7737 rpcz_store.cc:276] 0709 12:29:25.406427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.316315 (+909888us) service_pool.cc:224] Handling call
0709 12:29:26.430842 (+114527us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:26.515952  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 76671.3us
I20260709 12:29:26.516261  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3740) took 1102 ms. Trace:
I20260709 12:29:26.531872  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.434885 16710 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.529207  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.585320  7731 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:26.530081  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.462749 16698 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.502002 16797 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.534633  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.535068 16637 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.535588 16545 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.535269  7553 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 20, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 20 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.514948 16715 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.536829  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3727) took 1128 ms. Trace:
I20260709 12:29:26.462558 16616 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.531693  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:26.532102  7729 rpcz_store.cc:276] 0709 12:29:25.413447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.354082 (+940635us) service_pool.cc:224] Handling call
0709 12:29:26.516190 (+162108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.535296  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.531198  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:26.539337 16802 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.539948  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3741) took 1126 ms. Trace:
I20260709 12:29:26.539363 16779 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.534821  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.628813  7728 rpcz_store.cc:276] 0709 12:29:25.413864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.428799 (+1014935us) service_pool.cc:224] Handling call
0709 12:29:26.539871 (+111072us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.546782  7656 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:26.546952  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
W20260709 12:29:26.630147  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 99043.6us
I20260709 12:29:26.550172  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:26.551687  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:26.553360  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.555555  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.630988  7662 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:26.555994 16806 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.184s	user 0.001s	sys 0.003s
I20260709 12:29:26.558043  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.560294  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.562781  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.566146  7804 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:29:26.568816  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:26.570990  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.573410 16709 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:26.574378 16599 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.577350 16798 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.532557  7736 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:26.637506  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3733) took 1227 ms. Trace:
I20260709 12:29:26.637916  7736 rpcz_store.cc:276] 0709 12:29:25.410044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.506064 (+1096020us) service_pool.cc:224] Handling call
0709 12:29:26.637423 (+131359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.589416  7735 raft_consensus.cc:2468] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:26.639096  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4058) took 1232 ms. Trace:
W20260709 12:29:26.639403  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52455.6us
I20260709 12:29:26.639639  7735 rpcz_store.cc:276] 0709 12:29:25.406671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.936408 (+529737us) service_pool.cc:224] Handling call
0709 12:29:26.639008 (+702600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.586404  7733 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:26.586818 16697 raft_consensus.cc:697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:26.641444  7803 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:26.640779  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3721) took 1234 ms. Trace:
I20260709 12:29:26.604683 16728 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:26.621783  7659 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:26.544412  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.547940  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.631357  7730 rpcz_store.cc:276] 0709 12:29:25.407880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.448379 (+1040499us) service_pool.cc:224] Handling call
0709 12:29:26.536743 (+ 88364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.640740  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3747) took 1224 ms. Trace:
I20260709 12:29:26.465826 16792 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.643388  7734 rpcz_store.cc:276] 0709 12:29:25.406261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.501529 (+1095268us) service_pool.cc:224] Handling call
0709 12:29:26.640715 (+139186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.648898  7733 rpcz_store.cc:276] 0709 12:29:25.416252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.501469 (+1085217us) service_pool.cc:224] Handling call
0709 12:29:26.640672 (+139203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:26.615078  7626 connection.cc:256] RPC call timeout handler was delayed by -1.2406553659998281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 12:29:26.635241  7657 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 27.
I20260709 12:29:26.636894 16679 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:26.645188  7732 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:26.652738  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:26.657882  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:26.660775  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3718) took 1255 ms. Trace:
I20260709 12:29:26.661425  8409 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:26.646169  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 114566us
I20260709 12:29:26.656778  8413 raft_consensus.cc:1275] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:26.637383  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.665282  7653 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:26.661557  7732 rpcz_store.cc:276] 0709 12:29:25.405205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.501550 (+1096345us) service_pool.cc:224] Handling call
0709 12:29:26.660697 (+159147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.644944  7809 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:26.668740  8415 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:29:26.659680  7737 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:26.672717  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.674330  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3726) took 1266 ms. Trace:
I20260709 12:29:26.674882  7735 rpcz_store.cc:276] 0709 12:29:25.407635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.659332 (+1251697us) service_pool.cc:224] Handling call
0709 12:29:26.674228 (+ 14896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.683307  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.691795  7654 raft_consensus.cc:1240] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 22. Ops: []
I20260709 12:29:26.659250 16377 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.694394  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.697052  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.698206  7729 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:29:26.705899  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.700490  7734 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:26.706918  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3743) took 1292 ms. Trace:
I20260709 12:29:26.630050 16666 consensus_queue.cc:1048] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:29:26.708225  7734 rpcz_store.cc:276] 0709 12:29:25.414584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.659262 (+1244678us) service_pool.cc:224] Handling call
0709 12:29:26.706845 (+ 47583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.690794  7553 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:26.694187  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.669842 16679 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:26.717535  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:26.718014  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 364625us
I20260709 12:29:26.718600  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.719750  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.721705  7735 raft_consensus.cc:2410] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:29:26.722110  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.722651  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.722980  7660 raft_consensus.cc:2393] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:26.723486  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3745) took 1308 ms. Trace:
I20260709 12:29:26.723968  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.717245  7806 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:26.725311  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:26.730957  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 68495us
I20260709 12:29:26.732568  7553 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:26.723875  7735 rpcz_store.cc:276] 0709 12:29:25.415494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.722353 (+1306859us) service_pool.cc:224] Handling call
0709 12:29:26.723433 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.659101 16545 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:26.726464 16754 consensus_meta.cc:287] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.004s
I20260709 12:29:26.743712  7662 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:26.742002 16820 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:26.750294  8409 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:26.751940  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56969.6us
I20260709 12:29:26.752575 16698 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.753665  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.753885 16697 consensus_queue.cc:237] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:26.755553  7627 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.625s (SENT)
I20260709 12:29:26.756271  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:26.726490 16679 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:26.749557  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.759177  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.748553  7553 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 12 to 21, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 21 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:26.761137  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:26.672256 16802 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.763841  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.765230  7656 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:29:26.757493  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:26.757310 16754 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.766309  7660 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:26.771600  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.771651  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.771829  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.772696  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.774619  8414 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:26.775660  7655 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:26.773437  7736 raft_consensus.cc:2468] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:26.775643  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.736756 16817 raft_consensus.cc:2804] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:26.775411  7734 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:26.773306  8379 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.608s (SENT)
I20260709 12:29:26.778339  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.780051  7732 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:26.786561  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:26.786295  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:26.790752  7661 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:29:26.787418  8413 raft_consensus.cc:2376] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19. Re-sending same reply.
I20260709 12:29:26.792575  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.795770  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:26.789479 16808 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:29:26.798496 16668 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:26.798943 16754 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.796638  8380 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:26.805362  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.806427  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3750) took 1389 ms. Trace:
I20260709 12:29:26.807651  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.793891 16821 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:26.810266 16815 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:26.810827 16792 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.811684 16815 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:26.813139 16599 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.813299 16815 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:26.815704 16545 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:26.816462  7659 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:26.816605 16802 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:26.817698  7731 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:26.818065  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.818329  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3729) took 1410 ms. Trace:
I20260709 12:29:26.818760  7731 rpcz_store.cc:276] 0709 12:29:25.408409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.501379 (+1092970us) service_pool.cc:224] Handling call
0709 12:29:26.818247 (+316868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.780119 16817 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:26.819519  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.742810 16377 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:26.789409 16753 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.877s	user 0.014s	sys 0.002s
I20260709 12:29:26.806824  7733 rpcz_store.cc:276] 0709 12:29:25.417312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.663728 (+1246416us) service_pool.cc:224] Handling call
0709 12:29:26.805144 (+141416us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:29:26.806348 (+  1204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7102336}
I20260709 12:29:26.821663  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.810636 16821 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:26.826246  7654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45918 (request call id 3634) took 1011 ms. Trace:
I20260709 12:29:26.832420  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.827620  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.811983 16637 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:26.833482  7731 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:26.834616  7730 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:29:26.834111  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:26.837075  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.832621  7654 rpcz_store.cc:276] 0709 12:29:25.814958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.815127 (+   169us) service_pool.cc:224] Handling call
0709 12:29:25.815431 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:26.701257 (+885826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:26.822407 (+121150us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b540033e5d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:26.826167 (+  3760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014914944}
I20260709 12:29:26.842603  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.842518  7553 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 25, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 25 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:26.844851  7728 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:26.845088  7735 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:26.845677  8415 raft_consensus.cc:1240] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:26.841728  7624 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:26.843954  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:26.849314  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84566.5us
I20260709 12:29:26.851532  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.851742  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.851390 16821 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.853214  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.853410  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.852887  7553 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.854315  7736 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:26.856623  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:26.856949 16797 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.854912  7553 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 21, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 21 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:26.854832  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:26.851444  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:26.862596  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:26.864161  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.861621  7734 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:26.880589  7737 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:26.881287  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3722) took 1474 ms. Trace:
I20260709 12:29:26.881625  7737 rpcz_store.cc:276] 0709 12:29:25.406507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.531287 (+1124780us) service_pool.cc:224] Handling call
0709 12:29:26.881211 (+349924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.860857  7728 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24. Re-sending same reply.
I20260709 12:29:26.825525 16817 raft_consensus.cc:3060] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:26.888711  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66067.9us
I20260709 12:29:26.889734  7732 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 29.
I20260709 12:29:26.890002  8379 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:26.857753 16710 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:26.866356  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:26.865087  7736 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:26.894197  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.895270  7737 raft_consensus.cc:2393] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:26.895748  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.893785 16797 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:26.896997 16698 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:26.895598  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:26.900230  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3728) took 1492 ms. Trace:
W20260709 12:29:26.860991 16747 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 1.086s	user 0.015s	sys 0.004s
W20260709 12:29:26.901803  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120292us
I20260709 12:29:26.927394  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.904353  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.793701 16820 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:26.906514  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40608 (request call id 1282) took 1124 ms. Trace:
W20260709 12:29:26.931954  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 176479us
I20260709 12:29:26.835182 16753 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:26.921700  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: true dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.923806  7700 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:26.944654  7803 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:26.922940  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:26.922147  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.921844  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.921325  7553 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.922571  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:26.921414  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.923075  7737 rpcz_store.cc:276] 0709 12:29:25.408110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.893813 (+1485703us) service_pool.cc:224] Handling call
0709 12:29:26.900115 (+  6302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:26.921958  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.925719  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:26.931684  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:26.931739  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.928335  7657 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
I20260709 12:29:26.936803  7735 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:26.937001  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.937489  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.937906  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:26.927502 16698 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:26.899107 16616 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:26.931476 16754 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:26.993441  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:26.941929  7807 raft_consensus.cc:1240] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
W20260709 12:29:26.982784  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:26.926548 16797 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.022817  7736 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:26.996044  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.004742  7809 raft_consensus.cc:2393] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:26.948518  7653 raft_consensus.cc:2393] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 22: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:27.031150  7805 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:26.934232  8406 rpcz_store.cc:276] 0709 12:29:25.773945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:25.774112 (+   167us) service_pool.cc:224] Handling call
0709 12:29:25.777759 (+  3647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:26.356127 (+578368us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:29:26.431012 (+ 74885us) log.cc:844] Serialized 14 byte log entry
0709 12:29:26.483786 (+ 52774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:26.483875 (+    89us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:26.500354 (+ 16479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:26.824236 (+323882us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b180043e4c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee032a6 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419
0709 12:29:26.894977 (+ 70741us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54005e01d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:26.895033 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:26.895149 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5178,"prepare.queue_time_us":36296,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":52,"raft.queue_time_us":100,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":13328,"spinlock_wait_cycles":1285298688,"thread_start_us":48114,"threads_started":1,"wal-append.queue_time_us":198803}
I20260709 12:29:27.035604  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3751) took 1098 ms. Trace:
I20260709 12:29:27.030416  7658 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
W20260709 12:29:27.038492  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 96284.1us
I20260709 12:29:27.044824  7735 rpcz_store.cc:276] 0709 12:29:25.937810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.859615 (+921805us) service_pool.cc:224] Handling call
0709 12:29:27.035519 (+175904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.047003  8405 raft_consensus.cc:1240] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:27.039681  7553 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 22, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 22 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:27.048144  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.049121  7735 raft_consensus.cc:2410] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:29:26.937563 16821 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:27.041440  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 121668us
I20260709 12:29:26.905539 16545 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.096s
I20260709 12:29:27.052557  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:26.947948 16595 raft_consensus.cc:2764] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:29:26.994235 16779 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:27.056013  8409 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:27.056263  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:27.058632  7655 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:27.053545  8413 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:27.048910  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:26.942544 16351 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:26.942217 16747 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.020183 16698 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:27.030133 16754 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:26.931565 16663 consensus_queue.cc:1059] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.133s
I20260709 12:29:27.065865  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:26.917618 16710 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 12:29:27.093464 16723 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:27.094019 16710 consensus_queue.cc:1243] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:29:27.096993  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.098474  7806 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:27.066934  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:27.067447 16819 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:29:26.992714 16715 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:27.074393  7737 raft_consensus.cc:2376] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23. Re-sending same reply.
I20260709 12:29:27.114737  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.115805  7737 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:27.117063  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.117897  7737 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:27.076236  7730 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 18.
I20260709 12:29:27.119565  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3724) took 1712 ms. Trace:
I20260709 12:29:27.119896  7730 rpcz_store.cc:276] 0709 12:29:25.406984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.666869 (+1259885us) service_pool.cc:224] Handling call
0709 12:29:26.821506 (+154637us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006ee80a0 00007fc006ee7984 00007fc0004626d4 00007fc000462662
0709 12:29:27.119497 (+297991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12170112}
I20260709 12:29:27.120679 16824 consensus_queue.cc:1059] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:29:27.057701  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.121999 16824 raft_consensus.cc:3055] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:27.122385 16824 raft_consensus.cc:740] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:27.084023  7778 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:27.123498 16778 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:27.123924 16778 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:27.124320 16778 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:27.125227 16747 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:27.126046  7731 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:27.127718 16663 raft_consensus.cc:3055] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:29:27.128201 16663 raft_consensus.cc:740] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:27.129329 16663 consensus_queue.cc:260] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.088688 16754 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:27.036437 16797 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:27.054610 16666 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:27.080487 16779 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.104221  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:27.151413  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.111613  7733 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:27.112907  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.121140  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.131008 16663 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:27.076989  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.153719  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:27.155174  7807 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:27.156479  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:27.131750  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47797.5us
I20260709 12:29:27.160264  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.161808  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.162200  7778 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:26.994748 16812 raft_consensus.cc:2764] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:29:27.165586  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.167837  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.168959  7730 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:27.170629  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:27.172178  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.174526 16800 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:27.175154 16800 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:27.175604 16800 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:27.176846 16753 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:27.132035  7627 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.686s (SENT)
I20260709 12:29:27.178596  7627 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:27.179888  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92443.2us
I20260709 12:29:27.180377  7806 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
W20260709 12:29:27.180640  7627 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.706s (SENT)
I20260709 12:29:27.181180  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.119073  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.181577  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.182225  7737 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 19.
I20260709 12:29:27.182802  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46562 (request call id 3753) took 1244 ms. Trace:
I20260709 12:29:27.183137  7737 rpcz_store.cc:276] 0709 12:29:25.938739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.118712 (+1179973us) service_pool.cc:224] Handling call
0709 12:29:27.182738 (+ 64026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.185014  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4054) took 2326 ms. Trace:
I20260709 12:29:27.185268  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.186090  7809 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:27.188094  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4055) took 2305 ms. Trace:
I20260709 12:29:27.188608  7732 rpcz_store.cc:276] 0709 12:29:25.012703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.186876 (+2174173us) service_pool.cc:224] Handling call
0709 12:29:27.187305 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.187618 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.187648 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.187726 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:27.187784 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.187850 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.187999 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.189596 16779 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.192412  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.193042 16779 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:27.193647  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.194257 16822 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:27.195338  7658 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: []
W20260709 12:29:27.090536  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 97466us
I20260709 12:29:27.173002  7730 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:27.142187  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.194392  7655 raft_consensus.cc:3060] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:27.198098  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.197705  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.199091  7656 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:29:27.199436  7811 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:27.199740  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:27.200556  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.185635  7737 rpcz_store.cc:276] 0709 12:29:24.859015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.183975 (+2324960us) service_pool.cc:224] Handling call
0709 12:29:27.184344 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.184582 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.184609 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.184679 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:27.184714 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.184774 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.184925 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.148859  8409 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 28.
I20260709 12:29:27.187135  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.214519  7809 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:27.094098 16723 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.227198  7654 raft_consensus.cc:2393] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 21: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:27.228638  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1916) took 2194 ms. Trace:
I20260709 12:29:27.229128  7732 rpcz_store.cc:276] 0709 12:29:25.006064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.189863 (+2183799us) service_pool.cc:224] Handling call
0709 12:29:27.190257 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.190651 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.190682 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.190754 (+    72us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:27.190791 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.190861 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.199882 (+  9021us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:27.230365  7729 consensus_meta.cc:287] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.002s
I20260709 12:29:27.068769 16728 consensus_queue.cc:1059] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.082s
I20260709 12:29:27.195153 16822 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:27.232613  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.111202 16351 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:27.256887 16351 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.260185  7735 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:27.261188  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3731) took 1851 ms. Trace:
I20260709 12:29:27.261601  7735 rpcz_store.cc:276] 0709 12:29:25.409359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.230230 (+1820871us) service_pool.cc:224] Handling call
0709 12:29:27.259753 (+ 29523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.260815 (+  1062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.261080 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.186627 16754 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.265060 16754 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:27.230983  7729 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:27.234217  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 72113.3us
I20260709 12:29:27.266544  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:27.266811  7729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4056) took 1860 ms (client timeout 2288 ms). Trace:
W20260709 12:29:27.267436  7729 rpcz_store.cc:269] 0709 12:29:25.406110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.660216 (+1254106us) service_pool.cc:224] Handling call
0709 12:29:27.266746 (+606530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.267748  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:27.253639  7626 connection.cc:256] RPC call timeout handler was delayed by -1.17907569099998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 12:29:27.272140 16728 raft_consensus.cc:3055] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:27.272821 16599 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.273795 16599 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.275516 16377 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:27.277347 16599 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:27.277678  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:27.278299 16679 consensus_meta.cc:287] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.005s
I20260709 12:29:27.278461  7731 raft_consensus.cc:2376] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25. Re-sending same reply.
I20260709 12:29:27.281867 16824 consensus_queue.cc:260] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.283584 16824 raft_consensus.cc:3060] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:27.283928  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.284686  7805 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:27.285873  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.286183 16545 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:27.237296  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.290112  7660 raft_consensus.cc:2368] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22.
I20260709 12:29:27.232769 16723 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:27.238153  7657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.294625  7657 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
W20260709 12:29:27.271735  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 91127.1us
I20260709 12:29:27.243495 16697 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:27.246744  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.250012  7736 raft_consensus.cc:2468] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
I20260709 12:29:27.253494  7734 raft_consensus.cc:2468] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:27.271679 16825 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:27.272295 16794 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:27.236614 16820 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:27.235049  7808 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.000s
I20260709 12:29:27.272789 16728 raft_consensus.cc:740] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:27.337093  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.342450  7807 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
W20260709 12:29:27.338498  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 91831.2us
I20260709 12:29:27.349725  7808 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:27.351395  7730 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:27.278887 16679 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.352819  7809 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:27.353979  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.313305  7655 raft_consensus.cc:2468] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:27.310256  7735 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 12:29:27.357616  7737 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:27.363376  7809 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:27.359355  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4061) took 1444 ms. Trace:
I20260709 12:29:27.359599 16679 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:27.295747 16779 raft_consensus.cc:2804] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:27.365284  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.365306  7736 rpcz_store.cc:276] 0709 12:29:25.915311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.358324 (+1443013us) service_pool.cc:224] Handling call
0709 12:29:27.358735 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.358961 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.358990 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.359034 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:27.359067 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.359124 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.359259 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.390941  7812 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
W20260709 12:29:27.320639  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:27.388706  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.388706  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:27.392030  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5403603709996787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
I20260709 12:29:27.392386  7732 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:27.392620  7702 connection.cc:256] RPC call timeout handler was delayed by -1.4350603709999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:29:27.393066  7702 connection.cc:256] RPC call timeout handler was delayed by -1.2118603709996023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
I20260709 12:29:27.347014 16825 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.396724  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3748) took 1946 ms. Trace:
I20260709 12:29:27.397203  7735 rpcz_store.cc:276] 0709 12:29:25.416636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.262597 (+1845961us) service_pool.cc:224] Handling call
0709 12:29:27.263031 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.362979 (+ 99948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.363339 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:27.401600  7626 leader_election.cc:336] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Timed out: RequestConsensusVote RPC to 127.6.76.195:37801 timed out after 1.927s (SENT)
I20260709 12:29:27.398298  7775 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:27.402189  7627 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.725s (SENT)
I20260709 12:29:27.402132  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:27.400940  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4060) took 1671 ms. Trace:
I20260709 12:29:27.402843  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.403733  7734 rpcz_store.cc:276] 0709 12:29:25.729152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.376221 (+1647069us) service_pool.cc:224] Handling call
0709 12:29:27.376594 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.399846 (+ 23252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.399923 (+    77us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.400126 (+   203us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:29:27.400247 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.400406 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.400785 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.406908  7812 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: []
W20260709 12:29:27.396451  7702 connection.cc:256] RPC call timeout handler was delayed by -1.207360370999595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 12:29:27.408409  7702 connection.cc:256] RPC call timeout handler was delayed by -1.1488603709999552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 12:29:27.408780  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0912603709998621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 12:29:27.409091  7702 connection.cc:256] RPC call timeout handler was delayed by -1.0885603709998577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 12:29:27.412653  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63118.9us
I20260709 12:29:27.366290 16822 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.415375  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.415763 16822 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.352027 16545 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:27.418756  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.418834  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:27.355002 16802 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:27.423766  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 77298.1us
I20260709 12:29:27.428872  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.370625 16794 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.266541 16778 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:27.443672  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45439.3us
I20260709 12:29:27.444671  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.433130  8380 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:27.447669  7736 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:29:27.448668  7736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4063) took 1465 ms. Trace:
I20260709 12:29:27.449105  7736 rpcz_store.cc:276] 0709 12:29:25.983407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.402001 (+1418594us) service_pool.cc:224] Handling call
0709 12:29:27.440501 (+ 38500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.448236 (+  7735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.448576 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.439335  7728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4062) took 1523 ms. Trace:
I20260709 12:29:27.450608  7728 rpcz_store.cc:276] 0709 12:29:25.915652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.374406 (+1458754us) service_pool.cc:224] Handling call
0709 12:29:27.420377 (+ 45971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.438651 (+ 18274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.438700 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.438783 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:27.438853 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.438962 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.439199 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.435143  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:27.461733  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4064) took 1168 ms. Trace:
I20260709 12:29:27.462630  7735 rpcz_store.cc:276] 0709 12:29:26.293079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.398095 (+1105016us) service_pool.cc:224] Handling call
0709 12:29:27.428174 (+ 30079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.450799 (+ 22625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.450831 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.461199 (+ 10368us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:27.461295 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.461386 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.461618 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.402211 16825 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:27.475996  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4065) took 1127 ms. Trace:
I20260709 12:29:27.422108 16822 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:27.463574  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 126683us
I20260709 12:29:27.477290  7734 rpcz_store.cc:276] 0709 12:29:26.330980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.433381 (+1102401us) service_pool.cc:224] Handling call
0709 12:29:27.457980 (+ 24599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.458244 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.458279 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.458332 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:27.458368 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.458432 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.458600 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.442693 16794 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:27.482769  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3744) took 2067 ms. Trace:
I20260709 12:29:27.483467  7737 rpcz_store.cc:276] 0709 12:29:25.414937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.228919 (+1813982us) service_pool.cc:224] Handling call
0709 12:29:27.357382 (+128463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.362492 (+  5110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.482383 (+119891us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54002ab550. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:27.482617 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294455040}
I20260709 12:29:27.484869 16778 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:27.485832 16815 consensus_meta.cc:287] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.002s
I20260709 12:29:27.486123  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4068) took 1020 ms. Trace:
W20260709 12:29:27.487118  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 120613us
W20260709 12:29:27.488298  7627 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.792s (SENT)
I20260709 12:29:27.488871  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:27.494618  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 59680.1us
I20260709 12:29:27.488314  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46562 (request call id 3723) took 2081 ms. Trace:
I20260709 12:29:27.495469  7730 rpcz_store.cc:276] 0709 12:29:25.406750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.230155 (+1823405us) service_pool.cc:224] Handling call
0709 12:29:27.351140 (+120985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.351936 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.487959 (+136023us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b54001450d0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:27.488214 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220868992}
W20260709 12:29:27.485924  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 121590us
I20260709 12:29:27.486954  7737 rpcz_store.cc:276] 0709 12:29:26.465424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.484788 (+1019364us) service_pool.cc:224] Handling call
0709 12:29:27.485303 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.485594 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.485635 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.485722 (+    87us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:29:27.485785 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.485870 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.486035 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:29:27.499240  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 52140.8us
I20260709 12:29:27.500929  7775 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:27.501669  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:27.472239  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 56043.5us
I20260709 12:29:27.454231 16820 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:27.432822 16830 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:27.508036  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1922) took 1080 ms. Trace:
I20260709 12:29:27.521076  7734 rpcz_store.cc:276] 0709 12:29:26.427495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.478539 (+1051044us) service_pool.cc:224] Handling call
0709 12:29:27.507442 (+ 28903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.507652 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.507681 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.507730 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:27.507764 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.507842 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.507972 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:27.535497  7658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.535763  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.536670  7658 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:27.436671 16709 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:29:27.521718 16778 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:27.552299  7734 raft_consensus.cc:1275] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:29:27.501761 16822 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:27.504052 16798 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:27.501754 16838 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:27.446352 16802 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:27.554065  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 53255.2us
I20260709 12:29:27.560681  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:27.560685  7803 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:27.561686 16820 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:27.562150 16820 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.562536 16820 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:27.563557  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.564388  7654 raft_consensus.cc:2376] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25. Re-sending same reply.
I20260709 12:29:27.564862  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:27.566777 16666 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:27.567406 16666 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:27.567448  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 71798.1us
I20260709 12:29:27.569262  7778 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:27.571843 16839 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:27.572592 16839 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.573274 16839 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:27.573931  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:27.575115 16377 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:29:27.575261  7736 pending_rounds.cc:85] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 10
I20260709 12:29:27.576638  7736 pending_rounds.cc:107] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:29:27.577018  7736 raft_consensus.cc:2889] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:27.563820  7627 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.874s (SENT)
I20260709 12:29:27.577736  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:27.582278  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.583086  7655 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:27.597725  7654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:27.598778  7654 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:27.505378 16815 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.576778 16710 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:27.550781 16797 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.626982  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:27.623374  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 62184.4us
I20260709 12:29:27.460187 16728 consensus_queue.cc:260] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.633081 16728 raft_consensus.cc:3060] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:27.634199 16815 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:27.533206 16715 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:29:27.629706  8413 raft_consensus.cc:2368] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22.
I20260709 12:29:27.600571 16595 raft_consensus.cc:2804] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:27.595088 16377 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:27.633194 16843 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:29:27.623447  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 43723.5us
I20260709 12:29:27.623490  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.642390  7811 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:27.642252  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68461us
I20260709 12:29:27.643963  7732 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:27.649617  7806 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:27.654331  7729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1918) took 1935 ms. Trace:
I20260709 12:29:27.614957 16798 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:27.628818 16351 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:27.674434  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:27.655045  7729 rpcz_store.cc:276] 0709 12:29:25.719096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.268436 (+1549340us) service_pool.cc:224] Handling call
0709 12:29:27.268753 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.269021 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:29:27.453656 (+184635us) log.cc:844] Serialized 14 byte log entry
0709 12:29:27.653804 (+200148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.653866 (+    62us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:29:27.653916 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.654122 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.654269 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":361558,"raft.queue_time_us":301,"thread_start_us":288996,"threads_started":2}
W20260709 12:29:27.660804  7662 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.016s	sys 0.026s
W20260709 12:29:27.678300  7662 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.016s	sys 0.026s
W20260709 12:29:27.676788  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 113004us
I20260709 12:29:27.679203  8411 raft_consensus.cc:1240] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:27.693086 16779 raft_consensus.cc:697] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:27.706251  8408 raft_consensus.cc:2393] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:27.639684 16830 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:27.610337 16800 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:27.714612  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 209444us
I20260709 12:29:27.650503 16843 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.650056 16377 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:27.628389 16845 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.080s
I20260709 12:29:27.671988 16595 raft_consensus.cc:697] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:27.731920  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.732316  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.732805  7660 raft_consensus.cc:3060] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:27.733484  7653 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:27.740199  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 212246us
I20260709 12:29:27.651141 16838 consensus_queue.cc:1059] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.079s
W20260709 12:29:27.743427  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 114670us
W20260709 12:29:27.743732  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69004.9us
I20260709 12:29:27.746438  7655 raft_consensus.cc:1275] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:27.721827 16679 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:27.722687 16377 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:27.755810  7656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:27.757193  7656 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:27.728293 16843 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:27.788484  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:27.818081 16723 consensus_meta.cc:287] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.002s
I20260709 12:29:27.822422  7660 raft_consensus.cc:2468] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:27.883631  7656 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:27.827706 16595 consensus_queue.cc:237] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.715126 16351 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:27.829836 16846 log.cc:927] Time spent T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:29:27.831501 16798 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.836484 16859 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:29:27.837590  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 82306us
W20260709 12:29:27.910006  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 255416us
I20260709 12:29:27.828603  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:27.884866  7627 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.977s (SENT)
I20260709 12:29:27.886801 16778 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.894294 16794 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.897145 16843 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.747658 16841 raft_consensus.cc:2804] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:27.911291  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:27.911952  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:27.809849 16715 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:27.923136  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:27.780751 16679 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.929672  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:27.937140  7803 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:27.925678  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 491733us
I20260709 12:29:27.945300  7653 raft_consensus.cc:2468] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:27.945389  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.946380  7811 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:27.946521  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.947522  7807 raft_consensus.cc:2393] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:27.850881 16820 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.901657 16723 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.960733  7624 leader_election.cc:400] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22.
I20260709 12:29:27.963953  7624 leader_election.cc:403] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:29:27.882130 16779 consensus_queue.cc:237] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:27.966172 16754 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:27.966564 16679 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:27.966769 16754 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.967345 16754 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:27.946755 16841 raft_consensus.cc:697] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:27.973212 16841 consensus_queue.cc:237] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:27.975863  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 147355us
I20260709 12:29:27.977473  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4071) took 1177 ms. Trace:
I20260709 12:29:27.977906  7737 rpcz_store.cc:276] 0709 12:29:26.800481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.498580 (+698099us) service_pool.cc:224] Handling call
0709 12:29:27.499187 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:27.499853 (+   666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.499900 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:27.500550 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:29:27.647555 (+147005us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:29:27.705871 (+ 58316us) log.cc:844] Serialized 14 byte log entry
0709 12:29:27.706209 (+   338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.706314 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:27.977179 (+270865us) raft_consensus.cc:1685] finished
0709 12:29:27.977209 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.977390 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47437,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":1931,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":159,"spinlock_wait_cycles":11812736,"thread_start_us":146414,"threads_started":1,"wal-append.queue_time_us":237525}
I20260709 12:29:27.945621 16663 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:28.015069  7777 leader_election.cc:400] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23.
I20260709 12:29:28.019160  7662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1677) took 1232 ms. Trace:
I20260709 12:29:28.031702  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:28.032240  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48401.7us
I20260709 12:29:27.925279 16778 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:28.003710  8379 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:28.001766 16698 consensus_meta.cc:287] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.900s	user 0.007s	sys 0.008s
I20260709 12:29:28.043174  8379 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:28.051688  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:28.052134  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:27.937716 16715 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:28.033759 16824 consensus_meta.cc:287] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.750s	user 0.015s	sys 0.000s
I20260709 12:29:27.920928 16859 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:27.977092 16852 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:29:28.019870  7662 rpcz_store.cc:276] 0709 12:29:26.752699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:26.765164 (+ 12465us) service_pool.cc:224] Handling call
0709 12:29:26.765627 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:26.768291 (+  2664us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:29:27.679139 (+910848us) log.cc:844] Serialized 14 byte log entry
0709 12:29:27.961662 (+282523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:27.961743 (+    81us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:27.962714 (+   971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.963093 (+   379us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:27.984733 (+ 21640us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5628,"prepare.queue_time_us":677761,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":8952,"thread_start_us":1166476,"threads_started":3,"wal-append.queue_time_us":284130}
I20260709 12:29:28.099805  7806 raft_consensus.cc:1240] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:28.352993  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:28.308051  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:28.068078  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:28.066390  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:28.273609  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:28.030829 16663 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:28.369969  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:28.368408  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:28.364169  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 400947us
I20260709 12:29:28.360381  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1917) took 3354 ms. Trace:
I20260709 12:29:28.373379  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:28.332069 16817 consensus_meta.cc:287] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.412s	user 0.012s	sys 0.004s
I20260709 12:29:28.377810  7626 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:28.067619 16840 raft_consensus.cc:2804] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:28.384487  7653 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:29:28.384171  7626 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:28.066362 16723 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:28.311906 16794 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:28.373505  7733 rpcz_store.cc:276] 0709 12:29:25.006445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.194565 (+2188120us) service_pool.cc:224] Handling call
0709 12:29:27.194932 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:27.195212 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:29:27.454665 (+259453us) log.cc:844] Serialized 14 byte log entry
0709 12:29:28.359869 (+905204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:28.359945 (+    76us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:28.360027 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:28.360123 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:28.360292 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":381675,"prepare.run_cpu_time_us":188,"prepare.run_wall_time_us":11476,"thread_start_us":1081106,"threads_started":3,"wal-append.queue_time_us":594571}
I20260709 12:29:28.081980 16545 raft_consensus.cc:2804] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:28.332275 16800 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:28.069922 16778 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:28.067819 16753 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:28.406561 16857 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:28.407827 16857 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.408540 16857 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:28.409468 16862 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:28.410135 16862 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:28.410668 16862 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:28.070776 16698 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:28.394361 16831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.008s
I20260709 12:29:28.399753  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:28.399641  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:28.423995  7737 raft_consensus.cc:3060] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:28.401608  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:28.045588 16679 consensus_queue.cc:1048] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.340s
I20260709 12:29:28.403668  7812 raft_consensus.cc:1240] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
W20260709 12:29:28.403714  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 564454us
I20260709 12:29:28.375540 16663 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:28.055850 16849 consensus_queue.cc:1048] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.333s
W20260709 12:29:28.420922  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 443888us
W20260709 12:29:28.390200 16825 consensus_meta.cc:287] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.855s	user 0.009s	sys 0.004s
W20260709 12:29:28.443409  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 43758.3us
I20260709 12:29:28.445459  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:28.446834  7806 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:28.448024  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:28.448329  8379 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:28.449040  7811 raft_consensus.cc:2376] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 27. Re-sending same reply.
W20260709 12:29:28.428292 16831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.004s	sys 0.009s
I20260709 12:29:28.385774 16817 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.426499  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:28.427400  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:28.460654  7803 raft_consensus.cc:2376] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24. Re-sending same reply.
I20260709 12:29:28.430439  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:28.389938 16351 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:28.404014 16666 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.836s	user 0.015s	sys 0.000s
W20260709 12:29:28.458791  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 100207us
I20260709 12:29:28.463610  7700 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:28.436530 16753 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:29:28.421360  7658 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.867s	user 0.007s	sys 0.007s
I20260709 12:29:28.485316  7658 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 28.
I20260709 12:29:28.437150 16856 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:28.437345 16876 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:28.489800  7810 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:28.472625 16843 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:28.482854  7737 raft_consensus.cc:2468] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
W20260709 12:29:28.542824  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 79318.6us
I20260709 12:29:28.562067  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:28.529384  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:28.525878 16867 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
I20260709 12:29:28.454619 16825 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:28.524210 16679 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:28.523474 16351 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:28.518966 16864 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.518121 16753 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:28.528258 16817 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:28.597050 16874 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:29:28.579814  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 95232.9us
I20260709 12:29:28.585314  8405 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:28.539031 16876 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:28.586655  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:28.601140  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 137622us
I20260709 12:29:28.484623 16666 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:28.589767  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 141525us
I20260709 12:29:28.524793 16849 raft_consensus.cc:2804] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:28.595405 16877 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.516319 16856 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.516547 16872 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487) with higher term. Message: Invalid argument: T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22.
I20260709 12:29:28.525776 16715 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:28.597664 16874 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.604054  7805 raft_consensus.cc:1240] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:29:28.523375 16873 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:28.573432 16820 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:28.605296  7700 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:28.614962  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 184586us
W20260709 12:29:28.610117 16754 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.005s
I20260709 12:29:28.579746 16840 raft_consensus.cc:697] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:28.529181 16663 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:28.637590  8413 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:28.616796 16856 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:28.645761  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:28.649647  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:28.656968  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:28.658608  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:28.659448  7736 raft_consensus.cc:3060] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:29:28.665777  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:28.666683  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 207351us
I20260709 12:29:28.666941  7729 raft_consensus.cc:3060] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:28.634539 16874 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:28.674031  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:28.637027 16849 raft_consensus.cc:697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:28.659668 16377 consensus_meta.cc:287] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.872s	user 0.015s	sys 0.000s
I20260709 12:29:28.642620 16545 raft_consensus.cc:697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:28.686087  7627 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.052s (SENT)
I20260709 12:29:28.692602  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:28.646459 16679 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 12:29:28.710210  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 328123us
I20260709 12:29:28.647864 16754 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.713116  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:28.695559 16377 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:28.528432 16871 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.318s
I20260709 12:29:28.723275  7736 raft_consensus.cc:2468] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
W20260709 12:29:28.723011  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 102056us
I20260709 12:29:28.693495 16885 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.689635 16698 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:28.525506 16879 consensus_queue.cc:1059] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.180s
I20260709 12:29:28.710942 16820 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:28.758839  7624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.010s	sys 0.007s
W20260709 12:29:28.759408  7624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.010s	sys 0.007s
I20260709 12:29:28.617573 16877 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:28.759784  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 46816.5us
I20260709 12:29:28.716420 16841 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:28.602973 16864 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:28.756457  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:28.686188 16753 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:28.759779  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 834621us
I20260709 12:29:28.785488  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:28.786654  8405 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:28.787334  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:28.789139  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 112833us
I20260709 12:29:28.732204 16840 consensus_queue.cc:237] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.742105  7729 raft_consensus.cc:2468] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:28.780545 16824 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:29:28.709822 16876 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:28.777479 16849 consensus_queue.cc:237] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.736371 16885 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.800535  7803 raft_consensus.cc:1275] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
W20260709 12:29:29.157184  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:29.106746  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:28.778126 16778 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:28.794535 16865 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
W20260709 12:29:29.186631  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 459207us
I20260709 12:29:29.198290 16859 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:29.253695 16909 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.004712 16896 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.254833  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 3151) took 2143 ms. Trace:
I20260709 12:29:28.775803 16841 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:29.049762  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.01655e+06us
I20260709 12:29:29.035856  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:29.065009  8413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40608 (request call id 1293) took 1010 ms. Trace:
W20260709 12:29:28.988580 16893 log.cc:927] Time spent T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 12:29:29.244083  7627 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.101s (SENT)
I20260709 12:29:28.920617  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:29.130692  8406 raft_consensus.cc:1240] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:29.044641  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:29.257302  7803 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 16
I20260709 12:29:29.273098  7803 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:29:29.258136  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.273619  7803 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:29.274051  7729 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:29.152257  7659 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:29.254352  7778 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.260991  7733 raft_consensus.cc:1240] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:29.262537  8413 rpcz_store.cc:276] 0709 12:29:28.054901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:28.055093 (+   192us) service_pool.cc:224] Handling call
0709 12:29:29.064624 (+1009531us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400587854. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:29.064901 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626980224}
I20260709 12:29:29.200940  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45832 (request call id 3809) took 2119 ms. Trace:
I20260709 12:29:29.137738  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:29.149339  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:28.876690 16870 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:29:29.267367  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:29.268255  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 232539us
I20260709 12:29:29.288151  7728 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:29.152769 16862 consensus_meta.cc:287] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.005s
I20260709 12:29:29.292840  8408 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:29:29.054947 16850 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.156s	user 0.003s	sys 0.015s
I20260709 12:29:28.920907 16902 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.255260  7808 rpcz_store.cc:276] 0709 12:29:27.111318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.425087 (+313769us) service_pool.cc:224] Handling call
0709 12:29:27.425831 (+   744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:27.674669 (+248838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.674711 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:27.732022 (+ 57311us) log.cc:844] Serialized 26 byte log entry
0709 12:29:27.985001 (+252979us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:28.581599 (+596598us) log.cc:844] Serialized 14 byte log entry
0709 12:29:28.585607 (+  4008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:28.585734 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.254578 (+668844us) raft_consensus.cc:1685] finished
0709 12:29:29.254605 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.254769 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28304,"prepare.queue_time_us":641104,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":81657,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":41481,"thread_start_us":879135,"threads_started":3,"wal-append.queue_time_us":331405}
I20260709 12:29:29.313771  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.264547  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:29.292695  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:29.045574 16861 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.776s	user 0.003s	sys 0.000s
W20260709 12:29:28.971112 16858 log.cc:927] Time spent T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:29:29.276412  7805 rpcz_store.cc:276] 0709 12:29:27.077447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.286937 (+209490us) service_pool.cc:224] Handling call
0709 12:29:27.304840 (+ 17903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:28.625986 (+1321146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:29.196928 (+570942us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003c55d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:29.197270 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2983382400}
W20260709 12:29:28.986758 16898 log.cc:927] Time spent T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:29:28.811990 16825 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:28.879031 16864 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:28.877426 16666 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:28.921859 16802 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:28.878150 16882 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:29:28.921797 16754 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:28.877003 16377 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.299283  7736 raft_consensus.cc:1240] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
W20260709 12:29:28.800519 16839 consensus_meta.cc:287] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 1.227s	user 0.013s	sys 0.006s
I20260709 12:29:29.302048 16862 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:28.876087 16888 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:28.800184 16709 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.488s
I20260709 12:29:28.879643 16723 consensus_queue.cc:1048] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.418s
I20260709 12:29:29.303611 16595 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:29:29.027758 16715 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.066277 16838 consensus_queue.cc:1059] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.240s
I20260709 12:29:29.306565 16728 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:29.308099 16899 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:29:29.309108 16892 raft_consensus.cc:3055] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:29.296043 16663 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:28.993912 16877 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:28.877136 16891 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:29.313251  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:29.322619  7732 raft_consensus.cc:1275] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:29:29.296120 16843 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.016s	sys 0.003s
I20260709 12:29:28.794943 16545 consensus_queue.cc:237] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.289032 16753 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.357976  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:29.359710  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.412635  7735 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:29.412748  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:29.365381  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 76426.9us
I20260709 12:29:29.320313 16820 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:29.399889 16874 consensus_meta.cc:287] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.706s	user 0.015s	sys 0.000s
W20260709 12:29:29.355880 16911 log.cc:927] Time spent T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:29:29.349578 16812 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:29.421634 16753 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.426844  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:29.377202  7728 raft_consensus.cc:2468] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:29.254380 16909 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:29.380993  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 126771us
I20260709 12:29:29.254604 16896 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.438460 16753 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.379539 16839 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:29.384868  7777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.011s	sys 0.046s
I20260709 12:29:29.439895 16896 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:29.439793 16753 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:29.389636  8412 raft_consensus.cc:1240] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:29:29.441229 16753 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:29.441642 16839 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:29.441836 16753 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:29.350063 16830 consensus_queue.cc:1059] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:29:29.442667 16830 consensus_queue.cc:1243] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:29:29.411854  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1930) took 2550 ms. Trace:
I20260709 12:29:29.442894 16753 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.443411  7734 rpcz_store.cc:276] 0709 12:29:26.861724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.553425 (+691701us) service_pool.cc:224] Handling call
0709 12:29:27.553832 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:27.554650 (+   818us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:29:27.743198 (+188548us) log.cc:844] Serialized 14 byte log entry
0709 12:29:28.642916 (+899718us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:28.643777 (+   861us) log.cc:844] Serialized 26 byte log entry
0709 12:29:28.643952 (+   175us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:28.643995 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:28.644116 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.411558 (+767442us) raft_consensus.cc:1685] finished
0709 12:29:29.411583 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.411775 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70103,"prepare.queue_time_us":85766,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":210579,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":34312,"thread_start_us":838372,"threads_started":3,"wal-append.queue_time_us":650430}
I20260709 12:29:29.158747 16856 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:29.416273  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:29.445604  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 132497us
I20260709 12:29:29.416467  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:29.306524 16854 raft_consensus.cc:2804] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:29.386744 16888 raft_consensus.cc:697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:29.401607 16843 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.017s	sys 0.003s
I20260709 12:29:29.449790  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:29.415320  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:29.353312 16902 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.426252 16812 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:29.375509  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 111166us
I20260709 12:29:29.431074 16874 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.457376 16856 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:29.429905  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 601432us
I20260709 12:29:29.392379 16891 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.458216 16856 raft_consensus.cc:3055] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:29.458375 16891 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:29.458770 16856 raft_consensus.cc:740] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:29.324965 16822 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.445956  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4070) took 2908 ms. Trace:
W20260709 12:29:29.440308  7777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.011s	sys 0.046s
I20260709 12:29:29.448414  7775 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:29.388674 16892 raft_consensus.cc:740] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:29.448123  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:29.452397  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:29.390944 16715 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.470973  7702 leader_election.cc:400] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 20. Current term is 21.
I20260709 12:29:29.473080 16715 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:29.388556 16728 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.473613 16879 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:29.474826 16663 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:29.475327 16663 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:29.475664 16847 raft_consensus.cc:3055] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:29.476135 16847 raft_consensus.cc:740] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.476244 16854 raft_consensus.cc:697] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.476527 16663 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.477159 16847 consensus_queue.cc:260] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.477541 16854 consensus_queue.cc:237] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.478845 16847 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:29.447006  7807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1474) took 1167 ms. Trace:
W20260709 12:29:29.463068  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 680950us
W20260709 12:29:29.441701 16910 log.cc:927] Time spent T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:29:29.468847 16892 consensus_queue.cc:260] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.470305  7810 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:29.446774 16920 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:29.480288  7729 raft_consensus.cc:1275] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:29:29.502301  8414 raft_consensus.cc:1240] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:29.481107 16715 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:29.461894  7731 rpcz_store.cc:276] 0709 12:29:26.537592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.490279 (+952687us) service_pool.cc:224] Handling call
0709 12:29:27.490995 (+   716us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:27.492086 (+  1091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.492145 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:27.493354 (+  1209us) log.cc:844] Serialized 52 byte log entry
0709 12:29:27.625323 (+131969us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:27.625402 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.625540 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.053654 (+1428114us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:29.430872 (+377218us) raft_consensus.cc:1685] finished
0709 12:29:29.430902 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.445865 (+ 14963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1012,"raft.queue_time_us":2551,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9886,"spinlock_wait_cycles":42462464,"thread_start_us":131107,"threads_started":1,"wal-append.queue_time_us":247782}
W20260709 12:29:29.491629 16848 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.608s	user 0.015s	sys 0.022s
I20260709 12:29:29.487211  7807 rpcz_store.cc:276] 0709 12:29:28.279436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:28.279780 (+   344us) service_pool.cc:224] Handling call
0709 12:29:28.345365 (+ 65585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:28.356458 (+ 11093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:28.356495 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:28.386940 (+ 30445us) log.cc:844] Serialized 26 byte log entry
0709 12:29:28.553810 (+166870us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:29:28.553885 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:28.554027 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.446756 (+892729us) raft_consensus.cc:1685] finished
0709 12:29:29.446779 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.446944 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39860,"raft.queue_time_us":11123,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":10884,"thread_start_us":142631,"threads_started":1,"wal-append.queue_time_us":271310}
I20260709 12:29:29.494474  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:29.508960  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:29.510538  7653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:29.511448  7653 raft_consensus.cc:3060] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:29.464164 16874 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:29.455680 16812 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:29.501504  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:29.481069  7778 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.521909  7735 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:29.496166  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:29.514634  7808 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 29. Current term is 30. Ops: []
I20260709 12:29:29.516484  7655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" is_pre_election: true
I20260709 12:29:29.512884  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.516484  7662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2"
I20260709 12:29:29.526194  7662 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:29.519573  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.527917  7729 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:29:29.507090  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:29.501233 16853 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.232s	user 0.000s	sys 0.005s
I20260709 12:29:29.532243 16916 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:29.522271  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.449601 16820 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:29.542384  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 84974.6us
I20260709 12:29:29.482659 16888 consensus_queue.cc:237] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.555241  7655 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:29.499321 16885 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:29.550199  7730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1926) took 2929 ms. Trace:
I20260709 12:29:29.567868  7778 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.567314  7730 rpcz_store.cc:276] 0709 12:29:26.621039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.496840 (+875801us) service_pool.cc:224] Handling call
0709 12:29:27.523626 (+ 26786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:27.524169 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.524214 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:27.554005 (+ 29791us) log.cc:844] Serialized 26 byte log entry
0709 12:29:27.910217 (+356212us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:27.910269 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.910367 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.491737 (+1581370us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:29.532879 (+ 41142us) raft_consensus.cc:1685] finished
0709 12:29:29.532921 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.550056 (+ 17135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245,"raft.queue_time_us":120,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11469,"spinlock_wait_cycles":18201984,"thread_start_us":295767,"threads_started":1,"wal-append.queue_time_us":445644}
I20260709 12:29:29.465943 16822 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.565300  7653 raft_consensus.cc:2468] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:29.543735 16892 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:29.541383 16916 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.506407 16754 consensus_queue.cc:1059] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:29:29.549522 16663 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:29.582032 16920 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:29.585480 16915 raft_consensus.cc:2804] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:29.550698 16715 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:29.550213 16841 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.593974  7662 raft_consensus.cc:2468] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
W20260709 12:29:29.600859  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71733.2us
W20260709 12:29:29.603935  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59762us
W20260709 12:29:29.609916  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 81185.2us
I20260709 12:29:29.575505 16885 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.612064 16885 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.617338  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:29.618131  7735 raft_consensus.cc:3060] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:29.614921  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:29.621086  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:29.622659  7624 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:29.608383  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
I20260709 12:29:29.623744  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:29.583617 16916 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:29.576424 16864 consensus_queue.cc:1059] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:29:29.644116  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.645171  7731 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:29.605952 16715 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:29.614562 16920 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:29.577286 16595 consensus_queue.cc:1048] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:29:29.654313  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:29.617635 16854 raft_consensus.cc:2804] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:29.618315 16915 raft_consensus.cc:697] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.673630  7730 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:29.631094 16830 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:29.644418 16843 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.584367 16856 consensus_queue.cc:260] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.694373  7735 raft_consensus.cc:2468] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:29.661957 16715 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:29.706216 16914 consensus_queue.cc:1059] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.140s
I20260709 12:29:29.707782 16914 raft_consensus.cc:3055] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:29.708294 16914 raft_consensus.cc:740] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.709534 16914 consensus_queue.cc:260] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:29.685623 16857 consensus_meta.cc:287] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 1.277s	user 0.009s	sys 0.009s
I20260709 12:29:29.666908 16377 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:29.721697 16923 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:29.722934 16915 consensus_queue.cc:237] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.723843 16844 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:29.724720 16921 log.cc:927] Time spent T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:29:29.759631  8406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1779) took 1185 ms. Trace:
I20260709 12:29:29.760008 16933 raft_consensus.cc:3055] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:29.760453 16933 raft_consensus.cc:740] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:29.760218  8406 rpcz_store.cc:276] 0709 12:29:28.574221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:28.580177 (+  5956us) service_pool.cc:224] Handling call
0709 12:29:29.068694 (+488517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:29.291494 (+222800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:29.759325 (+467831us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b5400587850. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:29.759541 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074207488}
I20260709 12:29:29.761556 16933 consensus_queue.cc:260] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.762732 16825 raft_consensus.cc:3055] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:29.763352 16825 raft_consensus.cc:740] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.763545 16797 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:29.764482 16797 raft_consensus.cc:3055] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:29.764293 16825 consensus_queue.cc:260] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.763365 16930 consensus_queue.cc:1048] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.116s
I20260709 12:29:29.766155 16930 consensus_queue.cc:1243] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer d72176ccaed049958ac68f706d4f052d log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:29:29.769345 16841 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:29.772265 16866 log.cc:927] Time spent T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 1.234s	user 0.003s	sys 0.000s
I20260709 12:29:29.772970 16885 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.774504  7657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1682) took 2054 ms. Trace:
I20260709 12:29:29.774374 16830 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:29.774998  7657 rpcz_store.cc:276] 0709 12:29:27.719803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.724790 (+  4987us) service_pool.cc:224] Handling call
0709 12:29:27.729262 (+  4472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:27.748176 (+ 18914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:27.748215 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:27.774374 (+ 26159us) log.cc:844] Serialized 26 byte log entry
0709 12:29:27.951722 (+177348us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:27.951785 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:27.951907 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:29.771076 (+1819169us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:29.774060 (+  2984us) raft_consensus.cc:1685] finished
0709 12:29:29.774103 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:29.774392 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":292064,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":96769,"spinlock_wait_cycles":19432704,"thread_start_us":137154,"threads_started":1,"wal-append.queue_time_us":598390}
W20260709 12:29:29.776577  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:29.782116 16820 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.783336 16909 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.784056 16909 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:29:29.787485 16920 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.790042 16879 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:29.703158 16843 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.792196 16879 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:29.792564 16822 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.743193 16857 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.794497 16843 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:29.687798 16877 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 12:29:29.796243 16877 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:29.796667 16877 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:29.797708 16877 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.788095 16902 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.764861 16797 raft_consensus.cc:740] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:29.793397 16856 raft_consensus.cc:3060] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:29.811686 16914 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:29.770093 16841 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:29.765446 16825 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:29.793558 16879 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.667191 16874 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:29:29.753255 16923 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.762988 16933 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:29.866806 16797 consensus_queue.cc:260] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.813687 16857 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.859022 16877 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:29.893139 16797 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:29.827167  8380 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:30.444801  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 617653us
I20260709 12:29:29.847496 16844 consensus_queue.cc:1048] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:29.904587 16923 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.912190 16841 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.930114 16891 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:30.449407 16891 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:29.930490 16812 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:29.936079 16916 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:30.453763 16916 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:30.149452 16892 consensus_meta.cc:287] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 12:29:30.454567  7734 raft_consensus.cc:1240] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
W20260709 12:29:30.213086  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:30.202337 16377 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:30.222688 16928 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:29:30.223507 16715 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:30.238739 16847 consensus_meta.cc:287] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.738s	user 0.015s	sys 0.000s
W20260709 12:29:30.257035 16932 log.cc:927] Time spent T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:29:30.277655 16862 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.410578 16938 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:30.463611 16938 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:30.456820 16377 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:30.466364  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:30.420959 16933 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:30.445020 16953 raft_consensus.cc:2804] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:29:29.728485  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 69136.9us
W20260709 12:29:30.418848  7702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.006s	sys 0.010s
I20260709 12:29:30.472160  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:30.841612  7730 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:30.469614 16953 raft_consensus.cc:697] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:30.478102  7737 raft_consensus.cc:1240] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 20. Current term is 21. Ops: []
I20260709 12:29:30.466383 16938 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.488911  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:30.473697 16933 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:30.507822  7659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45926 (request call id 3342) took 2989 ms (client timeout 1690 ms). Trace:
W20260709 12:29:30.578589  7775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.011s	sys 0.000s
I20260709 12:29:30.589238  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:30.642663  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1778) took 2076 ms. Trace:
W20260709 12:29:30.644804  8379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.005s	sys 0.009s
I20260709 12:29:30.664587  7734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4104) took 1226 ms. Trace:
W20260709 12:29:30.672487  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:30.674726  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:30.678987  7626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.001s	sys 0.013s
W20260709 12:29:30.680882  7778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.032s	sys 0.099s
I20260709 12:29:30.683305  7661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1685) took 2957 ms. Trace:
W20260709 12:29:30.687603  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:30.715415  7656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.026s	sys 0.035s
I20260709 12:29:30.464119 16715 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:30.489046 16728 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.453800 16812 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.676278 16753 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:30.920442 16753 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:30.698153 16666 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:30.150524 16854 raft_consensus.cc:697] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:30.926695 16753 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.926489 16854 consensus_queue.cc:237] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:30.474162 16892 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:30.760737  7627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.406s	user 0.023s	sys 0.097s
W20260709 12:29:30.843824  7702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.006s	sys 0.012s
I20260709 12:29:30.844530  7732 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 26: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:30.932062 16892 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:30.932720  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 1.19223e+06us
W20260709 12:29:30.861259  8379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.247s	user 0.005s	sys 0.015s
W20260709 12:29:30.933773  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.32276e+06us
I20260709 12:29:30.934572  7702 leader_election.cc:400] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22.
I20260709 12:29:30.860805  7734 rpcz_store.cc:276] 0709 12:29:29.437983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:29.453995 (+ 16012us) service_pool.cc:224] Handling call
0709 12:29:29.454404 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:30.455087 (+1000683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:30.664242 (+209155us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54000730d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:30.664499 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299670400}
I20260709 12:29:30.934957  7702 leader_election.cc:403] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:29:30.861631  7626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.316s	user 0.005s	sys 0.013s
W20260709 12:29:30.936414  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 1.42929e+06us
I20260709 12:29:30.936692  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:30.932196  7627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.578s	user 0.024s	sys 0.098s
I20260709 12:29:30.861826  7661 rpcz_store.cc:276] 0709 12:29:27.726089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.726283 (+   194us) service_pool.cc:224] Handling call
0709 12:29:30.510124 (+2783841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:30.510514 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:30.510556 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:30.510626 (+    70us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:29:30.512610 (+  1984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:30.512756 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:30.682928 (+170172us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b5400398fd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:30.683148 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6363371904}
W20260709 12:29:30.863736  7659 rpcz_store.cc:269] 0709 12:29:27.518147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:27.518597 (+   450us) service_pool.cc:224] Handling call
0709 12:29:30.507272 (+2988675us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400398fd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:30.507579 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3714441472}
I20260709 12:29:30.934880 16892 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:30.862439  8412 rpcz_store.cc:276] 0709 12:29:28.566692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:28.567714 (+  1022us) service_pool.cc:224] Handling call
0709 12:29:28.582959 (+ 15245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:29.441043 (+858084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:30.642248 (+1201205us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54004e5cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:30.642573 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854001920}
W20260709 12:29:30.866950  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:30.944671  7700 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.524s (SENT)
W20260709 12:29:30.945384  7700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.76.196:36487 timed out after 1.540s (SENT) [suppressed 22 similar messages]
I20260709 12:29:30.866259 16920 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:30.868031  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:30.868386  7737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4098) took 1610 ms. Trace:
I20260709 12:29:30.947531  7737 rpcz_store.cc:276] 0709 12:29:29.258032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:29.268229 (+ 10197us) service_pool.cc:224] Handling call
0709 12:29:29.268714 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:30.843741 (+1575027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:30.868146 (+ 24405us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400211e54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:30.868317 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2780823296}
W20260709 12:29:30.948087  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 80126us
I20260709 12:29:30.949332  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:30.868674 16377 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:30.870695 16933 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:30.882330  7778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.238s	user 0.032s	sys 0.103s
W20260709 12:29:30.885483  7775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.017s	sys 0.001s
I20260709 12:29:30.845075 16953 consensus_queue.cc:237] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:30.924531 16666 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:30.936905 16879 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:30.938486  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:30.938596  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 1.70792e+06us
I20260709 12:29:30.938772 16728 consensus_queue.cc:1059] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:30.939450 16891 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519) with higher term. Message: Invalid argument: T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 21. Current term is 22.
I20260709 12:29:30.940039 16812 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:29:30.863356  7656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.028s	sys 0.036s
I20260709 12:29:30.940919  7702 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:30.934887  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:30.868573 16916 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:30.990477  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:30.990895  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 52501us
I20260709 12:29:30.991873  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:30.992847  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 52018us
W20260709 12:29:30.991873  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57085.9us
I20260709 12:29:30.977075  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:30.950815 16377 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:30.974172 16715 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:30.974568 16723 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:30.946755 16920 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:30.978313 16666 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:30.978525  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:31.005466  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.005833  8412 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:30.985939  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:30.994350  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:30.995064  8407 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:30.995383 16933 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:29:30.995517 16892 raft_consensus.cc:2804] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:30.995543 16939 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:31.003880  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:30.997776  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:31.012315 16723 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.032428 16892 raft_consensus.cc:697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:31.033676 16892 consensus_queue.cc:237] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.001677 16802 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:30.996109 16916 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.035828 16802 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.032471 16723 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:31.011786  7627 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.167s (SENT)
I20260709 12:29:31.039786  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:30.989936 16812 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.043651 16812 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:31.045267 16377 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.046873 16802 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.001595 16918 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:31.047783 16920 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.006151 16753 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:31.048079 16918 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.049686  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:30.992770 16864 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:31.051704 16864 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.054263 16864 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:31.016561  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:31.079321  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 81663.7us
I20260709 12:29:31.073235  8412 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:31.084056  7803 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
W20260709 12:29:31.067428  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 122053us
I20260709 12:29:31.083262  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.083439  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:30.989880 16859 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:30.992480 16728 raft_consensus.cc:3055] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:31.087667  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.107358  7737 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:31.037925 16916 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:31.110852  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.112479  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.113276  7805 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:31.001596 16860 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:31.117149  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.118044  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.031553 16891 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:29:31.055797  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 45273.7us
I20260709 12:29:31.128890  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.081164  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:31.099491  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.128167  7804 raft_consensus.cc:2393] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 27: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:31.103039  7734 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:29:31.109972  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.132912  7810 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:31.088232  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 84463.8us
I20260709 12:29:31.112437 16939 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:31.130875  7730 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 26: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:31.102557  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.100965 16802 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:31.103502 16933 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:31.055286 16871 consensus_queue.cc:1059] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.089s
I20260709 12:29:31.170714 16871 raft_consensus.cc:3055] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:31.171348 16871 raft_consensus.cc:740] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:31.172523 16871 consensus_queue.cc:260] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.147008 16923 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:31.175012 16923 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.175762 16923 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.139832 16860 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.180279  7735 raft_consensus.cc:1275] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:29:31.180787  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.181782  8407 raft_consensus.cc:2468] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 28.
I20260709 12:29:31.184055 16860 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.184367  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:31.147068  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.149466  7734 raft_consensus.cc:1240] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:31.055585 16841 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.084s
I20260709 12:29:31.051031 16918 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.187011 16923 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.198990 16923 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:31.199667 16923 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.127211 16920 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.201395 16920 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.202160 16923 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.198998 16844 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:29:31.125648 16891 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:31.203925  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.205732 16920 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.203985 16844 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:31.158946  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.163291 16802 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.111171 16728 raft_consensus.cc:740] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:31.140942  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:31.188293  7808 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 22.
I20260709 12:29:31.112594 16859 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:31.216970  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 76128us
I20260709 12:29:31.218626  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.226359  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.229964  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.241153  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.240897 16812 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.216131 16891 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:31.242147  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.242623  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.244091  7729 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:29:31.244493  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.245587  7809 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 19: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:31.247830  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:31.248452 16967 log.cc:927] Time spent T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:29:31.249029  8406 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:31.243803  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.251866  8408 raft_consensus.cc:2393] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:31.255985 16728 consensus_queue.cc:260] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.257807 16841 consensus_queue.cc:1048] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:31.258152 16728 raft_consensus.cc:3060] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:31.259413  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 247658us
I20260709 12:29:31.267880 16871 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:31.148731 16854 consensus_queue.cc:1059] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:29:31.281420  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.282172 16854 consensus_queue.cc:1243] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:29:31.282873 16854 raft_consensus.cc:3055] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:31.163614 16939 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.219172 16844 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.286015 16859 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:31.286360  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.287454  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.287899  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.289686 16812 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:31.231739  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.295557  7806 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:31.298722  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:31.216575 16909 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.220206 16923 raft_consensus.cc:2804] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:31.218232 16802 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:31.396736  7803 raft_consensus.cc:2468] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
W20260709 12:29:31.401320  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 102773us
I20260709 12:29:31.402058 16860 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:31.402951  7808 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:31.374164  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:31.403846  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.319962  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.404897  7811 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:31.357170  7807 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
W20260709 12:29:31.327382  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.371409  7737 raft_consensus.cc:2468] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:31.313843 16916 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.424567 16916 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.402575 16860 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:31.431289 16860 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:31.403061 16896 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:31.435724 16896 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.436123 16896 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:31.447381 16933 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.448100 16933 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.450614 16933 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:31.452390 16844 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:31.453454  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.454244  7806 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:31.311678 16824 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.394445 16923 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.469666  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.414196  7803 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:29:31.346822  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 265744us
I20260709 12:29:31.423812 16939 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:31.450928  7553 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 272) took 42811 ms (client timeout 15000 ms). Trace:
I20260709 12:29:31.459590  8379 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.311671 16854 raft_consensus.cc:740] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:31.474623  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.342125 16377 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.500474  7778 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:31.427111 16916 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:31.507754  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48299.1us
I20260709 12:29:31.472553 16923 raft_consensus.cc:3060] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:31.471630 16753 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:31.532812 16859 consensus_queue.cc:1059] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:31.526353  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
I20260709 12:29:31.402848 16909 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.524302  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:31.501843  7553 rpcz_store.cc:269] 0709 12:28:48.639806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.693290 (+ 53484us) service_pool.cc:224] Handling call
0709 12:29:31.450810 (+42757520us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:27.475807 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:27.529592 (+ 53785us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:27.529677 (+    85us) write_op.cc:435] Acquired schema lock
0709 12:29:27.529709 (+    32us) tablet.cc:662] Decoding operations
0709 12:29:27.700688 (+170979us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:27.700838 (+   150us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:27.700883 (+    45us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:29:27.703404 (+  2521us) tablet.cc:701] Row locks acquired
0709 12:29:27.703462 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:29:27.703922 (+   460us) write_op.cc:270] Start()
0709 12:29:27.704179 (+   257us) write_op.cc:276] Timestamp: P: 1783600167703724 usec, L: 0
0709 12:29:27.704205 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:29:27.728718 (+ 24513us) log.cc:844] Serialized 8631 byte log entry
0709 12:29:29.147239 (+1418521us) op_driver.cc:464] REPLICATION: finished
0709 12:29:29.429955 (+282716us) write_op.cc:301] APPLY: starting
0709 12:29:29.453714 (+ 23759us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:29.901253 (+447539us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:29.901434 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:30.290914 (+389480us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:30.291653 (+   739us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:29:30.291708 (+    55us) write_op.cc:312] APPLY: finished
0709 12:29:30.292463 (+   755us) log.cc:844] Serialized 196 byte log entry
0709 12:29:30.295440 (+  2977us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:30.295759 (+   319us) write_op.cc:454] Released schema lock
0709 12:29:30.295886 (+   127us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:30.296545 (+   659us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:30.296569 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4016059,"reported_tablets":49,"spinlock_wait_cycles":8960,"child_traces":[["op",{"apply.queue_time_us":195299,"apply.run_cpu_time_us":45321,"apply.run_wall_time_us":892496,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":307243,"prepare.run_cpu_time_us":25562,"prepare.run_wall_time_us":1098294,"raft.queue_time_us":617924,"raft.run_cpu_time_us":75286,"raft.run_wall_time_us":211912,"replication_time_us":1442812,"thread_start_us":1076336,"threads_started":5,"wal-append.queue_time_us":689136,"wal-append.run_cpu_time_us":65180,"wal-append.run_wall_time_us":2313217}]]}
I20260709 12:29:31.521720 16953 raft_consensus.cc:2804] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:31.554740  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.472851 16824 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.555691  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:31.555914  7806 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
W20260709 12:29:31.571177  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.575822  7730 raft_consensus.cc:1275] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 21. (index mismatch)
I20260709 12:29:31.596690  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:31.597388  7810 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:31.577277  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.507984 16723 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.524320 16377 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.581032 16854 consensus_queue.cc:260] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.539515 16918 raft_consensus.cc:2764] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:29:31.529079 16864 raft_consensus.cc:2804] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:29:31.617362  7625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.014s
W20260709 12:29:31.620261  7625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.012s	sys 0.014s
I20260709 12:29:31.585670  7734 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
W20260709 12:29:31.586831  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 62641.1us
I20260709 12:29:31.587440 16933 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:31.576571 16779 consensus_queue.cc:1059] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:31.601545  7811 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
W20260709 12:29:31.607434  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 347169us
I20260709 12:29:31.550055 16907 raft_consensus.cc:3055] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:31.616353  7809 pending_rounds.cc:85] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 15
I20260709 12:29:31.643289  7809 pending_rounds.cc:107] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:29:31.643710  7809 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:31.585080 16909 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.622572  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:31.624400  7702 connection.cc:256] RPC call timeout handler was delayed by -3.1254449420002857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 12:29:31.630671  8405 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:29:31.642659  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:31.646903  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:31.650971  7702 connection.cc:256] RPC call timeout handler was delayed by -3.0795449420002114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 12:29:31.651728  7702 connection.cc:256] RPC call timeout handler was delayed by -2.9949449420000747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 12:29:31.652359  7702 connection.cc:256] RPC call timeout handler was delayed by -2.9463449419999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 12:29:31.652791  7702 connection.cc:256] RPC call timeout handler was delayed by -2.9166449420004028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 12:29:31.653250  7702 connection.cc:256] RPC call timeout handler was delayed by -2.9157449420004014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:29:31.653648  7702 connection.cc:256] RPC call timeout handler was delayed by -2.8725449420003315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 12:29:31.654026  7702 connection.cc:256] RPC call timeout handler was delayed by -2.7852449420001903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 12:29:31.654354  7702 connection.cc:256] RPC call timeout handler was delayed by -2.782544942000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
I20260709 12:29:31.604564 16715 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:29:31.655145  7702 connection.cc:256] RPC call timeout handler was delayed by -2.77894494200018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 12:29:31.655411  8412 raft_consensus.cc:1240] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
W20260709 12:29:31.661007  7702 connection.cc:256] RPC call timeout handler was delayed by -2.7564449420001438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 12:29:31.661456  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6988449420000507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 12:29:31.604851 16953 raft_consensus.cc:697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:31.661758  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6853449420000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
I20260709 12:29:31.661922  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:31.662352  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6727449420000085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 12:29:31.662731  7702 connection.cc:256] RPC call timeout handler was delayed by -2.6592449419999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20179999999999998s for a response
W20260709 12:29:31.664126  7702 connection.cc:256] RPC call timeout handler was delayed by -2.5917449420003322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 12:29:31.664460  7702 connection.cc:256] RPC call timeout handler was delayed by -2.3415449420003824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2371s for a response
W20260709 12:29:31.664738  7702 connection.cc:256] RPC call timeout handler was delayed by -2.25334494200024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2469s for a response
W20260709 12:29:31.665030  7702 connection.cc:256] RPC call timeout handler was delayed by -1.9725449420002406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2781s for a response
W20260709 12:29:31.665328  7702 connection.cc:256] RPC call timeout handler was delayed by -1.7502449420003359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3028s for a response
I20260709 12:29:31.666445  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:31.666785  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.668143  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:31.668568  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.668964  7807 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:31.670462  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.671422  7728 raft_consensus.cc:2393] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:31.671680  7553 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 273) took 18948 ms (client timeout 15000 ms). Trace:
W20260709 12:29:31.672322  7553 rpcz_store.cc:269] 0709 12:29:12.716252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.643935 (+18927683us) service_pool.cc:211] Skipping call since client already timed out
0709 12:29:31.654953 (+ 11018us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:29:31.670346  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.678758  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.680239  8412 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 25: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:31.681813  7733 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:29:31.627287 16933 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:31.686967  7810 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:31.687005  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:31.689801  8405 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:31.679988  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.691568  7728 raft_consensus.cc:2393] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:31.641063 16907 raft_consensus.cc:740] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:31.709038  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.709563  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.710719  7733 raft_consensus.cc:2393] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 27: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:31.694428  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.705572  7803 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 26: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:31.619737 16871 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
I20260709 12:29:31.704607  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.713507  7731 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
I20260709 12:29:31.683759 16920 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:31.627825 16938 consensus_queue.cc:1048] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.057s
I20260709 12:29:31.719563  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.729434  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.720388  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:31.734347  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 49211.2us
I20260709 12:29:31.737699  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.692755 16933 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:31.647332 16891 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.732923  7734 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:31.704624 16854 raft_consensus.cc:3060] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:31.735455  7732 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.010s	sys 0.031s
W20260709 12:29:31.749881  7732 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.011s	sys 0.031s
I20260709 12:29:31.748327  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.755687 16824 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.759418  8408 raft_consensus.cc:1275] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:29:31.759583  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.779543  7737 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:31.742242 16892 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:31.787514  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:31.775592 16879 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:31.809953  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.779768 16907 consensus_queue.cc:260] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:31.826320 16971 log.cc:927] Time spent T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.110s	user 0.002s	sys 0.000s
I20260709 12:29:31.840775  7656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40408 (request call id 1691) took 2345 ms. Trace:
W20260709 12:29:31.844146  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.845044 16909 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.841542  7656 rpcz_store.cc:276] 0709 12:29:29.494828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:29.494971 (+   143us) service_pool.cc:224] Handling call
0709 12:29:29.495398 (+   427us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:29:29.511917 (+ 16519us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:29:29.672705 (+160788us) log.cc:844] Serialized 14 byte log entry
0709 12:29:31.394455 (+1721750us) log.cc:844] Serialized 14 byte log entry
0709 12:29:31.675576 (+281121us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:29:31.677425 (+  1849us) log.cc:844] Serialized 78 byte log entry
0709 12:29:31.677607 (+   182us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:31.678002 (+   395us) log.cc:844] Serialized 14 byte log entry
0709 12:29:31.682628 (+  4626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:31.682965 (+   337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:31.836413 (+153448us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b18003a9ba0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee353f3 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:31.840198 (+  3785us) raft_consensus.cc:1685] finished
0709 12:29:31.840243 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:31.840652 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36127,"prepare.queue_time_us":197323,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":153,"raft.queue_time_us":150408,"raft.run_cpu_time_us":734,"raft.run_wall_time_us":23411,"spinlock_wait_cycles":7909376,"thread_start_us":1946993,"threads_started":4,"wal-append.queue_time_us":896012,"wal-append.run_cpu_time_us":1965,"wal-append.run_wall_time_us":1017904}
I20260709 12:29:31.815843 16545 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.816434 16860 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.846805 16879 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:31.847517  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:31.847925 16377 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:31.848532 16879 raft_consensus.cc:2764] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:29:31.848809 16860 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.849134  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.846853 16545 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.852288  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.850847  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:31.853113  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:31.821624 16723 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:31.854104 16916 raft_consensus.cc:2764] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:29:31.739030 16920 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.823973 16891 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:31.809783 16953 consensus_queue.cc:237] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.845729 16909 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:31.853848  8436 debug-util.cc:398] Leaking SignalData structure 0x7b080026a420 after lost signal to thread 6454
W20260709 12:29:31.866201  8436 debug-util.cc:398] Leaking SignalData structure 0x7b080032d380 after lost signal to thread 7593
W20260709 12:29:31.868414  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800329a00 after lost signal to thread 7676
W20260709 12:29:31.869015  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08001c1340 after lost signal to thread 7677
W20260709 12:29:31.869442  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800322360 after lost signal to thread 7678
W20260709 12:29:31.869841  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800256d40 after lost signal to thread 7679
W20260709 12:29:31.870208  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9160 after lost signal to thread 7680
W20260709 12:29:31.870589  8436 debug-util.cc:398] Leaking SignalData structure 0x7b080003fee0 after lost signal to thread 7681
W20260709 12:29:31.870980  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800350020 after lost signal to thread 7682
W20260709 12:29:31.871675  8436 debug-util.cc:398] Leaking SignalData structure 0x7b0800320de0 after lost signal to thread 7686
I20260709 12:29:31.814543  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.874079 16545 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:31.877728 16907 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:31.879648 16976 consensus_queue.cc:1059] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:31.877974  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:31.880954 16879 consensus_queue.cc:1048] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:29:31.881768 16976 raft_consensus.cc:3055] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:31.881961  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:31.882371 16976 raft_consensus.cc:740] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:31.884189  7700 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:31.889140  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08003509a0 after lost signal to thread 7761
W20260709 12:29:31.890818  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5740 after lost signal to thread 7836
W20260709 12:29:31.891163  8436 debug-util.cc:398] Leaking SignalData structure 0x7b080037d2a0 after lost signal to thread 7837
W20260709 12:29:31.892529  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08002ffe60 after lost signal to thread 8439
W20260709 12:29:31.892858  8436 debug-util.cc:398] Leaking SignalData structure 0x7b08001f5c00 after lost signal to thread 8440
I20260709 12:29:31.896540  8405 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 26.
W20260709 12:29:31.902361  7777 leader_election.cc:336] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:31.898030 16545 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:31.906546  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 92170.4us
I20260709 12:29:31.920415  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:31.916726  7777 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:31.809255 16812 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 12:29:31.929563  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.933348  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.916384 16891 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:31.906235 16974 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:31.943141  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:31.916267 16545 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:31.950465  7627 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.939240  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:31.952431  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 338283us
I20260709 12:29:31.952521  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:31.934057 16916 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:31.947428 16856 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:31.965173  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:31.957605  7778 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:31.944574 16891 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.956037 16545 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:31.932242 16896 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:31.973295  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.973554  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.973986  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.974941  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:31.982816  7733 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:31.941416 16977 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:31.982102  7737 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 28.
W20260709 12:29:31.984400  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 274098us
I20260709 12:29:31.962306 16974 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:31.965364 16892 raft_consensus.cc:2764] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:29:31.992966  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:31.991138  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:31.990010  8380 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:31.996624  8379 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.968603 16891 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:31.997689  7736 raft_consensus.cc:1275] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:31.997583  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:31.998621  8414 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 30.
I20260709 12:29:31.966894 16864 raft_consensus.cc:697] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:31.999730  7731 raft_consensus.cc:1275] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:29:32.008921  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:32.011044  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44313.6us
I20260709 12:29:32.001008  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:31.967314 16856 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:32.001626  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:31.980821 16728 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:31.983393 16723 raft_consensus.cc:2804] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:31.965416 16779 consensus_queue.cc:1059] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:29:32.015314  7729 raft_consensus.cc:1275] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:29:32.019869  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:32.002437 16824 raft_consensus.cc:3055] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:31.994999 16916 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:31.996865 16937 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:32.003412 16666 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:31.968823 16990 consensus_queue.cc:1059] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:29:32.015327 16939 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:32.003293 16844 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:31.981148 16753 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:29:32.008061 16974 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:32.040645  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:32.045186  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 80147.6us
I20260709 12:29:32.064460  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:32.046479  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 103599us
I20260709 12:29:32.017961 16856 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:32.053450  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:32.057552  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.058276  7810 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 12:29:32.058715  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.063257  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:31.962666 16976 consensus_queue.cc:260] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:31.960783  7552 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:32.055622 16984 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.066097  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.072950 16986 raft_consensus.cc:3055] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:32.073625 16991 raft_consensus.cc:2764] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:29:32.073757 16920 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.075304 16864 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.060943 16844 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.037922 16666 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:32.033185 16824 raft_consensus.cc:740] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:32.124089 16918 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:32.104132  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:32.036849 16909 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:32.108999  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.111979  7730 raft_consensus.cc:1275] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:29:32.155725  7808 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:32.086241 16939 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:32.119086  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 118163us
I20260709 12:29:32.101017 16975 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.132606  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.109889 17000 raft_consensus.cc:2804] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:32.141386 16986 raft_consensus.cc:740] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:32.105747 16871 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:29:32.114173 16723 raft_consensus.cc:697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:32.094581  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:32.150550 16844 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:32.054443 16937 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.171836  7552 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 15 to 16, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:32.174320 16937 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.128013 16977 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.175544 16977 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.178494 16977 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:32.115880  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62548.7us
I20260709 12:29:32.114794 16930 consensus_queue.cc:1059] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
W20260709 12:29:32.181802  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.182649  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4111) took 1086 ms. Trace:
I20260709 12:29:32.154402 16918 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:32.207345 16918 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:32.214854  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:32.183485 16377 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:29:32.169695  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 75242.9us
I20260709 12:29:32.152097 16666 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:32.145421  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86779us
I20260709 12:29:32.163880 16920 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:29:32.224771  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:32.226500  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:32.185302 16991 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:29:32.146862  7812 raft_consensus.cc:1240] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:32.147671 16984 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.252449  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:32.231735  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:32.243918  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 179549us
W20260709 12:29:32.242590  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 83434.3us
I20260709 12:29:32.250016  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.248303  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:32.251083  7804 raft_consensus.cc:1275] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:29:32.258948  7804 pending_rounds.cc:85] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 15
W20260709 12:29:32.247561  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 114123us
W20260709 12:29:32.259352  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.259363  7804 pending_rounds.cc:107] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:29:32.260234  7804 raft_consensus.cc:2889] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:32.242144 16991 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.260910 16991 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:29:32.262743  7777 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.263581  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:32.265142  7777 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.267464 17000 raft_consensus.cc:697] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:32.268355  7777 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:32.269145 17000 consensus_queue.cc:237] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.258164  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:32.236994 16879 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:32.227492 16990 consensus_queue.cc:1048] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:29:32.237380 16896 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:32.169056 16975 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.318356  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:32.306686 17002 log.cc:927] Time spent T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:29:32.310223  7806 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:29:32.287649 16728 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:32.280799 17004 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:32.326192  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 99631.6us
I20260709 12:29:32.284075  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:32.169726 16974 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.221266 16987 consensus_queue.cc:1048] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
W20260709 12:29:32.340129 16995 log.cc:927] Time spent T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:29:32.332695  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:32.275638 16753 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:29:32.227722 16824 consensus_queue.cc:260] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.344240  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:32.227989 16939 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:32.347035  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.281450 16891 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.352263  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:32.354422  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70563.6us
I20260709 12:29:32.171954 16986 consensus_queue.cc:260] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:32.345451  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.326952 16871 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:29:32.336524 16802 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.008s
I20260709 12:29:32.329594 16896 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:32.399921  7729 raft_consensus.cc:1275] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:29:32.400338  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50060.4us
I20260709 12:29:32.410918  7736 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.19-25.19]
W20260709 12:29:32.378513  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 113389us
W20260709 12:29:32.406785  7627 connection.cc:256] RPC call timeout handler was delayed by -4.1434617250001793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
I20260709 12:29:32.341156 16916 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:32.360795 16728 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:32.360020 16879 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:32.360213 16859 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:32.406245  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122183us
W20260709 12:29:32.409233 16923 consensus_meta.cc:287] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.870s	user 0.009s	sys 0.008s
W20260709 12:29:32.417968  7627 connection.cc:256] RPC call timeout handler was delayed by -3.2425577640001393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 12:29:32.428205  7627 connection.cc:256] RPC call timeout handler was delayed by -3.1741577640000287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 12:29:32.414279 16891 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.420804 16986 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:32.408493 16377 raft_consensus.cc:2764] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
W20260709 12:29:32.428599  7627 connection.cc:256] RPC call timeout handler was delayed by -3.1453577639999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 12:29:32.625082  7627 connection.cc:256] RPC call timeout handler was delayed by -2.9923617250001371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2913s for a response
W20260709 12:29:32.625684  7627 connection.cc:256] RPC call timeout handler was delayed by -2.9302577640000891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:29:32.626085  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7817577640003037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
W20260709 12:29:32.626502  7627 connection.cc:256] RPC call timeout handler was delayed by -2.7106577640001888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 12:29:32.423027  7777 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.411386 16802 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.412847 16824 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:32.431768 16923 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:32.514943 16998 log.cc:927] Time spent T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 12:29:32.582973  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.325348 17005 consensus_queue.cc:1048] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:29:32.413293 17003 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.737597 17003 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.427546 16871 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.739727 16871 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:32.610963  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 180301us
I20260709 12:29:32.418535 16994 consensus_queue.cc:1048] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:29:32.420673 16920 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:32.615435  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:32.615537  8407 raft_consensus.cc:1240] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: []
I20260709 12:29:32.312346 16723 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.658039  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:32.664892  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:32.646785 16984 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:32.644932  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 353984us
W20260709 12:29:32.650302 16545 consensus_meta.cc:287] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.011s
W20260709 12:29:32.626916  7627 connection.cc:256] RPC call timeout handler was delayed by -1.826857764000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3113s for a response
I20260709 12:29:32.701363  7809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1495) took 1087 ms. Trace:
I20260709 12:29:32.687352  7804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1492) took 1132 ms. Trace:
I20260709 12:29:32.183130  7732 rpcz_store.cc:276] 0709 12:29:31.096045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.096365 (+   320us) service_pool.cc:224] Handling call
0709 12:29:31.124632 (+ 28267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:31.124928 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:29:31.753508 (+628580us) log.cc:844] Serialized 14 byte log entry
0709 12:29:32.015649 (+262141us) log.cc:844] Serialized 14 byte log entry
0709 12:29:32.181989 (+166340us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b180048bce0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffedafc0a 00007fc0084ef219 00007fbffee37a40 00007fbffee324e4 00007fbffee32455 00007fc001f1a060 00007fc001f2c716 00007fc001f3d287 00007fc001f3d208
0709 12:29:32.182194 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:32.182255 (+    61us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:32.182322 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:32.182423 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:32.182578 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155282,"prepare.queue_time_us":714844,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":89,"raft.queue_time_us":178657,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":28079,"spinlock_wait_cycles":13267328,"thread_start_us":859439,"threads_started":2,"wal-append.queue_time_us":397465}
I20260709 12:29:32.650110 16860 raft_consensus.cc:3055] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:32.669338 16991 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:32.653903 16856 consensus_meta.cc:287] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.003s
W20260709 12:29:32.647311 16982 log.cc:927] Time spent T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.193s	user 0.010s	sys 0.000s
I20260709 12:29:32.653995 16844 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.735517  7805 raft_consensus.cc:1240] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
W20260709 12:29:32.755678  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 335644us
W20260709 12:29:32.758149 16918 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.548s	user 0.017s	sys 0.004s
W20260709 12:29:32.760144  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 129205us
I20260709 12:29:32.780871  7804 rpcz_store.cc:276] 0709 12:29:31.555358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.555470 (+   112us) service_pool.cc:224] Handling call
0709 12:29:31.565365 (+  9895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:32.260900 (+695535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:32.686976 (+426076us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54001333d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:32.687273 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562010368}
I20260709 12:29:32.782167  7809 rpcz_store.cc:276] 0709 12:29:31.613564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.613760 (+   196us) service_pool.cc:224] Handling call
0709 12:29:31.616025 (+  2265us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:31.982123 (+366098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:31.982175 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:31.983065 (+   890us) log.cc:844] Serialized 52 byte log entry
0709 12:29:32.127843 (+144778us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:29:32.127991 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:32.128207 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:32.624763 (+496556us) raft_consensus.cc:1685] finished
0709 12:29:32.624788 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:32.701258 (+ 76470us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3374,"raft.queue_time_us":138162,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11215,"thread_start_us":442067,"threads_started":2,"wal-append.queue_time_us":230724}
I20260709 12:29:32.758993 16715 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:32.796531  7777 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:32.802042  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66823.1us
I20260709 12:29:32.802407  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:32.791586 16856 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:32.796478 16933 consensus_meta.cc:287] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.013s
I20260709 12:29:32.807993  7805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45836 (request call id 3167) took 1659 ms. Trace:
I20260709 12:29:32.814630  7805 rpcz_store.cc:276] 0709 12:29:31.148960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.157097 (+  8137us) service_pool.cc:224] Handling call
0709 12:29:31.157446 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:32.796024 (+1638578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:32.807683 (+ 11659us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000681d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:32.807913 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3623161728}
I20260709 12:29:32.808297 16974 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:32.810312  7777 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:32.821919  7777 leader_election.cc:336] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.822604  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:32.812546  8414 raft_consensus.cc:1240] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:29:32.807699 16844 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:32.778961 16984 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.785637 16860 raft_consensus.cc:740] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:32.825361 16933 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.741609 17021 consensus_queue.cc:1048] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
W20260709 12:29:32.840173  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 224917us
I20260709 12:29:32.824890  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:32.788981 16545 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.803489 16923 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.814298 16715 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.837064  8407 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40588 (request call id 1796) took 1205 ms. Trace:
W20260709 12:29:32.840724  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 90740.5us
W20260709 12:29:32.847693  7777 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.832382  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:32.859385  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:32.814163 16953 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.786670 16920 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.839030 16916 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:32.845692 16854 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.846107 16864 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:32.861274  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:32.857548  8407 rpcz_store.cc:276] 0709 12:29:31.632029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.632280 (+   251us) service_pool.cc:224] Handling call
0709 12:29:31.632774 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:32.742435 (+1109661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:32.836784 (+ 94349us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54007b3c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:32.836988 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835224832}
I20260709 12:29:32.866786  8408 raft_consensus.cc:1240] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:32.816625 16918 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.882905  8412 raft_consensus.cc:1240] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:32.884709  7731 raft_consensus.cc:2393] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 26: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:32.830818 16939 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:32.870657  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:32.886613  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:32.840481 17010 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:32.881834  7729 raft_consensus.cc:1240] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:32.874832 17000 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:32.888579  7777 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:32.888797  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:32.889369  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:32.893592 17011 consensus_queue.cc:1059] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:29:32.894246 16812 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:32.895299 16991 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 31 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.895669 17024 consensus_queue.cc:1059] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.077s
W20260709 12:29:32.882671 16907 consensus_meta.cc:287] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.997s	user 0.025s	sys 0.006s
I20260709 12:29:32.900851 16824 raft_consensus.cc:3055] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:32.901733 16824 raft_consensus.cc:740] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:32.903946 17018 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:32.904521 17018 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.904659 16939 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.905659 16824 consensus_queue.cc:260] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.907421 16824 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:32.908429 16892 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:32.909075 16892 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.909525 16892 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:32.886399 16666 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.009s
I20260709 12:29:32.911821 16377 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:32.912029 16666 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.832981  7552 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:32.893365 17000 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.833324 16984 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:32.903431 17008 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 12:29:32.883160 16854 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:32.916795  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:32.906213 16715 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.923650  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:32.924480  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.964279  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:32.925801  8411 raft_consensus.cc:1240] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:32.927042 16933 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:32.918628 16975 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.915488 16992 consensus_queue.cc:1048] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:29:32.895082 17011 raft_consensus.cc:3055] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:32.918207  7552 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:32.875387 16802 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:32.922055  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:32.978044  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 56074.4us
I20260709 12:29:32.940678 17003 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:32.960821  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44108us
I20260709 12:29:32.985540  7702 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:32.986553  7702 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:32.970945 16812 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:32.891430 16871 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.874832 17014 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:32.936159 17018 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:33.002413 16916 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.006628 17019 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:32.915194 16666 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:32.908790 16545 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:32.959048  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 177523us
W20260709 12:29:32.983994  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:33.027735  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.028693 16856 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.030218  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1651) took 1035 ms. Trace:
I20260709 12:29:33.030712  8411 rpcz_store.cc:276] 0709 12:29:31.994526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.994671 (+   145us) service_pool.cc:224] Handling call
0709 12:29:32.017267 (+ 22596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:32.972620 (+955353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:33.029949 (+ 57329us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400321c54. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:33.030144 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074682240}
I20260709 12:29:33.032779 17018 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.035472 17018 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.039966 16909 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:33.040743 16909 raft_consensus.cc:697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:33.041560 16984 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:33.042109  7777 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:32.872629 16953 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:33.043758  7777 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.008597 17016 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:33.044931  7777 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.045060 17016 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:33.011811  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.017450 16939 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:32.974711 17011 raft_consensus.cc:740] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:33.015014  7627 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.648s (SENT)
I20260709 12:29:33.049376  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.049659 17011 consensus_queue.cc:260] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:33.051116 17031 log.cc:927] Time spent T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.170s	user 0.003s	sys 0.000s
I20260709 12:29:32.995932 16891 raft_consensus.cc:2804] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:33.026734  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.051630 17011 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:33.041774 16909 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.042280 16856 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.063792 16856 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.064824 17008 consensus_queue.cc:1059] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:33.066516 17008 raft_consensus.cc:3055] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:33.067013 17008 raft_consensus.cc:740] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:33.045147  8412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40594 (request call id 1649) took 1065 ms. Trace:
I20260709 12:29:33.070871  8412 rpcz_store.cc:276] 0709 12:29:31.985667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.985828 (+   161us) service_pool.cc:224] Handling call
0709 12:29:31.986317 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:32.884713 (+898396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:33.044701 (+159988us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400499154. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:33.045057 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061225216}
W20260709 12:29:33.074086  7775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.071491 17008 consensus_queue.cc:260] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:32.970124  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.051707  7627 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 2.131s (SENT)
W20260709 12:29:33.076280  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 110118us
I20260709 12:29:33.076493  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.061592 16920 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.061923 17004 raft_consensus.cc:3055] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:33.078279 17004 raft_consensus.cc:740] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:33.079675  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.079967  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:33.047158  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.080511 17030 log.cc:927] Time spent T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:29:33.080904  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:33.082646  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.080526  8414 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:33.083567 17003 consensus_queue.cc:1059] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:33.084434 17003 consensus_queue.cc:1243] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 26.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:29:33.084223 17004 consensus_queue.cc:260] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.084846 17016 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.085112 16859 raft_consensus.cc:3055] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:33.085466 17004 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:33.085551 16859 raft_consensus.cc:740] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:33.086589 16859 consensus_queue.cc:260] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.090278 16990 consensus_queue.cc:1059] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:33.076478 17008 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:33.091595 16864 raft_consensus.cc:3055] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:33.092183 16864 raft_consensus.cc:740] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:33.093075 16864 consensus_queue.cc:260] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.094185 16953 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.091435  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.095329  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.095587  8408 raft_consensus.cc:2410] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:29:33.083009 16976 raft_consensus.cc:3060] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:33.098160  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.099821 16864 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:33.051863 16918 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:33.102330 17000 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:33.103404  8379 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.103972  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:33.105670  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 58568.5us
I20260709 12:29:33.106585 17019 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:33.107558  7777 leader_election.cc:336] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.108048  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:33.109350  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.109997  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.110616  7777 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.097249  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.112635  7736 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:33.111699 16975 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:33.113761  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.114208  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 62536.5us
W20260709 12:29:33.115047  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.066974 16856 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:33.120255  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.104034 16859 raft_consensus.cc:3060] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:33.122009  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.135771  8380 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:33.127821  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.138020  8412 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:33.115101  7777 leader_election.cc:336] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.126408  8414 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 31.
I20260709 12:29:33.133155  7805 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:33.139786  7777 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:33.027881 16854 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:33.134198  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.111589 17003 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:33.164871  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:33.159173  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44103.1us
I20260709 12:29:33.160516  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:33.118042 16939 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:33.161659  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:33.163674  7809 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:33.164765  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.158951  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:33.166147  7811 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
W20260709 12:29:33.168201  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 181721us
I20260709 12:29:33.142513 16918 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:33.120776 16984 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:33.163599  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.037999 16860 consensus_queue.cc:260] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.169483  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:33.180908  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.182204  7737 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:29:33.165851 16930 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:29:33.181221  8379 leader_election.cc:336] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.199806  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:33.195802  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:33.127633 16975 consensus_queue.cc:1059] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
W20260709 12:29:33.211464  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.175405 16812 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:33.212946  7737 raft_consensus.cc:1240] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:29:33.215829  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:33.225399  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.204859 16891 raft_consensus.cc:697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:33.172961  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.243989  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.194701 16930 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:33.208124 17003 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.254172  7775 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:33.244501  7777 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.221892 16953 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:33.224756 16860 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:33.249884  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:33.251361  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.257752  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:33.261766  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 88850.1us
I20260709 12:29:33.252051 16930 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:33.252027 16879 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:33.258244 16918 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:33.274511  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.259240 16975 raft_consensus.cc:3055] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:33.275677  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:33.268325 16990 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:33.290179  7777 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.302373  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.303818  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.290184  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:33.284375  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.262845 16854 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:29:33.292133  8378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.273334 16802 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:29:33.291615  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 122232us
I20260709 12:29:33.286644  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.309507  7736 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:33.284770 16812 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.270907 16377 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:33.320256  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.322685  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.321540  8379 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.262080 17003 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.327208  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.332244  7737 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:29:33.322229 16879 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:33.330588 17039 consensus_queue.cc:1059] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:33.305738 16854 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:33.333945  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:33.323473  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.300490 16974 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:33.293951 16975 raft_consensus.cc:740] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:33.347196  7777 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.353904  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:33.354369  7803 raft_consensus.cc:1275] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:29:33.355345  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.337123  7809 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:29:33.337488  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.338853  7731 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:33.357692  7810 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
W20260709 12:29:33.356135  7777 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.341351  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.274255 16892 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.343670  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.343812  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.384112  7812 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:29:33.344336 16854 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:33.344082 16891 consensus_queue.cc:237] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.345053 17039 raft_consensus.cc:3055] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:33.406363  7804 raft_consensus.cc:2393] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:33.326241 16802 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:33.301276 16918 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.352458  8412 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:33.412755  7737 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 31.
I20260709 12:29:33.424679  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.380987 17012 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:33.434677  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.436930  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.437973  7803 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:33.413789 17005 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.444662  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.406837 17039 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:33.382210  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.474165  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 44516.5us
I20260709 12:29:33.456676  7737 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:29:33.455399  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 73243us
I20260709 12:29:33.482697  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:33.480820  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.487938 17046 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:29:33.488729 16939 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.489437 16939 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.491458 16856 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.491565 17049 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:33.492501 17049 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.493475  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.494269  7811 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:33.496275 17049 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.497843  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.500221  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:33.506476  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.506239  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:33.511602  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48900.9us
I20260709 12:29:33.512811  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.432639 16984 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 12:29:33.508700  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.492256 16728 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.514658  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:33.521713  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 174227us
W20260709 12:29:33.522887  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.525543  7809 raft_consensus.cc:2410] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
W20260709 12:29:33.525885  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.510128 16871 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:33.450726 17043 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:29:33.535080  7734 raft_consensus.cc:2468] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:33.527242  8379 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:33.534374  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.550961  7737 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:29:33.552047  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.553297  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.492192 16856 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.520744 16864 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:33.562089  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.477931 17005 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.566643  7728 raft_consensus.cc:2410] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:29:33.531843 17000 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:33.528481 16974 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.444331 16975 consensus_queue.cc:260] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:33.574589  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47447.5us
I20260709 12:29:33.590626  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.577684  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.485016 16879 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.591086 16974 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.543843 17014 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:33.584695  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.514680 16992 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 12:29:33.548166 16871 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:33.613487 16871 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:33.615888 16974 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.585265  7803 raft_consensus.cc:2468] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 29.
I20260709 12:29:33.548435 17012 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.618309  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:33.517153 16933 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.045s
I20260709 12:29:33.596503  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.619107  7809 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:33.619911  7728 raft_consensus.cc:2410] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:29:33.520253 16802 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.584890 17049 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.587188 17016 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:29:33.606154  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:33.534685 16728 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.633788  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.637823  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.603466  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.623549  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.639117  7737 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:33.646831  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.657070 16856 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:33.528769 17039 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.659094 17039 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:33.662009 17005 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:33.621279 17003 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:33.667510 16918 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:33.629992  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.612056 16939 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:33.620255 16938 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:29:33.685683  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.628739 17016 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:33.702941  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.645398 16974 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.698530  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.700103  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:33.730943  7807 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:33.682813  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.734413  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.618794 17012 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.704542  7809 raft_consensus.cc:2468] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:33.711637  7805 raft_consensus.cc:1240] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:29:33.708937  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:33.746336  7810 raft_consensus.cc:1240] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:33.737255  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:33.762176  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59224.4us
I20260709 12:29:33.637229 17049 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:33.763487  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.765429  7728 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 22: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:33.768352  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:33.769287  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 263082us
I20260709 12:29:33.708421 17016 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:33.772809  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.782662  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.783809  7728 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:29:33.743497 16974 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.768981 16933 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:33.770923  7778 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.707878 16938 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.782053 16988 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:29:33.820345 16859 consensus_meta.cc:287] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.004s
I20260709 12:29:33.812674 16974 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:33.803798 17011 consensus_meta.cc:287] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.008s
I20260709 12:29:33.834533  7808 raft_consensus.cc:1275] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
I20260709 12:29:33.835307  7808 pending_rounds.cc:85] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 17
I20260709 12:29:33.835656  7808 pending_rounds.cc:107] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:29:33.835444  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.836035  7808 raft_consensus.cc:2889] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:33.828876  8379 leader_election.cc:336] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.803455 16977 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.838876  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:33.843011  8379 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.843052  7777 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.841739 16859 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:33.855468  7777 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:33.865682  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:33.837714  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66956us
I20260709 12:29:33.866649  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:33.867182  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.866539  7737 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:33.869102  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.871652  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:33.872411  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:33.872684  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:33.873395  7777 leader_election.cc:336] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:33.873706  7737 raft_consensus.cc:2410] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:29:33.873893  7777 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:33.845221 17004 consensus_meta.cc:287] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.759s	user 0.018s	sys 0.000s
I20260709 12:29:33.838999 17063 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:33.884693  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:33.866427 16930 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.006s
I20260709 12:29:33.891559  7729 raft_consensus.cc:2393] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:33.902729  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.904343  7737 raft_consensus.cc:2376] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
I20260709 12:29:33.906108  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.908149  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.913924  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.928577  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:33.930215  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:33.931619  7737 raft_consensus.cc:2393] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:33.852465 16977 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.881507 17005 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:33.957985  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:33.876549 16859 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:33.932372 16802 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:33.928329 16930 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:33.956816 16728 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:33.934598 17014 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:29:34.000669  8411 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 29: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:33.977684 17012 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:34.054148 17061 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
I20260709 12:29:34.055596 16859 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.055912  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.056542  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.057214  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.058347  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.011650 17049 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:34.058910  7734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4124) took 2085 ms (client timeout 2015 ms). Trace:
W20260709 12:29:34.064666  7734 rpcz_store.cc:269] 0709 12:29:31.973518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.973693 (+   175us) service_pool.cc:224] Handling call
0709 12:29:34.058715 (+2085022us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400248bd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.058842 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3570431232}
I20260709 12:29:34.066411  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.068427  7732 raft_consensus.cc:1275] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:29:34.069913  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.070832  7734 raft_consensus.cc:2376] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21. Re-sending same reply.
I20260709 12:29:34.076478  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.078928  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.080653  7730 raft_consensus.cc:2393] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:34.093632  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:34.096055  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 161945us
I20260709 12:29:34.094978  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:34.097931  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 224573us
I20260709 12:29:34.107654  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:34.114698  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 166895us
W20260709 12:29:34.119103  7627 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.614s (SENT)
I20260709 12:29:34.120980  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:34.127641 16860 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.852s	user 0.018s	sys 0.012s
I20260709 12:29:34.127965 17049 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:34.132045  7806 raft_consensus.cc:2410] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:29:34.135841  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.137593  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.138283  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.144577  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:34.140309 16871 consensus_meta.cc:287] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.015s
I20260709 12:29:34.155498  7734 raft_consensus.cc:2393] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:34.139923 17049 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:34.142768 16854 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.012s
I20260709 12:29:34.161880  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.142788 16938 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.165819  7734 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:29:34.149279 16923 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:34.166972  7734 pending_rounds.cc:85] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 13
I20260709 12:29:34.167845  7734 pending_rounds.cc:107] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:29:34.168484  7734 raft_consensus.cc:2889] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:34.165819 17052 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.542s	user 0.010s	sys 0.000s
I20260709 12:29:34.169512  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.174062 16841 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:34.174698 16841 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:34.176004 16841 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.177080  6451 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:34.179147  7728 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:34.198062  7728 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:29:34.181162  7732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1974) took 1087 ms. Trace:
I20260709 12:29:34.199730  7732 rpcz_store.cc:276] 0709 12:29:33.093841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:33.094532 (+   691us) service_pool.cc:224] Handling call
0709 12:29:34.067917 (+973385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:34.069087 (+  1170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:34.180938 (+111851us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b5400248bd0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f218ad 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.181092 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2227280256}
I20260709 12:29:34.200589 16975 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:34.200932  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.201059 17016 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.182301  7735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46548 (request call id 4138) took 1838 ms (client timeout 1748 ms). Trace:
W20260709 12:29:34.205044 17071 log.cc:927] Time spent T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 12:29:34.205283 17019 consensus_queue.cc:1048] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:34.206743 16938 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.184020  7734 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:29:34.208670  7734 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:34.209184  7734 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:34.207548 16938 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.210726  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 109284us
I20260709 12:29:34.211946 16938 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.214488 17003 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:34.168642 17055 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:34.167480 16854 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.187471  7731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53564 (request call id 1977) took 1092 ms. Trace:
I20260709 12:29:34.217486  7811 raft_consensus.cc:2393] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 26: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:34.218101  7730 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:29:34.218557 16854 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.218195  7810 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 22. Current term is 23. Ops: []
I20260709 12:29:34.217295  7731 rpcz_store.cc:276] 0709 12:29:33.094746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:33.094905 (+   159us) service_pool.cc:224] Handling call
0709 12:29:34.187262 (+1092357us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003b4cd4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.187396 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558138368}
I20260709 12:29:34.219417 16977 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.160472 16871 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.189358  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.190764  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.199062 16916 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:34.204659  7735 rpcz_store.cc:269] 0709 12:29:32.343824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:32.343947 (+   123us) service_pool.cc:224] Handling call
0709 12:29:34.182046 (+1838099us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54000726d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.182199 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2943289088}
I20260709 12:29:34.221834 16916 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.186640 16892 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.223845 16871 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:34.188807 17039 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.010s
I20260709 12:29:34.224354 16916 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.221776  7732 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 12:29:34.213958 16841 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:34.226954  7732 pending_rounds.cc:85] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 15
I20260709 12:29:34.212075  7778 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:34.227028  7808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40506 (request call id 1503) took 1097 ms. Trace:
I20260709 12:29:34.231580  7808 rpcz_store.cc:276] 0709 12:29:33.129578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:33.130972 (+  1394us) service_pool.cc:224] Handling call
0709 12:29:33.131661 (+   689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:33.837050 (+705389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:34.226725 (+389675us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b5400332554. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f22e01 00007fc001f2183e 00007fc006ee6419 00007fc00045dbd4 00007fc00045db62 00007fc00045da91 00007fc00045da0c 00007fc00045ccc2 00007fc00025d48c 00007fc00025c8c6
0709 12:29:34.226950 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1615133440}
I20260709 12:29:34.230309  7730 pending_rounds.cc:85] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 11
I20260709 12:29:34.233822  7730 pending_rounds.cc:107] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:29:34.234246  7730 raft_consensus.cc:2889] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:34.232817  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.234067 16871 raft_consensus.cc:2764] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:29:34.238631 17076 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:34.210153  7734 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.242298  7734 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:34.245070  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.245993  7735 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:34.246459  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.247939  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.247059  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.249782  7735 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:34.250762  7732 pending_rounds.cc:107] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 12:29:34.251174  7732 raft_consensus.cc:2889] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:34.251047  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.252008  7735 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:34.254421  7729 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:34.256238  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.256966  7731 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:29:34.255625  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.262960  7729 raft_consensus.cc:3060] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 29
W20260709 12:29:34.268815  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 90364.4us
I20260709 12:29:34.271540  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.272686  8413 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 28: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:34.241417 17016 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.277931  7734 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:29:34.297071  7734 pending_rounds.cc:85] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 12
I20260709 12:29:34.286530  8411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40588 (request call id 1806) took 1207 ms. Trace:
I20260709 12:29:34.258124 16871 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.298565  7734 pending_rounds.cc:107] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:29:34.298806  8411 rpcz_store.cc:276] 0709 12:29:33.078820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:33.127304 (+ 48484us) service_pool.cc:224] Handling call
0709 12:29:34.286304 (+1159000us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400691254. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.286459 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919567744}
I20260709 12:29:34.302181  7734 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:34.310252  7778 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.274677 17049 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:34.308029 16871 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.320065 17078 log.cc:927] Time spent T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:29:34.305063  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 135628us
I20260709 12:29:34.328164  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.332183  7729 raft_consensus.cc:2468] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 29.
I20260709 12:29:34.332206  7731 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:34.338027  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.341924  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.352128  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.329265 17012 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:34.360208  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.341022 17049 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:34.347834 16937 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:34.344720 16871 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.345455 17055 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:34.377140  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.377960  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.386659  7733 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:34.374992 17008 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:34.397738  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.364449 16938 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.360419 16923 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:34.400597  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.364212 17005 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.410132  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.379540 16802 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:34.400326 17082 log.cc:927] Time spent T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:29:34.435271 16984 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.436064 16984 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.412395 16864 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:34.438396 16864 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.428956 16802 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.441848 17069 log.cc:927] Time spent T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.419s	user 0.000s	sys 0.004s
I20260709 12:29:34.417048 16938 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.444308 17073 log.cc:927] Time spent T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.388s	user 0.000s	sys 0.002s
I20260709 12:29:34.439059 16864 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:34.461992  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.469159 17055 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:34.469998 17055 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.378666 17012 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.474206  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:34.421172 17017 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:34.478214  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:34.471622 17047 log.cc:927] Time spent T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:29:34.479108 16923 raft_consensus.cc:3060] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.457211 16907 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:34.439015 16984 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.482057 17017 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:34.496270  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 86275.8us
I20260709 12:29:34.481982 17008 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.425127 17005 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.485309 16859 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:34.505192  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:34.506807  7811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40506 (request call id 1514) took 1129 ms. Trace:
I20260709 12:29:34.492033 16907 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.509637  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.508172  7811 rpcz_store.cc:276] 0709 12:29:33.377089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:33.494924 (+117835us) service_pool.cc:224] Handling call
0709 12:29:34.506539 (+1011615us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54001324d4. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc001e8b571 00007fc001f262fa 00007fc006ee7b17 00007fc0004626d4 00007fc000462662 00007fc000462591 00007fc00046250c 00007fc0004617c2 00007fc00025d48c 00007fc00025c8c6 00007fc00025c26f
0709 12:29:34.506740 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1650636032}
W20260709 12:29:34.523825  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 123367us
I20260709 12:29:34.500552 17017 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.528002  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.532613 17005 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.533396 17005 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:34.537093 17012 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.529579 16859 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:29:34.541344 17017 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.543639  7778 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.544986 16918 raft_consensus.cc:2804] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:34.545589 16918 raft_consensus.cc:697] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:34.546631 16918 consensus_queue.cc:237] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.549098 16864 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:34.557248  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.557480 16864 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.549804  7778 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.513381 16907 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:34.519372 16802 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.558531  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.524549 17049 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:34.506583 17008 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.568038  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.568686 16975 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:34.569962  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.573664 16802 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:34.573712 16937 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:34.580691 16937 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.581328 16937 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.583812 17055 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.574244 16802 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.585763 16802 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:34.585848  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.586753  7730 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:34.592496  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:34.593636  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.594475  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.593283  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.605535  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.603103 16892 consensus_queue.cc:1048] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:29:34.606976  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.607697  7731 raft_consensus.cc:3060] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.610111 17008 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:34.611006 17091 log.cc:927] Time spent T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:29:34.571658 16977 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:34.611697 16728 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:29:34.605844  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.613791 16854 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.614494 16854 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.615724 16930 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.617329 16854 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.568403 17014 raft_consensus.cc:2804] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:34.625944  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:34.627158  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.635041  7734 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:34.641147  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.642949  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.645694  7728 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:34.627601  7731 raft_consensus.cc:2468] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:34.627985  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.628269  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:34.629108  8379 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:34.629737  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.648582  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
W20260709 12:29:34.607023 17088 log.cc:927] Time spent T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:29:34.631680 16891 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:29:34.631899  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 105080us
I20260709 12:29:34.632408  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:34.648826  8414 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 29: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:34.649394  8408 raft_consensus.cc:3060] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:34.650820  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:34.629907  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.653497  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.657064  8405 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 28.
W20260709 12:29:34.656216  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:34.658105  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.658620  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:34.650791  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.660104  8412 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 29: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:34.660537  7736 raft_consensus.cc:2393] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 29: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:34.664158  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.664728  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.678584  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.680928  8405 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:34.681526  7810 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 20.
I20260709 12:29:34.638762 17008 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:34.678884  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.679984  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.652263 17003 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:34.666075  8379 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:34.683393  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55486us
I20260709 12:29:34.694366  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.686929  7778 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:34.690892  7806 raft_consensus.cc:2393] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 24: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:34.690541  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.689242  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:34.666286 16728 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:34.651881 16916 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:34.652220 17049 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:34.688748  7734 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:34.654664 16937 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.666775 17001 raft_consensus.cc:2804] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:29:34.761039  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.765829  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:34.678961 16977 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:34.810776  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 150485us
I20260709 12:29:34.811378  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:34.724310  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:34.776829  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.684556 17076 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:34.678679 16859 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:34.759037  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 93073.7us
I20260709 12:29:34.766018  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.806623  7810 raft_consensus.cc:3060] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:34.832921  8379 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:34.659875 17014 raft_consensus.cc:697] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 29 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:34.814827  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:34.759454  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:34.835307  8411 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:34.812322  7730 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 19.
I20260709 12:29:34.691871 16975 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:34.689093 17055 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:34.660913 16891 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.850558 16891 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:34.679389 16812 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:34.851425 16812 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.851922 16812 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.839877  8405 raft_consensus.cc:2393] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 24: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:34.837572  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.827577  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.855650  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:34.856978  7807 raft_consensus.cc:2368] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24.
I20260709 12:29:34.805568  8412 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 20.
I20260709 12:29:34.697589 17003 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.842195  8406 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
W20260709 12:29:34.841857  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 155051us
W20260709 12:29:34.828644  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 134376us
I20260709 12:29:34.775206 16953 raft_consensus.cc:2804] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:34.696890 16854 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:34.802762 16728 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.835868 17014 consensus_queue.cc:237] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.743186 17061 log.cc:927] Time spent T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:29:34.771724 16879 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:34.862551  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.805069 17005 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:34.821025 17075 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.830523 16930 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:34.867293 17003 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:34.853303 17016 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.837744 16864 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:34.821663 16892 consensus_queue.cc:1059] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.053s
I20260709 12:29:34.853528 17001 raft_consensus.cc:697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:34.928552 17001 consensus_queue.cc:237] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.856395 16860 raft_consensus.cc:2804] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:34.869386 16975 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:34.869993 16854 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.936637 16854 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.908339 17039 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.849370 16916 consensus_queue.cc:1059] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:29:34.938979 17039 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.939944 16916 raft_consensus.cc:3055] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:34.853856 17004 consensus_queue.cc:1048] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 12:29:34.883417 16953 raft_consensus.cc:697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:34.902956 16879 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.916220 16930 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.943336 16953 consensus_queue.cc:237] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.940503 16916 raft_consensus.cc:740] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:34.944618 16879 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:34.944859 16860 raft_consensus.cc:697] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:34.945385 16916 consensus_queue.cc:260] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.946056 16860 consensus_queue.cc:237] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.842442 16859 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:34.928560 17094 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:29:34.949222 16859 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.949074 16907 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.947005 16916 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:34.951740 16864 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.956641 16879 raft_consensus.cc:3055] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:34.957266 16879 raft_consensus.cc:740] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:34.958264 16879 consensus_queue.cc:260] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:34.959360 17097 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
I20260709 12:29:34.959895 16879 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:34.908011 17075 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.939157 17016 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.904124 17005 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:34.885421 16728 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:34.977049 16864 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.978470 17017 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:34.983824 16938 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.001711 16802 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.004639 17005 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.009652 17016 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.016333 17075 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.018713 16907 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:35.019984 16923 consensus_meta.cc:287] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.008s
I20260709 12:29:35.020692 16923 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:35.021296 16930 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:34.942173 17039 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.023607 16812 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.052920 16891 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:34.866129  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:35.123697  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 435677us
W20260709 12:29:35.258035  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:35.230824  8379 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:34.897986  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.261560  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.262714  7803 raft_consensus.cc:2393] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 28: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:35.176909  7805 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:29:34.891314  7807 raft_consensus.cc:1240] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 19. Current term is 20. Ops: []
I20260709 12:29:34.870888  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:35.270967 16545 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:35.271745 16545 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.271828  7809 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:35.272416 16545 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:35.182931  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:34.901050  7811 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:29:35.276081  7808 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 27.
I20260709 12:29:35.277477  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.277884  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.188071  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.280278  7812 raft_consensus.cc:2410] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 12:29:35.196193  6451 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:35.209115 17115 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.052909 17003 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.219980  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.234453 16802 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:35.237269  8408 consensus_meta.cc:287] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.019s
I20260709 12:29:35.239418 16812 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.246424 16844 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.306337 17003 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.307528 17039 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:35.307353 16844 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.308565 17039 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.312454 17101 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.313638 17101 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.247211 16854 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.270982  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.282835  7812 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:29:35.041049 17008 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:35.217748  7810 raft_consensus.cc:2468] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:35.328459 17049 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:35.297919  7808 raft_consensus.cc:2393] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 29: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:35.284881  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:35.328528  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:35.299597 16977 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:35.288707 17115 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:35.317768  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 465295us
I20260709 12:29:35.324136  7807 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:35.312359 16844 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:35.336834  7700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:35.336912 17001 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:35.322767  8408 raft_consensus.cc:2468] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
W20260709 12:29:35.335630  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 389051us
W20260709 12:29:35.332626  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48105.8us
I20260709 12:29:35.341781 16994 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:35.342645  7806 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:29:35.343588 16854 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.348954 16891 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.352174  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:35.352242  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.355732 16977 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:35.356575 17115 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.359012 17107 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.360249 17107 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.364285 17107 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.378443 16854 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.387558  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:35.373847 17115 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:35.379832 16854 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.403381 17101 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.404845  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:35.410643  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:35.393381 17107 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:35.413132  8380 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.415387 16994 raft_consensus.cc:2804] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:35.416311 16994 raft_consensus.cc:697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:35.420492 16994 consensus_queue.cc:237] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:35.424103  7702 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.414019 17107 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.433007 16854 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.420676  7775 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:35.435588 16844 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:35.436584 16844 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.437284 16844 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:35.438045  7775 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:35.408972 17039 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.443558  7702 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:35.454967  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.456218 16854 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:35.449193 17115 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.473513  8408 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 29.
I20260709 12:29:35.462570 17049 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:35.470948 16854 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.478317 17039 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:35.490489 17003 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:35.488483  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:35.486315 16854 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:35.516987  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 50639.7us
I20260709 12:29:35.509333 17107 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.516093 16860 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:35.537721 17014 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.538507 17014 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.525565 16860 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.521180 17049 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:35.544376  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 150605us
I20260709 12:29:35.541491 16545 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.560961  7809 raft_consensus.cc:2410] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
W20260709 12:29:35.545785  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70739.5us
I20260709 12:29:35.586248  7803 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:35.547128 17115 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.549408 17039 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.592453 17039 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.593824 17049 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:35.595772 16939 consensus_queue.cc:1059] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:35.598229  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.599408  7734 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
I20260709 12:29:35.600855 17101 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:35.603680  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.604285 17049 raft_consensus.cc:3055] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:35.604808  7732 raft_consensus.cc:2393] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 25: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:35.606251  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.607360  7730 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:35.610703  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 70129us
I20260709 12:29:35.606436 17039 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.543947 16860 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:35.612421  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.612421  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:35.613481  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.612694 16939 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:29:35.619675  7728 raft_consensus.cc:3060] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:35.621352 17101 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:35.623456  7729 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:35.624250 17101 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:35.628721  8408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:35.628721 17101 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:35.629987  8408 raft_consensus.cc:3060] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:35.632731 17101 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:29:35.639384 16728 consensus_meta.cc:287] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.023s
I20260709 12:29:35.640105 16728 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.641530  8406 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:29:35.645457  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:35.646708  8406 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:35.651705  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.652920  7734 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 22.
I20260709 12:29:35.560894 17014 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.642876 16728 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:35.665730  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 54060.7us
I20260709 12:29:35.604904 17049 raft_consensus.cc:740] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:35.676218  7702 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.683416 17101 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.684234 17101 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.691159  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.691251  7734 raft_consensus.cc:1275] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:29:35.692250  7732 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:35.701787  7736 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:29:35.710631  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:35.710978  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 100630us
I20260709 12:29:35.711740  7731 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 31.
W20260709 12:29:35.712196  7702 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.713832  7728 raft_consensus.cc:2468] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:29:35.714025  7702 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.718076 16854 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:35.719102  7702 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.720110  7626 leader_election.cc:400] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24.
I20260709 12:29:35.719870  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:35.726673  7702 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.727502 17039 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:35.728823 17014 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.729236  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.729509 17014 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.723673 17003 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:35.731143 17115 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:35.731637 17003 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.732128 16977 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:35.731967 17115 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.732846 16977 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.736168 16977 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.736492  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.737037 16977 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.708060  7729 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:35.737785  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.729543  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.735935 17115 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.740401 17106 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:35.740615  8380 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.741026 17106 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.741443 17106 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:35.730075  7735 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:35.745771 17016 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:29:35.746451 17016 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.746949  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.747759  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:35.749133 17117 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.301s	user 0.000s	sys 0.006s
I20260709 12:29:35.749230 17107 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:35.749925  7737 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:29:35.750023 17107 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.750584 17107 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:35.751024  7733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53564 (request call id 1993) took 1117 ms. Trace:
I20260709 12:29:35.746910 17016 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:35.751588  7733 rpcz_store.cc:276] 0709 12:29:34.633304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:34.634175 (+   871us) service_pool.cc:224] Handling call
0709 12:29:35.361254 (+727079us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b58001b2498. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffedfdb71 00007fbffedfda22 00007fc007d168d9 00007fc006e70ee0 00007fc006f3a0ab 00007fc006f3a131 00007fc006efbfc0 00007fc006ee6929 00007fc006ee63d2 00007fc00045dbd4
0709 12:29:35.361672 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:35.362357 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:35.362388 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:35.363064 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:29:35.390777 (+ 27713us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:29:35.390829 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:35.390909 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:35.748044 (+357135us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:35.750645 (+  2601us) raft_consensus.cc:1685] finished
0709 12:29:35.750672 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:35.750809 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2514,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":304,"spinlock_wait_cycles":15369472,"thread_start_us":23424,"threads_started":1,"wal-append.queue_time_us":60234}
I20260709 12:29:35.753592 17003 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.748040  7734 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 21.
I20260709 12:29:35.754456  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.750401 16844 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.755549  7737 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:35.755707  7734 raft_consensus.cc:1275] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:35.747700  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.743046 17014 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.756947  7728 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:35.757329 16844 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.760548 16545 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.737957 16977 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.769970 16802 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:35.771458  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:35.772819 17004 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:35.769312 17003 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.002s
W20260709 12:29:35.773161  7702 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.687027 17101 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.774010  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:35.771044  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:35.775410 17039 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:35.775835  7729 raft_consensus.cc:2393] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:35.777280  7702 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.754458 17001 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:35.778877 16977 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.780092  7626 leader_election.cc:403] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:35.780246  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.781267  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.782054 16802 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.782783  7778 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.785805  8412 raft_consensus.cc:1275] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:29:35.786880  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.698834  8408 raft_consensus.cc:2468] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:35.795135  7730 raft_consensus.cc:2468] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:35.793094  7702 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:35.821089  8406 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 29.
I20260709 12:29:35.825464 17014 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.825897  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.826260 17014 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.827358 17107 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.830153 17107 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.831176  7735 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 28.
I20260709 12:29:35.834595  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:35.834120  7808 raft_consensus.cc:1275] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:35.835160  8405 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
W20260709 12:29:35.835410  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 115744us
I20260709 12:29:35.835728 16977 raft_consensus.cc:3060] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:29:35.847820  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:35.835986 17049 consensus_queue.cc:260] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.850395 17049 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:35.851477 16938 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.840505 17003 raft_consensus.cc:2804] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:29:35.852888  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:35.841322  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:35.831516  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 48862.4us
W20260709 12:29:35.846416  8379 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.842208  7807 raft_consensus.cc:1275] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:29:35.853560  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:35.860271  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 79141.4us
I20260709 12:29:35.862954  7778 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:35.851899  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:35.863582  8379 leader_election.cc:336] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:35.839742  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:35.839164 17014 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:35.877908  7804 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 27: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:35.875804  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:35.878944  7736 raft_consensus.cc:3060] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:29:35.882992  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:35.837692 16854 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:35.890726  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:35.897863  8405 raft_consensus.cc:2393] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 27: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:35.852447 16938 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:35.911204 16916 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:35.911751 16916 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting leader election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:35.912185 16916 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:29:35.911077  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:35.915211 16953 consensus_queue.cc:1048] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:35.916765  7624 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
W20260709 12:29:35.917759  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63692.6us
I20260709 12:29:35.918617  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:35.886394 17107 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.894018 16891 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:35.898897 17115 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:35.882403 16802 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.954978  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:35.920166 17014 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.927547 16728 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:35.960520 16891 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:35.964025 16891 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:35.966833 16891 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.960327 17127 consensus_queue.cc:1048] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:29:35.936347 17101 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:29:35.971437  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 119725us
W20260709 12:29:35.974220  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 66614.2us
I20260709 12:29:35.935822 16854 raft_consensus.cc:2804] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:35.975672  8380 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:35.998425  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 221198us
I20260709 12:29:35.960701 16844 consensus_queue.cc:1048] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.016s
I20260709 12:29:35.997105 17003 raft_consensus.cc:697] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:35.972821 17101 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.971319 17004 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:35.977834 16728 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
W20260709 12:29:36.027161  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 72308.6us
I20260709 12:29:35.968397 16938 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:35.974498 17107 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.036002  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.016568 16994 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:29:36.042441  8379 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 160 similar messages]
I20260709 12:29:36.024219 17101 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:36.051939  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 197458us
I20260709 12:29:36.005788 17014 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.053871  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.054602  7732 raft_consensus.cc:2468] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:29:36.062294 16728 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.065847 17131 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:29:36.066349 16860 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.073482  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 68714.6us
I20260709 12:29:36.075909 17129 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.076933 17129 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.077316 16916 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.079344  7702 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.074815  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.098186  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:36.102020  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 94690.6us
I20260709 12:29:36.115283  7812 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 in current term 21: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
W20260709 12:29:36.124332  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49635.3us
I20260709 12:29:36.109323 16854 raft_consensus.cc:697] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:36.131712  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.120253 16891 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:36.132012 17003 consensus_queue.cc:237] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.155584 16977 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24.
I20260709 12:29:36.132493 16939 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:36.133044 17099 log.cc:927] Time spent T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:29:36.131423 17115 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.159307 17115 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.159602  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.116864 16802 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:36.148687  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.161525  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.152295  7736 raft_consensus.cc:2468] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 21.
I20260709 12:29:36.099424 17106 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.174562  7811 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:29:36.172875  8380 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.132493 17129 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.164425 16916 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.180033  7732 raft_consensus.cc:1275] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:29:36.186990  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 55318.4us
I20260709 12:29:36.187999 16994 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:36.188282 17105 raft_consensus.cc:2804] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:36.193712 17105 raft_consensus.cc:697] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:36.177349 16939 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:36.194928 17105 consensus_queue.cc:237] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.205099 17106 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:36.206187  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:36.207086  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:36.207774 16854 consensus_queue.cc:237] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.209543 17014 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:36.211535  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 160502us
W20260709 12:29:36.212702  7777 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Timed out: RequestConsensusVote RPC to 127.6.76.193:43519 timed out after 1.516s (SENT)
I20260709 12:29:36.212611 17107 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.216200 16977 raft_consensus.cc:3060] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:36.211755  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.227716  7733 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:36.226266  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 173308us
I20260709 12:29:36.186986 17115 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.233785  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.236677 17014 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:36.226841 16844 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:29:36.212694 17101 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.271603  8413 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 29.
W20260709 12:29:36.252874  7702 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:36.263044  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 101690us
I20260709 12:29:36.248358  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:36.276911  8412 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 29.
I20260709 12:29:36.237571  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.280771  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:36.279822  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:36.282925  8412 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:36.284204  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:36.299378  8406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:36.244652 17132 log.cc:927] Time spent T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:29:36.282737  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 45303.7us
W20260709 12:29:36.259824 17016 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.003s
I20260709 12:29:36.238127 17014 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:36.288098  6451 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:36.258447 17101 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.236083 17107 consensus_queue.cc:1048] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.064s
I20260709 12:29:36.258330 16860 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.312126  7778 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:36.251677 16939 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:36.310500 16916 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:29:36.314128 17014 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:36.306815 17016 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.338333  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 85692.8us
W20260709 12:29:36.359316  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47273.9us
I20260709 12:29:36.313519 16802 consensus_queue.cc:1048] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:29:36.337402 17146 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:36.374426  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:36.374945 17142 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:36.371464  7778 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:36.384794  7702 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.330258 16728 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:29:36.357826 16860 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:36.390599  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:36.397109  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:36.365736 17003 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:36.338754 16939 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.392930  7778 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.399292  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.395635 16918 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.402309  8412 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:36.377665 17146 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:36.402792  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.384544 17142 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:36.402968 17003 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:36.413621 17003 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:36.409090  7624 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 
I20260709 12:29:36.409663  8414 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 31. Current term is 32. Ops: [31.19-31.19]
I20260709 12:29:36.416446 17101 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:36.417089 17101 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:36.418167 17101 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.422099 17115 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.422860 17115 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.426091 17115 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:36.409821 16860 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.427439  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:36.429639 16860 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:36.396245 17049 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.431519 17049 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.434247 17049 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:36.426129 16939 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.439311 17145 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:36.440361 17145 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.442624  7555 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:36.451766 16918 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.410123 17146 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:36.455535 17003 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.460260  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 93641.5us
W20260709 12:29:36.468210  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 71591.7us
I20260709 12:29:36.460081 16916 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:36.484890  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:36.459118 17003 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.486048 16939 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:36.486250 17003 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:36.487269 17003 raft_consensus.cc:697] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:36.487885 16728 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:36.488560 17003 consensus_queue.cc:237] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.486532 16939 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:36.489677 17049 consensus_queue.cc:1059] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:36.490082 16939 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:36.474879  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:36.492383  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49786.1us
I20260709 12:29:36.493551 16916 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:36.494231 16916 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:36.495517 17049 raft_consensus.cc:3055] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:36.495689 17154 consensus_queue.cc:1059] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:36.459661 16545 consensus_queue.cc:1048] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:29:36.468371 17142 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:36.454967 16918 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:36.509598  7733 raft_consensus.cc:2468] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 23.
I20260709 12:29:36.443080 17145 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:36.482360  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89535.1us
I20260709 12:29:36.496465 17049 raft_consensus.cc:740] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:36.512753  8380 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.507416 16854 raft_consensus.cc:2804] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:29:36.519419  8379 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:36.527679  7702 leader_election.cc:336] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.530108  7702 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:36.526989  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.544870  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.547868  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:36.548929  7812 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:29:36.512491 17115 raft_consensus.cc:3060] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:36.546085  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.551802  7778 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:36.542613 16953 raft_consensus.cc:2804] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:36.572719  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.495878 16916 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.585721  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.550518 17129 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:36.599138  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.568635 16854 raft_consensus.cc:697] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:36.603195  7805 raft_consensus.cc:2393] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:36.608881  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.611101  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.613008  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.614830  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.616001  7728 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:36.608703  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.632769  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.628696  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:36.587908 17154 raft_consensus.cc:2804] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:36.632409  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:36.631843  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:36.624065  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:36.601087 16953 raft_consensus.cc:697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:36.654749  7809 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:29:36.673009  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:36.632319 16977 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:36.677390  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 159222us
I20260709 12:29:36.643389 17101 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:36.689692  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 84326us
I20260709 12:29:36.690661  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:36.664461 17016 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 32 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:36.698061  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:36.703146  8407 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:36.704293  7702 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.705020  7702 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.708150  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.715134  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.679400 17154 raft_consensus.cc:697] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:36.723816  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:36.725476  7736 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:29:36.725664  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:36.728219  7734 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:29:36.644204 17049 consensus_queue.cc:260] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.743381  8411 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:29:36.731065 16977 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:36.748401 16977 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:36.748935  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.750236  7737 raft_consensus.cc:2393] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d72176ccaed049958ac68f706d4f052d in current term 21: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:36.751638 16939 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:36.714586  7552 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 287) took 81146 ms (client timeout 15000 ms). Trace:
W20260709 12:29:36.726497 17153 log.cc:927] Time spent T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:29:36.755061  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:36.751377 16854 consensus_queue.cc:237] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.759879  7732 raft_consensus.cc:3060] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:36.768308 16977 consensus_queue.cc:1059] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:36.769584 16977 raft_consensus.cc:3055] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:36.770095 16977 raft_consensus.cc:740] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:36.751654 17154 consensus_queue.cc:237] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:36.768656  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71193.7us
I20260709 12:29:36.775286 17107 raft_consensus.cc:3055] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:29:36.775889 17107 raft_consensus.cc:740] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:36.726454 16953 consensus_queue.cc:237] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.777127 17107 consensus_queue.cc:260] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.777583  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:36.779431  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:36.780114 17049 raft_consensus.cc:3060] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:36.730091 17129 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:36.733023 17145 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:36.775363  7702 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.797015  7728 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
W20260709 12:29:36.802727 17155 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:29:36.727169 17101 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:36.807121  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 256192us
I20260709 12:29:36.795548 16728 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.042s
I20260709 12:29:36.849874  8407 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:36.852401  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77141.1us
W20260709 12:29:36.848282  7777 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.854648  7732 raft_consensus.cc:2468] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 25.
W20260709 12:29:36.855346 17156 log.cc:927] Time spent T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:29:36.796139 16977 consensus_queue.cc:260] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.829708 17145 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:36.865590  7812 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 29.
W20260709 12:29:36.888797  7702 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:36.897316  7702 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:36.901710 16939 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:36.902616 16939 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.904631 17101 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:36.905411 17101 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.906032 17101 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.907294 16939 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:36.752645  7552 rpcz_store.cc:269] 0709 12:28:15.394198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.400878 (+  6680us) service_pool.cc:224] Handling call
0709 12:29:36.540580 (+81139702us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:33.594075 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:33.793645 (+199570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:33.799859 (+  6214us) write_op.cc:435] Acquired schema lock
0709 12:29:33.803548 (+  3689us) tablet.cc:662] Decoding operations
0709 12:29:33.821440 (+ 17892us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:33.821526 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:33.821579 (+    53us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:29:33.822116 (+   537us) tablet.cc:701] Row locks acquired
0709 12:29:33.822129 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:29:33.822336 (+   207us) write_op.cc:270] Start()
0709 12:29:33.822517 (+   181us) write_op.cc:276] Timestamp: P: 1783600173822268 usec, L: 0
0709 12:29:33.822541 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:29:33.882341 (+ 59800us) log.cc:844] Serialized 1694 byte log entry
0709 12:29:34.570544 (+688203us) op_driver.cc:464] REPLICATION: finished
0709 12:29:34.694528 (+123984us) write_op.cc:301] APPLY: starting
0709 12:29:34.694643 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:34.932315 (+237672us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:34.932504 (+   189us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:34.987072 (+ 54568us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:34.987363 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:29:34.987417 (+    54us) write_op.cc:312] APPLY: finished
0709 12:29:34.987708 (+   291us) log.cc:844] Serialized 49 byte log entry
0709 12:29:34.987913 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:34.988040 (+   127us) write_op.cc:454] Released schema lock
0709 12:29:34.988091 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:34.988597 (+   506us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:34.988631 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":663218,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":122320,"apply.run_cpu_time_us":10790,"apply.run_wall_time_us":294046,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20847,"num_ops":4,"prepare.queue_time_us":269229,"prepare.run_cpu_time_us":64849,"prepare.run_wall_time_us":582930,"raft.queue_time_us":199395,"raft.run_cpu_time_us":17177,"raft.run_wall_time_us":164181,"replication_time_us":735482,"thread_start_us":508723,"threads_started":4,"wal-append.queue_time_us":277253,"wal-append.run_cpu_time_us":57782,"wal-append.run_wall_time_us":1889806}]]}
I20260709 12:29:36.914279 17146 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:36.921205  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 92106.8us
W20260709 12:29:36.923255  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:36.854605 17014 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:36.920219 17161 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:36.931020 16977 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:36.941735  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:36.929150 16916 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:29:36.943132  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.951300  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 76652.2us
I20260709 12:29:36.922030 17145 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:36.961747 17107 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:29:36.971810  7777 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:36.972581  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.983862 17145 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:36.990233  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:36.998487 17146 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:36.939850 17161 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:37.000171 17014 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:37.000645  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:37.020421  7702 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.011775 17145 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:37.031912 17049 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.036044 17145 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.033321 17182 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.037473 17182 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.036846 17115 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:37.036218 17049 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.042786 17101 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:37.043540 17101 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.044137 17101 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.043766 17182 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:37.046847 17101 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:37.047204 17049 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:37.052127 17142 consensus_meta.cc:287] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.000s
W20260709 12:29:37.052973  7777 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.056252 16939 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:37.059388  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 157335us
I20260709 12:29:37.080487  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.012629 17161 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:37.083921 16860 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:29:37.071439  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 57877.8us
W20260709 12:29:37.085191  7702 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.085265  7775 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:37.085718  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:37.087440 17129 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:37.087760 17154 raft_consensus.cc:2804] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:37.088371 17154 raft_consensus.cc:697] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:37.089198  7806 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:29:37.089362 17154 consensus_queue.cc:237] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:37.039023 17145 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:37.091213  7812 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:37.092629  7778 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.092692  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:37.095250  7803 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:37.099150  7805 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28. Ops: []
I20260709 12:29:37.101351  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:37.103991  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:37.107501  8413 raft_consensus.cc:1240] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 31. Current term is 32. Ops: [31.19-31.19]
I20260709 12:29:37.108732  7812 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:37.098562 16918 raft_consensus.cc:2749] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:29:37.106180  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:37.109620  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:37.115532  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:37.095410 17142 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:37.111755  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:37.114159  7806 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:29:37.118669  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:37.104585  7778 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:37.126341  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:37.130679 16939 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:29:37.131731  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.131863 17049 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:37.132531 17049 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:37.133695 17049 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.128494 16918 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:37.134994 16918 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.135637  7735 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:29:37.136447 17146 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:37.139334  7808 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 20: Already voted for candidate d72176ccaed049958ac68f706d4f052d in this term.
I20260709 12:29:37.140600  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:37.147061  8407 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:29:37.137616 16918 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:37.145453 17146 raft_consensus.cc:697] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:37.157279 17049 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:37.139098 17101 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:29:37.188887  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:37.203420  7735 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:29:37.205570  7803 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 24.
W20260709 12:29:37.223935  7627 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Timed out: RequestConsensusVote RPC to 127.6.76.194:35283 timed out after 1.578s (SENT)
I20260709 12:29:37.224642  7627 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.226120 17183 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:37.152587 16939 consensus_queue.cc:1059] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:29:37.227350 16939 raft_consensus.cc:3055] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:37.227854 16939 raft_consensus.cc:740] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:37.228904 16939 consensus_queue.cc:260] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.155323 16728 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
W20260709 12:29:37.237775  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:37.238538  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 122356us
I20260709 12:29:37.166844 17142 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:37.255833  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:37.258477  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:37.259016  7737 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:37.260666  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:37.186084 17115 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.147054 16953 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 12:29:37.265441  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:37.270076  7808 raft_consensus.cc:1275] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:29:37.272028  7806 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:29:37.265488  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 178406us
I20260709 12:29:37.276746  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.277786  7736 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:37.280046  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.257999 16939 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:37.285787  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 55009.3us
I20260709 12:29:37.308758  8414 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:29:37.312255  6451 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:37.317780  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:37.320992  7728 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:37.322419 16916 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.004s
I20260709 12:29:37.316412 17146 consensus_queue.cc:237] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:37.332171  7776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.332692  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.283424 17115 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:37.353205  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.355291  7732 raft_consensus.cc:1275] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:37.357295  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.358438  7730 raft_consensus.cc:2368] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28.
I20260709 12:29:37.384020  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.398846  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:37.436780  8414 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:29:37.462064  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57146.5us
W20260709 12:29:37.475490 17107 consensus_meta.cc:287] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.005s
I20260709 12:29:37.407486 17145 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:37.532500  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 129998us
I20260709 12:29:37.533128  7803 raft_consensus.cc:2410] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 32 because replica has last-logged OpId of term: 31 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
W20260709 12:29:37.534992  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:37.533737  7777 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:37.543246  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 243727us
I20260709 12:29:37.543696  7736 raft_consensus.cc:1275] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:29:37.541246  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:37.545713  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 82103.1us
I20260709 12:29:37.548460  8407 raft_consensus.cc:1275] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:29:37.550048  7731 raft_consensus.cc:1240] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:29:37.562407  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.550611 16918 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:37.556363  7728 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 28.
I20260709 12:29:37.557235  7737 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
I20260709 12:29:37.569739  7733 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:29:37.558131  7732 raft_consensus.cc:1240] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:29:37.584926  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.591621 16728 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.592660 16728 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.602586  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.607909  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.624337  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.626659  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.615703  7626 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:37.634876  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:37.655201  7737 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
W20260709 12:29:37.639982  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 258990us
W20260709 12:29:37.641106  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.655746  7732 raft_consensus.cc:2410] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:29:37.635196  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.540339 17145 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.659904  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.661101  7733 raft_consensus.cc:2410] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 12:29:37.659926  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 70991.1us
W20260709 12:29:37.662729  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47236.2us
I20260709 12:29:37.661909  7778 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:37.671854  7627 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.607185 17129 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:37.675680  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 141979us
I20260709 12:29:37.677024  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.675612  7731 raft_consensus.cc:1240] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:29:37.641299 17134 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:37.695358  7734 raft_consensus.cc:1240] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
W20260709 12:29:37.697611  7626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.700244  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:37.703809  7777 leader_election.cc:336] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:37.709928  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 146291us
I20260709 12:29:37.713056  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:37.713617  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 51814us
I20260709 12:29:37.721031  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:37.694942 17183 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:37.723773 16728 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:37.694811 17161 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.699977 17101 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.678020 17136 consensus_queue.cc:1048] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 12:29:37.735857  7777 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.766147  7735 raft_consensus.cc:2393] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:37.770215  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.699594 17129 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.723299 17194 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:37.754384 17154 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:29:37.780985  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 130376us
I20260709 12:29:37.757633 17183 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:37.815418  7732 raft_consensus.cc:1240] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Rejecting Update request from peer be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:29:37.772506 16728 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
W20260709 12:29:37.823410  7625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.826216  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.836339  8405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:37.838426  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 102604us
I20260709 12:29:37.772691 17161 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:37.772408 17101 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.842096 16728 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.850090  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.852725 17175 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:37.853312 17204 log.cc:927] Time spent T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:29:37.853377  7777 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.829859 17129 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:37.864970  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.865424 17183 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.835708 17115 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:37.872267 17145 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:37.898869  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45541.3us
I20260709 12:29:37.875355  7731 raft_consensus.cc:1275] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:37.884044  8414 raft_consensus.cc:2468] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 30.
I20260709 12:29:37.888607  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.905025  7737 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
W20260709 12:29:37.900483  7777 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:37.906374  7777 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.907799  8407 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 29. Current term is 30. Ops: [29.22-29.22]
I20260709 12:29:37.898270  7627 leader_election.cc:400] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28.
I20260709 12:29:37.913993  7627 leader_election.cc:403] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:37.918866  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.920047  7730 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:37.925716  7627 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.892465 17049 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.930968  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:37.931885  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:37.942255  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:37.942589  7627 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.892690 17209 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:29:37.893103 17175 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:37.939126 17145 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:29:37.956459  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:37.960909 17219 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:37.961901 17219 consensus_queue.cc:1059] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:29:37.962883 17219 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:37.972878 17200 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: Append to log took a long time: real 0.182s	user 0.003s	sys 0.000s
I20260709 12:29:37.984395 17205 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:37.999933 17205 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.001470  7728 raft_consensus.cc:1275] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:38.005970  8411 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:38.012061 16728 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.012851 16728 raft_consensus.cc:3055] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:29:38.013268 16728 raft_consensus.cc:740] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:37.986951 17137 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:29:38.016556 17175 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.017757 16728 consensus_queue.cc:260] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.025007 17101 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.028970 17161 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:37.949404 16545 consensus_queue.cc:1048] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:29:37.944172 17049 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:37.992549 16939 consensus_meta.cc:287] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.693s	user 0.016s	sys 0.000s
I20260709 12:29:38.034068 17049 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:38.039321  7702 connection.cc:256] RPC call timeout handler was delayed by -1.5117939169995225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 12:29:38.046257 17129 consensus_queue.cc:1059] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:29:38.049966  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:38.050735 16916 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:38.002913 17205 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.067119  7737 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:29:38.018314 17171 consensus_queue.cc:1048] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:29:38.090643  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 190214us
W20260709 12:29:38.091972  7777 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:38.093550  7777 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:38.094862  7777 leader_election.cc:336] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.099710 17154 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:38.099915  7777 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:38.100929 17154 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:38.101388  7777 leader_election.cc:336] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.103132 17197 consensus_queue.cc:1048] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:38.103513  7777 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:38.108918  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 229493us
W20260709 12:29:38.207422  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 264907us
I20260709 12:29:38.106933 17197 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:38.208758  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.090801 17198 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:38.216935  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 115572us
I20260709 12:29:38.062209 17194 raft_consensus.cc:697] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:38.219108  7777 leader_election.cc:336] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.102095 16802 consensus_queue.cc:1048] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
W20260709 12:29:38.220803  7777 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.205467 17219 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:38.223300  7777 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.225968  7777 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.235767  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.237159  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:38.238667  7777 leader_election.cc:336] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.239319  7777 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:38.241217 17195 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:38.241415 17161 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:38.246026  7627 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.247505 17101 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:38.252666 17222 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:38.254405  7627 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.256099  7627 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.258176 17161 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:38.259634 17161 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:38.263357 16939 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:38.264204 16939 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.265715  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.269954 16728 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:38.272702 17197 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:38.217370 17209 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.275889 17209 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.284858  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.275830 17222 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:38.294443  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.296670  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.300655  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 291605us
I20260709 12:29:38.300879  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.302872  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.304001  7803 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 25.
I20260709 12:29:38.304572  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.308840  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.309247  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:38.310962  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 72352.9us
W20260709 12:29:38.312852  6451 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 7687)
W20260709 12:29:38.314100  7777 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:38.315799  7777 leader_election.cc:336] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.319576  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.284930 17195 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:38.324572  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.327139  7806 raft_consensus.cc:2393] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 23: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:38.316391  7777 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:38.298314 17049 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:38.310343 17209 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.331362  7775 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.324038 17129 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:38.306293 17138 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.050s
W20260709 12:29:38.415544  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 368565us
I20260709 12:29:38.322353 17198 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:38.341032  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.349963 16918 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:38.430301  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.429922  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 125499us
I20260709 12:29:38.295413 17194 consensus_queue.cc:237] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.395159 17205 raft_consensus.cc:3055] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:29:38.370996 17196 raft_consensus.cc:2804] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:38.355947 17175 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:38.321310 17219 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.403954 17230 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:38.372262 17101 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:38.340543 17195 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.345801 17003 consensus_queue.cc:1048] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.086s
I20260709 12:29:38.451411  7626 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:38.442749 17205 raft_consensus.cc:740] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:38.448691 17198 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:38.471742 17227 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:38.463248 17115 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.476325 17101 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.477626 17197 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:38.480086 17145 consensus_meta.cc:287] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:29:38.483321 17223 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:38.488777 17226 log.cc:927] Time spent T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 12:29:38.453354  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 112386us
I20260709 12:29:38.493151 16939 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.421814 17129 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:38.500201  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 170501us
I20260709 12:29:38.452653 17196 raft_consensus.cc:697] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:38.512987  7777 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.515290  8413 raft_consensus.cc:1275] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:29:38.517674  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.519380  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.520131  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.501981 17219 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.528964  7806 raft_consensus.cc:2468] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:29:38.517527  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 162765us
W20260709 12:29:38.531535  7624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:38.532655  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:38.538015  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.530768  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.543684  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.545830  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:38.546875  7812 raft_consensus.cc:2393] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 24: Already voted for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in this term.
I20260709 12:29:38.547251  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.548252  7804 raft_consensus.cc:1275] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:29:38.550830 16916 consensus_queue.cc:1059] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:29:38.552103 16916 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:38.462813 17230 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.558135 17230 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.559448 17234 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:38.560022 17234 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.560386 17234 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:38.514106 17145 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283) with higher term. Message: Invalid argument: T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for earlier term 27. Current term is 28.
I20260709 12:29:38.505084 17115 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.503832 17227 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.569211 17227 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.569983 17227 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:38.570422 17227 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:38.509655 17101 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:38.580098  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 130223us
I20260709 12:29:38.584574  7804 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:38.535717 17161 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.567003 17222 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.559074 17230 raft_consensus.cc:3055] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:29:38.585796  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 44250.2us
W20260709 12:29:38.587141  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49160.1us
W20260709 12:29:38.603694  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.604197  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.605420  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:38.599886  7702 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:38.605913  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.598577  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.569216 17115 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.572078 17003 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.574146 17223 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:38.622340  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.622944  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.581576 17195 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.625949  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.599596 17198 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:38.601203 17145 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.617656 17154 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:38.603610 17129 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:38.628242  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 196455us
I20260709 12:29:38.649904  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.629639  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 178344us
I20260709 12:29:38.653115  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.607074 17230 raft_consensus.cc:740] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 LEADER]: Becoming Follower/Learner. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:38.570441 17205 consensus_queue.cc:260] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.655274  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.659410  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.677354  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.624809 17003 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.644883 16918 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:38.644711 16802 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:38.682117  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:38.682965  7627 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.677636  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 171637us
I20260709 12:29:38.647764 17145 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.627336 17223 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.680338 17239 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.699366  7803 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:38.681146 17161 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.709277  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:38.702191  7627 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.759421  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 57365.6us
I20260709 12:29:38.611136 17227 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:38.722101  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 224633us
I20260709 12:29:38.710933 17115 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:38.683326 17235 raft_consensus.cc:3060] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:38.678716 17196 consensus_queue.cc:237] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.770864 16728 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.771615 17239 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.773056 17243 raft_consensus.cc:2804] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:38.748661 17219 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:38.756227  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:38.775900  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:38.793638  7734 raft_consensus.cc:1275] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
W20260709 12:29:38.778594  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 151078us
I20260709 12:29:38.783732  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:38.784822  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.680964 17230 consensus_queue.cc:260] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.806813  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.798707  7627 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:38.808423 17237 log.cc:927] Time spent T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:29:38.809156 17101 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.809868 17101 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.798547 17227 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:38.808439 16728 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:38.818809 17239 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.820915 17145 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.821928  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.824925  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.815440 16802 consensus_queue.cc:1048] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:29:38.826603  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.827644  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.830062  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:38.798875  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:38.831420  8413 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 32.
I20260709 12:29:38.805001 17219 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.853389 17239 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.854162 17219 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.854101 17239 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.856676 17239 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.856837 17240 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:38.857713 17240 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.788220 17243 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.863440  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:38.854902 17219 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.869592  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.870810  7733 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:38.875039  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.873098  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:38.884330  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:38.868734 17243 raft_consensus.cc:3060] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:38.883119 17230 raft_consensus.cc:3060] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:38.859472 17223 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.883363 17219 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:38.911706 17195 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:29:38.888960  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:38.912498  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:38.915516  7777 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.915978  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:38.916581  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:38.918372 17161 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:38.924362  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.931779  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.938570  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:38.912456 17195 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.910956 17240 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:38.950488  7809 raft_consensus.cc:1240] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 LEADER]: Rejecting Update request from peer fc963afbbf4c4b34b161448ef79bd2ed for earlier term 29. Current term is 30. Ops: []
I20260709 12:29:38.933039 17219 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:38.945487 17195 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:38.970321  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:38.966360 17129 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:38.987499 17248 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:38.988297 17248 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:38.958001 17234 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:38.991760 17220 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 1.025s	user 0.011s	sys 0.000s
I20260709 12:29:38.993047 17234 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:38.997258  7702 consensus_peers.cc:597] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:38.997852  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 383753us
I20260709 12:29:39.008823  7735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46548 (request call id 4250) took 1251 ms. Trace:
I20260709 12:29:39.009480  7735 rpcz_store.cc:276] 0709 12:29:37.757974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:37.794124 (+ 36150us) service_pool.cc:224] Handling call
0709 12:29:37.794609 (+   485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:29:37.817930 (+ 23321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:37.817989 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:29:37.840666 (+ 22677us) log.cc:844] Serialized 52 byte log entry
0709 12:29:37.935374 (+ 94708us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:37.935436 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:37.935544 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:38.804222 (+868678us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fc001abd86e 00007fc001abcbb6 00007fc001abc7d8 00007fc001ac6221 00007fc001ac61d9 00007fc001ac6169
0709 12:29:39.008321 (+204099us) raft_consensus.cc:1685] finished
0709 12:29:39.008350 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:39.008726 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7026,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":11888,"spinlock_wait_cycles":3652352,"thread_start_us":84243,"threads_started":1,"wal-append.queue_time_us":97908}
I20260709 12:29:39.020618 17234 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.021250 17234 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:39.006489 17248 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:39.004094 17129 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:39.030732  7702 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.033881  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:39.053095  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.037125 17234 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:39.068262  7702 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.069310  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.075552  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 243138us
I20260709 12:29:39.063056 17235 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:39.088868  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:39.063683 17248 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:39.093231  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:39.115926  8413 raft_consensus.cc:2393] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 32: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:39.072789 17227 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:39.134040  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65833us
W20260709 12:29:39.131212  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:39.136808  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:39.143318  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.152595  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.153438  7729 raft_consensus.cc:2468] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:39.155505  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.157539  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.160553  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.163347  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.169032  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.169122  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:39.153712 17261 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:29:39.186699  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.190955  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.193701  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.194754  7737 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:39.198212  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.199003  7737 raft_consensus.cc:2468] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:39.201596  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.204779  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.205535  7734 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:39.211074  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.212229  7737 raft_consensus.cc:2468] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 28.
I20260709 12:29:39.204588 17160 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.196823 17243 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:39.227488 17243 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.239744  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:39.240618  7734 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:39.250485 17003 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:39.250810  7734 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:39.252151  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:39.253640 17195 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:39.256091 17195 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 33 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:39.260419  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.261565  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.262490  7808 raft_consensus.cc:2368] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 32. Current term is 33.
I20260709 12:29:39.260305  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.223935 17160 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:39.268983  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 120594us
W20260709 12:29:39.272569  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:39.273725  7732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.275007  7732 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 31.
I20260709 12:29:39.276386  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.277070  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.286386  7728 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:29:39.284574  7627 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.289723  7730 raft_consensus.cc:1275] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:29:39.293823  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:39.294503  7735 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:29:39.291639  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.298983  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.301653  7627 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:39.308674  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:39.295140 17222 raft_consensus.cc:2804] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:39.310847  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:39.311527  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:39.309252 17235 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:39.314353  6451 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:39.315919  7627 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.322301  7734 raft_consensus.cc:1275] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:29:39.316349  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.342856  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.345886  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.299990 17243 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:39.343181  7778 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.350245  7627 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.351358 17232 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.356971 17232 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:39.359437 17129 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:39.350981 17160 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.352556 16802 consensus_queue.cc:1048] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:29:39.366433 17255 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:39.367502 17255 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:39.369038 17239 raft_consensus.cc:3060] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:39.349139 17219 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:39.371093 17244 consensus_queue.cc:1048] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:39.373624  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.373974 17205 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:39.377015 17232 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.314131 16939 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:39.312023 17222 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:39.379966  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:39.381217  7627 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.391537  8411 raft_consensus.cc:3060] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:39.389979  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 46984us
W20260709 12:29:39.393950  7703 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:39.395125  8413 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
W20260709 12:29:39.386989  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70853.7us
W20260709 12:29:39.390079  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 100332us
I20260709 12:29:39.399067 16939 raft_consensus.cc:2804] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:39.399595  7627 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.399734 16939 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.380810 17049 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:39.389202 17235 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:39.419426  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:39.408748  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.410982  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:39.423095  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 505161us
I20260709 12:29:39.389986 17230 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:39.407812 16939 raft_consensus.cc:3060] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:39.426370 17197 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:29:39.396778 17127 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:39.432801  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.396963 17222 raft_consensus.cc:3060] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:39.390723 17239 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:39.436501  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.446111  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 27.
I20260709 12:29:39.438714  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.438751  7627 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.396749  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:39.387439 17161 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:29:39.443262  7735 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 33.
I20260709 12:29:39.421321 17209 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:39.451531  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54879.1us
I20260709 12:29:39.429899 17183 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:39.457090  7627 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.440243 17243 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.417282 17198 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:29:39.461031  7702 leader_election.cc:400] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 32 pre-election: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 32. Current term is 33.
I20260709 12:29:39.445854 17049 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:39.463387  7702 leader_election.cc:403] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 32 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:39.468741 17232 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: Vote denied by peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801) with higher term. Message: Invalid argument: T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed for earlier term 32. Current term is 33.
W20260709 12:29:39.492815  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:39.497367  7702 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.499969  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.508484  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.464182 17219 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:39.455096 17230 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:39.461666 17240 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.509347  7807 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:39.501497  7627 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:39.522119  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.465070 17049 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:39.518340  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.525748 17049 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:39.510677 17232 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:39.495687 17255 raft_consensus.cc:2804] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:39.505721 17195 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:39.542523  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45206.3us
W20260709 12:29:39.543694  7702 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.544281  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.545971 17129 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:29:39.547351  7702 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.548058  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.472043 17183 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:39.521551 17230 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.527427 17235 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:29:39.558341  7702 leader_election.cc:336] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.575742  7702 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.577389  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:39.578173  7702 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.558938 17129 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:39.471312 17243 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:39.579136  7702 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.603601 17235 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:39.604898 17235 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:39.606950 17277 log.cc:927] Time spent T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:29:39.540391 17255 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.600646  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.619041  7702 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.619942  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.621662  7702 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.576759 17230 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:39.622473  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.629510 17227 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:39.525928 17240 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:39.602519 17232 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:39.621126 17129 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:39.624328 17205 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:39.621747 17255 raft_consensus.cc:3060] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:39.645741  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:39.652395 17244 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:39.653038 17275 log.cc:927] Time spent T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:29:39.652992 17244 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.653733 17244 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:39.665012 17183 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:39.667609 17240 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.651719  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.677982 17205 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:39.680300 17234 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:39.687475  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.687952  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:39.688634  7735 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
W20260709 12:29:39.697172  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 274005us
W20260709 12:29:39.699198  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:39.710871  7702 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.711156  8411 raft_consensus.cc:2468] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 33.
I20260709 12:29:39.711545  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.714891  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:39.718554  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:39.719733  7702 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.720387  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:39.722025  7702 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.722638  7702 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:39.674474 17244 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:39.729518  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 305879us
W20260709 12:29:39.730793  7702 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.734359  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:39.763677  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 65458.5us
I20260709 12:29:39.723896 17129 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:39.679052 17049 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:39.728572 17232 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:39.687815 17234 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:39.733490 17248 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:39.779577  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.762398 17235 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:29:39.792476  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.794763  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:39.795523  7809 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:39.800266 17173 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:39.801164 17173 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.801677 17173 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:39.804471  7804 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:39.805141  7807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.805864  7807 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 21.
I20260709 12:29:39.805922  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:39.808516  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.802947  7775 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:39.815294  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.829425  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:39.833076  7809 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 26.
I20260709 12:29:39.825191 17273 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:39.845335 17244 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.846772 17243 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:39.792227 17235 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.848623 17235 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:39.855122  7777 consensus_peers.cc:597] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.872471  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:39.884732  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 111467us
W20260709 12:29:39.889565  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 86766.5us
W20260709 12:29:39.893244  7702 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:39.896044  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.896453  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:39.898025  7808 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:39.928081  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 65888.8us
I20260709 12:29:39.898998 17003 raft_consensus.cc:2804] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:29:39.872449 16939 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:39.866559 17273 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:39.972048  7732 raft_consensus.cc:1275] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 22. (term mismatch)
I20260709 12:29:39.972586  7732 pending_rounds.cc:85] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting all ops after (but not including) 21
I20260709 12:29:39.974102  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.934739 17239 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:39.975457  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:39.983632  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:39.984306  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 48292us
I20260709 12:29:39.984898  7729 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:39.985127  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:39.972844  7732 pending_rounds.cc:107] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: Aborting uncommitted NO_OP operation due to leader change: 29.22
I20260709 12:29:39.986689  7732 raft_consensus.cc:2889] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:39.928171 16802 consensus_queue.cc:1048] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:29:39.991431 17273 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.005221 17234 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.012818  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.019821  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:39.948072 17161 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:29:40.030113  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:40.035548  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:40.021736 17261 log.cc:927] Time spent T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:29:40.044559  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:39.976003 17284 consensus_queue.cc:1048] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 12:29:40.027870 17243 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:40.027988 17003 raft_consensus.cc:697] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:40.032276 17234 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.069901  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:40.094533  7729 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 23.
I20260709 12:29:40.096899  7734 raft_consensus.cc:1240] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Rejecting Update request from peer d72176ccaed049958ac68f706d4f052d for earlier term 22. Current term is 23. Ops: [21.13-22.14]
I20260709 12:29:40.049180 17240 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:40.106549  7555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 286) took 110595 ms (client timeout 15000 ms). Trace:
W20260709 12:29:40.094485 17257 log.cc:927] Time spent T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:29:40.052585 17194 consensus_queue.cc:1048] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.054s
I20260709 12:29:40.055708 17195 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:40.095016 17049 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:40.119992  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 134584us
W20260709 12:29:40.133131 17222 consensus_meta.cc:287] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.001s
I20260709 12:29:40.133755 17222 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.135452 17240 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:40.067411 17183 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:40.168824  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.173538  7627 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:40.118559 17243 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:40.183805  7730 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:29:40.102852 17234 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:40.107265  7555 rpcz_store.cc:269] 0709 12:27:49.510876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.511147 (+   271us) service_pool.cc:224] Handling call
0709 12:29:40.106411 (+110595264us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:36.663762 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:36.738389 (+ 74627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:36.738454 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:29:36.738493 (+    39us) tablet.cc:662] Decoding operations
0709 12:29:36.738991 (+   498us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:36.739102 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:36.739136 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:36.739561 (+   425us) tablet.cc:701] Row locks acquired
0709 12:29:36.739618 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:29:36.739831 (+   213us) write_op.cc:270] Start()
0709 12:29:36.740053 (+   222us) write_op.cc:276] Timestamp: P: 1783600176739743 usec, L: 0
0709 12:29:36.740090 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:29:36.740888 (+   798us) log.cc:844] Serialized 538 byte log entry
0709 12:29:37.243276 (+502388us) op_driver.cc:464] REPLICATION: finished
0709 12:29:37.604711 (+361435us) write_op.cc:301] APPLY: starting
0709 12:29:37.649488 (+ 44777us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:37.842880 (+193392us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:37.843030 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:37.846529 (+  3499us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:37.846808 (+   279us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:29:37.846877 (+    69us) write_op.cc:312] APPLY: finished
0709 12:29:37.847206 (+   329us) log.cc:844] Serialized 25 byte log entry
0709 12:29:37.848210 (+  1004us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:37.848314 (+   104us) write_op.cc:454] Released schema lock
0709 12:29:37.848377 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:37.849076 (+   699us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:37.849131 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":843227,"reported_tablets":49,"spinlock_wait_cycles":9088,"child_traces":[["op",{"apply.queue_time_us":346423,"apply.run_cpu_time_us":8347,"apply.run_wall_time_us":244334,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":131006,"prepare.run_cpu_time_us":23039,"prepare.run_wall_time_us":215145,"raft.queue_time_us":219633,"raft.run_cpu_time_us":48862,"raft.run_wall_time_us":379259,"replication_time_us":433967,"thread_start_us":321792,"threads_started":4,"wal-append.queue_time_us":92158,"wal-append.run_cpu_time_us":24687,"wal-append.run_wall_time_us":2023188}]]}
I20260709 12:29:40.215118 16939 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.184167 17183 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.238451 17183 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.247545  7808 raft_consensus.cc:2468] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:40.250150  7702 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:40.255893 17244 raft_consensus.cc:2804] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:40.256615 17244 raft_consensus.cc:697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:40.257673 17244 consensus_queue.cc:237] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.259899 17312 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.260842 17312 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.264935 17312 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.182116 17195 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.268855 17195 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.269790 17312 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.270578 17222 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.270813 17312 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.273568 17234 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.241652 17173 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.160543 17235 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:40.279512  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 85400.1us
I20260709 12:29:40.283635  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:40.286998  7728 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:40.300841  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.307644  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:40.309546  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:40.310501  7737 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:40.310762  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:40.312736  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:40.313339  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:40.281052 17235 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.319654  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
W20260709 12:29:40.322217  6451 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:40.331785  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:40.332594  7810 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:40.279479 17312 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.279208 17173 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.250600 17300 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:40.335906  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:40.193840 17255 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.279686 17243 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:29:40.315632 17232 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.185750 17003 consensus_queue.cc:237] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.359025  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.357774  7624 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:40.361438  7808 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:40.339823 17235 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.365988 17235 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.364461  7624 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.385802  7737 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:40.386590  7728 raft_consensus.cc:2410] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 992d6df95ef74f7f8b12534534a45501 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:29:40.360715 17222 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.388427  7627 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:40.359586 17300 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:40.392127  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:40.395572 17235 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.377909 17232 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.402446 17314 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:40.403439 17222 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.404664 17298 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.405337 17298 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.413177  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.414352  7810 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 28.
I20260709 12:29:40.384334 17234 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:40.393170 17280 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:40.415997 17280 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.362188 17312 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.426575  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:40.431998  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:40.425565  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.438730  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:40.390172 17205 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.444572  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.446187  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:40.448957  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:40.429369 17230 consensus_meta.cc:287] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Time spent flushing consensus metadata: real 0.777s	user 0.006s	sys 0.008s
I20260709 12:29:40.419202 17280 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.433892 17312 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.461476 17232 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.481024 17300 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.448808 17314 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:40.454002 17303 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:29:40.496433  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.497073  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.488031 16916 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.498852  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.499526 17318 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.500212 17318 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:40.502283 17304 log.cc:927] Time spent T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:29:40.496268  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:40.504830  7810 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:40.472381 17173 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:40.474123 17298 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.470459 17297 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:29:40.524549  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:40.459985 17205 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.461826 17230 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.509920 17317 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:40.554069  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:40.542969  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.515954 17303 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Becoming Leader. State: Replica: be9c043c6d424ba1ab8def55dc9bb3e2, State: Running, Role: LEADER
I20260709 12:29:40.552560 17317 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:40.562424  7810 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:40.564244  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:40.567356  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.511767 16916 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.559758 17317 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:40.579845 17318 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.537130 17173 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:40.592557 17312 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.565881 17235 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:40.577054 17314 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:40.568328 16939 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:40.620649  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:40.600107 17255 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.632781  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.620208 17314 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.639895  7810 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:29:40.636065 17313 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:40.677361 17255 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:40.627888 17303 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.643940 17314 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.713974 17195 consensus_queue.cc:1059] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:40.771034 17244 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.772248 17244 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.775776 17244 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:40.756227 17321 log.cc:927] Time spent T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 12:29:40.787154  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:40.788733 17323 log.cc:927] Time spent T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:29:40.789379  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:40.792333 17314 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.793182 17314 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:40.814394  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:40.814313  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.757896 17127 raft_consensus.cc:3055] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:29:40.815965  7730 raft_consensus.cc:2468] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 28.
I20260709 12:29:40.698504 17173 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:40.918103 17312 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.837298 17314 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.817306 17127 raft_consensus.cc:740] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:40.926779 17313 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:40.927129 17244 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:40.951939 17235 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:40.979465  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.953991 17312 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:40.976876 17235 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.000469 17235 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:40.969261 17313 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.007792  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:40.958964 17314 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.037277 17314 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.040220 17230 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:40.989706 17127 consensus_queue.cc:260] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.046788 17127 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:40.971354 17244 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.068037  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.079160  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.081539  8411 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:29:41.085453  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.088950 17222 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.092558 17312 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.094712 17244 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.098613  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.007082 17317 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.112002  7624 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.114971  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.115832  7808 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
W20260709 12:29:41.120560  8379 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 163 similar messages]
I20260709 12:29:41.121286 17314 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:41.124469  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 595060us
I20260709 12:29:41.127840  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:41.128964 17317 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.119778 17195 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.013s
I20260709 12:29:41.132824 17161 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:29:41.133358 17161 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:41.133771 17161 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:29:41.135773  7730 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:41.136677 17205 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.146485  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.143853 16939 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:41.150952  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.151000 16939 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.154780 16939 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.156178  7811 raft_consensus.cc:2393] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 24: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:41.159296  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.160245  7728 raft_consensus.cc:2468] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 22.
I20260709 12:29:41.161237  7734 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:41.164426 16916 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.181954  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.193254  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.198072  7627 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.207890  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:41.209317  7777 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:41.195892  7811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:41.210255  7811 raft_consensus.cc:3060] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:41.213501  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.216974  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:41.220204  7702 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:41.221349  7702 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.227391  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:41.231552  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:41.233072  7777 leader_election.cc:336] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:41.232723  7702 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.234639  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.232173  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.233943  7777 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:41.243868  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:41.239027  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.248636  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
W20260709 12:29:41.240945  7702 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.257725  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:41.260262  7702 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.260856  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.232661 17222 raft_consensus.cc:2804] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:41.258095 16545 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.265523 17222 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:41.265549  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.267170  7808 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 28.
I20260709 12:29:41.254783  7806 raft_consensus.cc:3060] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:29:41.273643  7702 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.244740 16916 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:41.251452 17230 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:41.288043 17333 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:29:41.291508 17173 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:41.292588 17173 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.266288 17317 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:41.301323  7777 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.302362  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:41.305488 17194 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.309109  7556 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 14 to 15, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 15 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.255400 17326 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:41.256945 17235 raft_consensus.cc:3060] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:41.322788  6451 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:41.325814  7811 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:41.326354  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.266078 17222 raft_consensus.cc:3060] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.319312  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:41.254665  7627 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:41.336222  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:41.345496  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 90965.7us
I20260709 12:29:41.359715  7729 raft_consensus.cc:1275] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:29:41.275800 17205 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:41.366878  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47687.7us
I20260709 12:29:41.370074 17227 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:41.370673 17227 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:41.371093 17227 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.386014  7702 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:41.399056  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 47027.8us
I20260709 12:29:41.365299  7556 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.335129 17173 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.414098  8407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:41.405256  7702 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.410799  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:41.455991  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.353715 17303 raft_consensus.cc:2804] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:41.418692  7806 raft_consensus.cc:2468] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 21.
I20260709 12:29:41.449818  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.383339 17312 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.419250 17230 raft_consensus.cc:2804] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:41.427569  7556 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 12 to 17, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 17 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:41.481729 17230 raft_consensus.cc:697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
W20260709 12:29:41.470989  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 65735.6us
W20260709 12:29:41.483728  7702 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.484956  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.384233 17300 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 12:29:41.480517  7700 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:41.473734  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63108us
I20260709 12:29:41.490198 17317 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:41.491969  7702 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:41.504511  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 56138us
W20260709 12:29:41.504968  7702 consensus_peers.cc:597] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:41.464872 17303 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:41.512161 17303 raft_consensus.cc:3060] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:41.492360 16545 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:41.502570 17317 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:41.476047 17244 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:41.527098  7556 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.475703 17312 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.532014 17230 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.534497 17312 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:41.539323  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 214909us
I20260709 12:29:41.548199  7556 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 18 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 19 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.549041 17313 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.555570  7700 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.553912 17209 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.562876 17209 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.565325 17235 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:41.577454 17255 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.578152 17255 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:41.579852  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 49464.7us
I20260709 12:29:41.565483 17209 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.582648  7556 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.597108  7556 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:41.598613 17350 log.cc:927] Time spent T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:29:41.615083  7731 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:29:41.618115  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:41.620910  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 153202us
I20260709 12:29:41.627254  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.628099  7731 raft_consensus.cc:2468] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9c043c6d424ba1ab8def55dc9bb3e2 in term 20.
I20260709 12:29:41.636871 17227 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.617908 16916 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:41.644039 17255 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:41.647485 17161 consensus_meta.cc:287] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.005s
I20260709 12:29:41.648319 17161 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.651096 17161 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.658599  7556 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:41.662536  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:41.666091 17313 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.666774 17313 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.669183 17313 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.669277 17326 consensus_queue.cc:1048] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:41.672669 17303 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.673273  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.687991  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:41.680408  7556 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 14 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 19 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:41.693215 17303 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.693204 17222 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.695986  7556 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 18 to 19, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 19 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:41.697885  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 251296us
I20260709 12:29:41.652307  7702 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:41.708568  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:41.722308  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 70106.3us
I20260709 12:29:41.726079  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.732968  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:41.733789  7806 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.661461 17227 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 election: Requested vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.724767  7556 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 13 to 17, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 17 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:41.748909  7803 raft_consensus.cc:1275] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 12:29:41.745836  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:41.761633  7806 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 29.
I20260709 12:29:41.761905  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.762730  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:41.762984  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.764523  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.766340  7805 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:41.757731  7627 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:41.776012  7728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:41.781502  7734 raft_consensus.cc:1240] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Rejecting Update request from peer 992d6df95ef74f7f8b12534534a45501 for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:41.786871 17312 consensus_queue.cc:1048] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:41.789274  7702 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:41.790944 17312 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:41.737411 16916 raft_consensus.cc:2804] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:29:41.800088  7805 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:29:41.805475  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:41.747540 17235 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.750250 17183 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:41.791837 17312 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:41.768841 17244 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:41.791733 17161 raft_consensus.cc:2804] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:41.840430 17161 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:41.840909 17161 raft_consensus.cc:3060] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:41.816066 16916 raft_consensus.cc:697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:41.844393 16916 consensus_queue.cc:237] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.749902 17255 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:29:41.821693 17183 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.819996 17235 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.854269 17235 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.828112  7803 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Refusing update from remote peer be9c043c6d424ba1ab8def55dc9bb3e2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:29:41.855120  7803 pending_rounds.cc:85] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Aborting all ops after (but not including) 13
I20260709 12:29:41.855473  7803 pending_rounds.cc:107] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:29:41.855724  7803 raft_consensus.cc:2889] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:41.816928 17297 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.857569  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:41.821928 17209 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:41.857211 17297 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.833775 17222 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:41.874899  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 69521.5us
I20260709 12:29:41.878239  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:41.882674  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.883786  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.872166 17209 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:41.887081 17209 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.839802 17244 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:41.889266 17297 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.889642 17209 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.830528 17312 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.890524 17183 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:41.894034  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.905444  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.906989  7627 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:41.908756  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:41.909525  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:41.916110  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:41.918320  7805 raft_consensus.cc:2393] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 in current term 21: Already voted for candidate fc963afbbf4c4b34b161448ef79bd2ed in this term.
I20260709 12:29:41.919493  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:41.926098  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:41.926290  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:41.928062 17297 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:41.934046 16545 consensus_queue.cc:1059] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:41.936718 16545 consensus_queue.cc:1243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Peer fc963afbbf4c4b34b161448ef79bd2ed log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:29:41.937476 16545 raft_consensus.cc:3055] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:41.937863 16545 raft_consensus.cc:740] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:41.889748 17244 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:41.939141 16545 consensus_queue.cc:260] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:41.949767 16545 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:29:41.951445  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:41.955091  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:41.959700  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:41.918040 17183 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.936928 17337 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:29:41.969348  7702 leader_election.cc:336] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:41.970239  7702 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:41.986908  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:41.970897  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.018996 17255 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:29:41.991642  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:41.980400 17337 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:42.037510  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.048202  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:42.074317  7627 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:42.080914  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63379.6us
I20260709 12:29:42.074664 17222 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.027395 17235 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:29:42.086956  8440 heartbeater.cc:646] Failed to heartbeat to 127.6.76.254:33941 (8 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.76.254:33941 timed out after 15.000s (SENT)
W20260709 12:29:42.120007  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.079499 16939 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:29:42.095378 17222 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:42.087311  8377 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:42.111032 17209 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:42.124495  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.117419 17297 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.162401  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.168081  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.168861 16939 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:42.170182 17255 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:29:42.172001  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.174472 17255 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.188745 17255 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.182080  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08777b07fa9d4993a24c05bb4e2db3c8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.184633  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.191489  7810 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:42.195312 17234 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:42.204977 17183 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.205732 17183 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:42.137125 17209 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:42.211691 17209 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:42.141211 17297 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:42.215864  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 64886.1us
I20260709 12:29:42.222936  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:42.151746 17161 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.235330 17161 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.240854 17222 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.241195 17297 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:42.245314  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:42.248018  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 54847.2us
I20260709 12:29:42.249001 17255 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.252125 17183 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:42.255048  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:42.264500  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.265465  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.267424 16916 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:29:42.277734 17183 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:42.279207 17183 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.292577  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.295012  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.298380  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.298650  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.293757 17255 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.310341 17255 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.314040  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:42.319842  7627 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:42.323580  6451 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:42.325956  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.329111  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.333879  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.337515  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.341696  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.335345 17161 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:42.325343 17297 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:42.334611 17222 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:42.354974 17235 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:42.357501  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:42.358744  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.361593  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.364531  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:42.366518  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:42.370431  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:42.368484 17255 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.411468 17234 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.380802 17161 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:42.368732 17297 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:42.431715  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 90162.7us
I20260709 12:29:42.413214 17183 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.436071  7626 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.438913  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.422142 17234 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.425992 17161 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.446089 17255 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:42.447770 17183 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:42.537606  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 89740.6us
W20260709 12:29:42.549492  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:42.558482  7806 raft_consensus.cc:1275] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:29:42.558918  7805 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:29:42.545014 17255 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.567875  8414 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:29:42.568252 17297 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:42.572227 17255 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.575505 17255 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.576983  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:42.584841 17161 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:42.589717  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:42.591003 17337 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:29:42.595417 17337 consensus_queue.cc:1243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Peer 992d6df95ef74f7f8b12534534a45501 log is divergent from this leader: its last log entry 28.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:29:42.594759 17234 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.600224 17183 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.617328 17183 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.627869  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.630045  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.630338  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.633085  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:42.638511 17365 log.cc:927] Time spent T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: Append to log took a long time: real 0.058s	user 0.001s	sys 0.003s
W20260709 12:29:42.640723  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48579.4us
I20260709 12:29:42.645958  8414 pending_rounds.cc:85] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Aborting all ops after (but not including) 17
I20260709 12:29:42.646358  8414 pending_rounds.cc:107] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:29:42.646687  8414 raft_consensus.cc:2889] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:42.659936  7702 consensus_peers.cc:597] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.666361  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.669289  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.671955  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.673187  7730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.673717  7626 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:42.675733  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.677821  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.678689  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.639895 17183 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.686838  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.688516  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:42.690814  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.693220 17244 consensus_queue.cc:1048] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:29:42.696426 17337 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:42.697678 17313 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:42.685985 17209 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:42.686287 17367 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:42.706425 17161 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.707317 17161 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.704430  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:42.739622 17161 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:42.755918  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 51597us
I20260709 12:29:42.757256 17368 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:42.758502  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:42.760255  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.771890  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:42.783237  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:42.774793 17368 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:42.806176 17368 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:42.806677  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.811854 16939 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:42.813174  7626 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.829375  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.833388 17234 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:42.831288  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:42.838071 17313 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:42.826823  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.858155 17368 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:42.888481  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:42.902715  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:42.905689  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.912050  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.920567  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.909191 17366 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:42.909384 16939 raft_consensus.cc:493] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.935735 17209 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:42.954344  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.960430  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:42.959554 17209 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:42.993261 17209 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:42.957577 16939 raft_consensus.cc:515] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.003499 16939 leader_election.cc:290] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:43.004670  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.005949  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.022197  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:43.027199  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:43.031385  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:43.042543  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.043642  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997d33a05304f31b50341cc8c4bbf9b" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.054227  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.066869  7626 leader_election.cc:304] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:43.069619  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:43.073173  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:43.087476  7702 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.073072 17209 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:43.085873 17234 raft_consensus.cc:2749] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:43.156788  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:43.157651  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.227473 17183 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.228268 17183 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:43.232129  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:43.237109  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:43.237680  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44627.5us
I20260709 12:29:43.264245 17183 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:43.285454  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 118220us
W20260709 12:29:43.296581  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.303118  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.307188  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.308687 17209 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:43.324273  6451 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 7687)
W20260709 12:29:43.357628  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 154645us
I20260709 12:29:43.360656 17183 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.361357 17183 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:43.364332 17183 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:43.365324  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.378394  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.393610  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:43.440217  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 63197.5us
I20260709 12:29:43.449261 17183 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:43.450302  7627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 53542.3us
W20260709 12:29:43.457254  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 63835.6us
W20260709 12:29:43.464011  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.469548 17209 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.470316 17209 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:43.473371 17209 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:43.477803  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.492322  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.506016  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.511699 17209 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:43.533020  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:43.575660 17209 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.576385 17209 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.583000  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.587105 17209 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:43.589409  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:43.598129  7556 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 288) took 67445 ms (client timeout 15000 ms). Trace:
W20260709 12:29:43.598891  7556 rpcz_store.cc:269] 0709 12:28:36.152418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.152781 (+   363us) service_pool.cc:224] Handling call
0709 12:29:43.597985 (+67445204us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:42.134825 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:42.176418 (+ 41593us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:42.176462 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:29:42.176484 (+    22us) tablet.cc:662] Decoding operations
0709 12:29:42.177688 (+  1204us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:42.177763 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:42.177790 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:29:42.188686 (+ 10896us) tablet.cc:701] Row locks acquired
0709 12:29:42.188727 (+    41us) write_op.cc:260] PREPARE: finished
0709 12:29:42.188912 (+   185us) write_op.cc:270] Start()
0709 12:29:42.189051 (+   139us) write_op.cc:276] Timestamp: P: 1783600182188853 usec, L: 0
0709 12:29:42.189076 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:29:42.248151 (+ 59075us) log.cc:844] Serialized 4380 byte log entry
0709 12:29:42.573575 (+325424us) op_driver.cc:464] REPLICATION: finished
0709 12:29:42.746911 (+173336us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fbffecbddb1 00007fbffed5bc85 00007fbffed5c6a2 00007fc001ace1db 00007fc001ac9aaf 00007fbffee180ac 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc005958ef0
0709 12:29:42.940756 (+193845us) write_op.cc:301] APPLY: starting
0709 12:29:42.940868 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:42.983866 (+ 42998us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:42.983892 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:42.991104 (+  7212us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:43.051962 (+ 60858us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:29:43.072450 (+ 20488us) write_op.cc:312] APPLY: finished
0709 12:29:43.152078 (+ 79628us) log.cc:844] Serialized 105 byte log entry
0709 12:29:43.158902 (+  6824us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:43.167387 (+  8485us) write_op.cc:454] Released schema lock
0709 12:29:43.172498 (+  5111us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:43.259539 (+ 87041us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:43.259583 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":68045,"reported_tablets":49,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":366800,"apply.run_cpu_time_us":18490,"apply.run_wall_time_us":267369,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50107,"num_ops":11,"prepare.queue_time_us":169560,"prepare.run_cpu_time_us":19027,"prepare.run_wall_time_us":244715,"raft.queue_time_us":138336,"raft.run_cpu_time_us":98966,"raft.run_wall_time_us":366664,"replication_time_us":370322,"spinlock_wait_cycles":37035264,"thread_start_us":654206,"threads_started":4,"wal-append.queue_time_us":144111}]]}
I20260709 12:29:43.610598  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.634768 17209 raft_consensus.cc:493] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.671607 17209 raft_consensus.cc:515] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.674400 17209 leader_election.cc:290] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:43.676972  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.679481 17183 raft_consensus.cc:493] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.661641 17313 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:43.680326 17183 raft_consensus.cc:515] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:43.693001  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79849a645ae64c3787630b03e7da6307" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.695784 17366 raft_consensus.cc:493] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.696493 17366 raft_consensus.cc:515] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.716388  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.723838  7626 leader_election.cc:304] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:43.698978 17366 leader_election.cc:290] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:43.742424  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bea8a36a5046b5b9fb3b3c3a5386f9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.747104 17209 raft_consensus.cc:2749] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:43.750588 17183 leader_election.cc:290] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:43.756454  7627 leader_election.cc:304] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.763492  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.763830 17183 raft_consensus.cc:2749] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:43.790508  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35934889fce04bc093f07a0f03c7165c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.796387 17366 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.801308 17366 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:43.798767  7626 leader_election.cc:304] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.811631  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.807757 17209 raft_consensus.cc:2749] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:43.816090 17366 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:43.816906  7559 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 14, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 14 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:43.833066  7558 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 285) took 150447 ms (client timeout 15000 ms). Trace:
W20260709 12:29:43.833705  7558 rpcz_store.cc:269] 0709 12:27:13.385936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.386225 (+   289us) service_pool.cc:224] Handling call
0709 12:29:43.832783 (+150446558us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84788,"reported_tablets":49}
I20260709 12:29:43.835618  7559 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 10 to 11, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 11 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:43.857316  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402d5cf4f7d24ac59dee1221fc2c7a3f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:43.860287  7626 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.862805 17366 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:43.921438 17366 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:43.922101 17366 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.926632 17366 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:43.928659  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.935617  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.937724  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:43.940442  7702 consensus_peers.cc:597] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:43.939451 17209 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:43.946331 17209 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:43.946985 17209 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.949549 17209 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:43.953809  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:43.957707 17209 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.961730  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.969743 17209 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:43.979934  7627 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:43.980983 17366 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:43.990231  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:43.996282 17183 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.996980 17183 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:43.997789  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:44.001917 17313 raft_consensus.cc:493] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:44.002619 17313 raft_consensus.cc:515] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.005223 17313 leader_election.cc:290] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:44.007517 17183 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.009318  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.016170  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.018748  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.021824  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248d73a0462c48fdad073c597b243806" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.026379  7626 leader_election.cc:304] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.037297  7556 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 28, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 28 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.049197 17313 raft_consensus.cc:2749] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:44.100606  7556 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:44.059376  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:44.062832  7626 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:44.080643 17209 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.080829  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24790f3be4e8465581029a2e0d526117" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.105072 17313 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:44.105441  7624 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:44.107251  7559 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 183) took 115503 ms (client timeout 15000 ms). Trace:
I20260709 12:29:44.108096 17209 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:44.094895 17183 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.116391 17183 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.119951 17183 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:44.121093  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 61791.3us
I20260709 12:29:44.126600  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.128877 17313 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.129657 17313 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.133946 17313 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:44.136222  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.138022  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.140620  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.141227  7626 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.142832 17183 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:44.102645  7556 catalog_manager.cc:5697] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.149165  7556 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.152091  7556 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:44.158601  7556 catalog_manager.cc:5697] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 13 to 21, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.165503  7626 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.166477 17313 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:44.161286  7556 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:44.170289  7556 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:44.174808  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:44.185799  7556 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 18, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 18 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.188788  7556 catalog_manager.cc:5697] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:44.121114  7559 rpcz_store.cc:269] 0709 12:27:48.611374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.611740 (+   366us) service_pool.cc:224] Handling call
0709 12:29:44.107128 (+115495388us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:43.892669 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:43.893368 (+   699us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:43.893418 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:29:43.893439 (+    21us) tablet.cc:662] Decoding operations
0709 12:29:43.894053 (+   614us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:43.894157 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:43.894190 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:29:43.894715 (+   525us) tablet.cc:701] Row locks acquired
0709 12:29:43.894741 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:29:43.894956 (+   215us) write_op.cc:270] Start()
0709 12:29:43.895160 (+   204us) write_op.cc:276] Timestamp: P: 1783600183894891 usec, L: 0
0709 12:29:43.895195 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:29:43.896344 (+  1149us) log.cc:844] Serialized 913 byte log entry
0709 12:29:43.923582 (+ 27238us) op_driver.cc:464] REPLICATION: finished
0709 12:29:43.925255 (+  1673us) write_op.cc:301] APPLY: starting
0709 12:29:43.925370 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:43.928230 (+  2860us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:43.928356 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:43.929765 (+  1409us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:43.930042 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:29:43.930106 (+    64us) write_op.cc:312] APPLY: finished
0709 12:29:43.930917 (+   811us) log.cc:844] Serialized 33 byte log entry
0709 12:29:43.931960 (+  1043us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:43.932192 (+   232us) write_op.cc:454] Released schema lock
0709 12:29:43.932286 (+    94us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":20027,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1339,"apply.run_cpu_time_us":5942,"apply.run_wall_time_us":9942,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12487,"num_ops":2,"prepare.queue_time_us":46972,"prepare.run_cpu_time_us":3999,"prepare.run_wall_time_us":4115,"replication_time_us":22412,"thread_start_us":55613,"threads_started":2}]]}
I20260709 12:29:44.191676  7556 catalog_manager.cc:5697] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 20 to 24, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 24 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.200585  7556 catalog_manager.cc:5697] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.205497  7556 catalog_manager.cc:5697] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.207695  7556 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 27, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 27 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.212805  7556 catalog_manager.cc:5697] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.223747  7556 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 20 to 23, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 23 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:44.230104  7556 catalog_manager.cc:5697] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 23, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 23 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:44.235262  7556 catalog_manager.cc:5697] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 20, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.238629  7556 catalog_manager.cc:5697] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.247933  7556 catalog_manager.cc:5697] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:44.259940  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.258555  7556 catalog_manager.cc:5697] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.266976  7556 catalog_manager.cc:5697] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:44.268625  7556 catalog_manager.cc:5697] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:44.273849  7556 catalog_manager.cc:5697] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.275372  7556 catalog_manager.cc:5697] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 28, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 28 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:44.278110  7808 consensus_queue.cc:237] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:44.287632  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:44.277668  7556 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:44.325024  6451 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:44.327044  8414 raft_consensus.cc:1275] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:29:44.328943 17371 consensus_queue.cc:1048] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:29:44.330106  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.351428 17313 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.355544 17313 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:44.359407 17313 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.360198  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.360538  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.364244  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.369833 17313 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:44.427958  8409 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 8 to 17, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.448808 17280 raft_consensus.cc:2955] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Committing config change with OpId 24.17: config changed from index 8 to 17, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:44.461828  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 67845.2us
I20260709 12:29:44.487165 17313 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.488076 17313 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:44.493656 17313 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.495841  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.498919  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.501374  7624 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:44.502947  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.511809 17183 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:44.532899  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:44.545974  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 77188.1us
W20260709 12:29:44.618930  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:44.640090  7545 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:29:44.647541  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.663470 17209 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:44.664289 17209 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.667991  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.669658 17209 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:44.674712  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.688360  7626 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.696517 17183 raft_consensus.cc:2749] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:44.720606  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:44.736958  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.767647 17209 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.768455 17209 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.781611 17209 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:44.782986  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:44.791381  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 56720.6us
I20260709 12:29:44.801537 17209 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.802232 17209 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:44.806325  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.808525  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.814762  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.815295 17313 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.815884 17313 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:44.816743  7627 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.818717 17313 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:44.819675 17183 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:44.821738 17209 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.841954  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:44.848394  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:44.849125  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:44.849782  7702 consensus_peers.cc:597] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:44.856676  7551 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:44.861320 17391 raft_consensus.cc:493] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.862193 17391 raft_consensus.cc:515] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:44.867885  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.869992  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.882880 17209 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:44.883558 17209 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:44.887843  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.888056  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:44.890756  7624 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.891799 17183 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:44.893462 17209 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:44.893914  7551 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 24 to 25, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 25 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:44.905920  7551 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:44.900928  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:44.903669  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.939108 17313 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:44.899784 17391 leader_election.cc:290] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:44.933180  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.939693  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:44.954428 17313 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:44.993667  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.995298  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40860756ca234260bb6147416168cd6a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:44.985426  7551 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 14 to 23, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 23 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:45.099903  7556 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 274) took 9400 ms. Trace:
I20260709 12:29:45.109577  7551 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 21 to 24, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 24 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:45.121583  7551 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 19, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 19 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:45.123468  7551 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.100414  7556 rpcz_store.cc:276] 0709 12:29:35.700165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:43.700686 (+8000521us) service_pool.cc:224] Handling call
0709 12:29:45.099782 (+1399096us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:44.376095 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:44.376674 (+   579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:44.376740 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:29:44.376758 (+    18us) tablet.cc:662] Decoding operations
0709 12:29:44.379071 (+  2313us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:44.379136 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:44.379164 (+    28us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:29:44.382852 (+  3688us) tablet.cc:701] Row locks acquired
0709 12:29:44.382876 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:29:44.383035 (+   159us) write_op.cc:270] Start()
0709 12:29:44.383161 (+   126us) write_op.cc:276] Timestamp: P: 1783600184382984 usec, L: 0
0709 12:29:44.383182 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:29:44.384013 (+   831us) log.cc:844] Serialized 10100 byte log entry
0709 12:29:44.394650 (+ 10637us) op_driver.cc:464] REPLICATION: finished
0709 12:29:44.395241 (+   591us) write_op.cc:301] APPLY: starting
0709 12:29:44.395354 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:44.602002 (+206648us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:44.602103 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:44.721205 (+119102us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:44.721579 (+   374us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 12:29:44.721631 (+    52us) write_op.cc:312] APPLY: finished
0709 12:29:44.722199 (+   568us) log.cc:844] Serialized 228 byte log entry
0709 12:29:44.739936 (+ 17737us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:44.740160 (+   224us) write_op.cc:454] Released schema lock
0709 12:29:44.740234 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:44.740657 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:44.740699 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4280,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":311,"apply.run_cpu_time_us":36748,"apply.run_wall_time_us":345388,"cfile_cache_hit":104,"cfile_cache_hit_bytes":108680,"num_ops":26,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":6622,"prepare.run_wall_time_us":8235,"replication_time_us":11283}]]}
I20260709 12:29:45.239326  7551 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 12 to 21, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:45.270073  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.314116  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44115.4us
W20260709 12:29:45.317277  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.328145  6451 thread.cc:527] Waited for 12000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:45.346161 17313 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.346822 17313 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.350014 17313 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:45.293897 17391 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.382306 17391 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.397573 17391 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:45.412597  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.426375 17313 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.427237 17313 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.431200 17313 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:45.302225  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44357.1us
I20260709 12:29:45.443166 17209 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.465569 17209 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.468312 17209 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:45.451851  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.492247  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.504426  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.453022 17391 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.540446 17391 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.554877 17391 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:45.566689  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:45.384827  7551 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 11 to 20, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 20 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.607895  7551 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 15 to 23, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 23 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:45.457734  7626 leader_election.cc:304] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:45.644886  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.670884 17391 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.804167 17391 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.809971 17391 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:45.763446  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.812453  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 49280.1us
W20260709 12:29:45.813633  7702 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.794404  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:45.763988 17337 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:45.826061 17337 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:45.830291 17337 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:45.837862 17391 raft_consensus.cc:2749] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:45.839951  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 382137us
W20260709 12:29:45.878409  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 80442.2us
I20260709 12:29:45.901822  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.900095 17313 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.904350 17313 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.907441  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.908403  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.909698  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.914274 17313 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:45.915879  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c7a755001445e2895ad66ede4c8f60" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.916460  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b8edd0c3f4472bb922dba093edf25a" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.916884  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.917959  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:45.925316  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:45.929749 17313 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.930881 17313 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:45.961696  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:45.983033  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:45.983794 17313 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:45.985080  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.989029  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.990696  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:45.992642  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.004400  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:46.008232  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:46.011529  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.018358 17313 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:46.020489 17313 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.021203 17313 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:46.025861  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:46.028052  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.028368 17313 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:46.033303  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:46.030970  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
W20260709 12:29:46.037485  8380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 60583.6us
W20260709 12:29:46.039553  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:46.051950  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 49268.9us
I20260709 12:29:46.053249  7626 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.054436 17391 raft_consensus.cc:2749] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:46.067698  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:46.073117  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:46.081507 17399 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:46.083701  7626 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.091686 17313 raft_consensus.cc:2749] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:46.099160  7627 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.099388  7626 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.101570 17209 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:46.104784 17209 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:46.099146  7702 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:46.113484  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:46.120628  7702 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 325 similar messages]
W20260709 12:29:46.135382  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 107423us
I20260709 12:29:46.135316  7626 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.152361 17209 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:46.152750  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 50718.7us
I20260709 12:29:46.153736  7626 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.158357 17313 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:46.160522  7626 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.163396  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.164390 17209 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:46.184336  7626 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.185384 17209 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:46.188318  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.196920  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.200670  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.201545  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.202438  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:46.252686 17313 raft_consensus.cc:493] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.253567 17313 raft_consensus.cc:515] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:46.257820 17313 leader_election.cc:290] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:46.261274  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.282621  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee2ab66e8604e5bbd1f08acbef110d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:46.286336  7624 leader_election.cc:304] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:46.288113 17209 raft_consensus.cc:2749] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:46.328962  6451 thread.cc:527] Waited for 13000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:46.342446 17209 raft_consensus.cc:493] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:46.343621 17209 raft_consensus.cc:515] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:46.355527  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:46.354012 17209 leader_election.cc:290] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:46.375172  7702 consensus_peers.cc:597] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:46.394862  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 63537.5us
I20260709 12:29:46.401245  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbccb309817428d8d84eeecd4ac4410" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.412907  7626 leader_election.cc:304] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:46.414104 17209 raft_consensus.cc:2749] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:46.433418  7556 catalog_manager.cc:5697] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 28 to 30, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 30 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:46.444357  7551 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 186) took 43367 ms (client timeout 15000 ms). Trace:
I20260709 12:29:46.448283  7556 catalog_manager.cc:5697] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.457705 17209 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.458616 17209 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:46.450778  7556 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 26, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 26 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.463593  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:46.467314 17209 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:46.469460  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.473209  7626 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:46.445003  7551 rpcz_store.cc:269] 0709 12:29:03.077493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:11.227800 (+8150307us) service_pool.cc:224] Handling call
0709 12:29:46.444186 (+35216386us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:45.703475 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:45.704344 (+   869us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:45.704403 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:29:45.704439 (+    36us) tablet.cc:662] Decoding operations
0709 12:29:45.705743 (+  1304us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:45.705854 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:45.705891 (+    37us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:29:45.707153 (+  1262us) tablet.cc:701] Row locks acquired
0709 12:29:45.742992 (+ 35839us) write_op.cc:260] PREPARE: finished
0709 12:29:45.743669 (+   677us) write_op.cc:270] Start()
0709 12:29:45.743907 (+   238us) write_op.cc:276] Timestamp: P: 1783600185743178 usec, L: 0
0709 12:29:45.743980 (+    73us) op_driver.cc:348] REPLICATION: starting
0709 12:29:45.745416 (+  1436us) log.cc:844] Serialized 3954 byte log entry
0709 12:29:45.904175 (+158759us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18002b99c0. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fbffee35522 00007fbffee377c1 00007fbffee37779 00007fbffee37709 00007fbffee376d1 00007fbffee369cd 0000000000388634 0000000000388479 00007fbffee1883a
0709 12:29:46.033136 (+128961us) op_driver.cc:464] REPLICATION: finished
0709 12:29:46.108469 (+ 75333us) write_op.cc:301] APPLY: starting
0709 12:29:46.108840 (+   371us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:46.132866 (+ 24026us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:46.132918 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:46.168714 (+ 35796us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:46.169071 (+   357us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:29:46.169136 (+    65us) write_op.cc:312] APPLY: finished
0709 12:29:46.206099 (+ 36963us) log.cc:844] Serialized 97 byte log entry
0709 12:29:46.206338 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:46.206493 (+   155us) write_op.cc:454] Released schema lock
0709 12:29:46.206564 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:46.207300 (+   736us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:46.207336 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":63535,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":74883,"apply.run_cpu_time_us":18423,"apply.run_wall_time_us":98799,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45927,"num_ops":10,"prepare.queue_time_us":70164,"prepare.run_cpu_time_us":82913,"prepare.run_wall_time_us":192028,"raft.queue_time_us":141689,"raft.run_cpu_time_us":9040,"raft.run_wall_time_us":45963,"replication_time_us":288932,"spinlock_wait_cycles":13184256,"thread_start_us":289822,"threads_started":4,"wal-append.queue_time_us":141609}]]}
I20260709 12:29:46.475047 17209 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:46.476449  7556 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 17 to 22, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 22 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.478683  7556 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 23, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 23 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.480561  7556 catalog_manager.cc:5697] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:46.482784  7556 catalog_manager.cc:5697] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.499553 17313 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.500501 17313 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:46.504019  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:46.504729  7556 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.508371  7556 catalog_manager.cc:5697] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.519498  7556 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 24, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 24 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.520274 17313 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:46.522053  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.530799  7556 catalog_manager.cc:5697] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.533622  7626 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.533111  7556 catalog_manager.cc:5697] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.534615 17209 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:46.535089  7556 catalog_manager.cc:5697] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.540478  7556 catalog_manager.cc:5697] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 23 to 25, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 25 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.542775  7556 catalog_manager.cc:5697] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.549228  7556 catalog_manager.cc:5697] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 26, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 26 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.558413  7556 catalog_manager.cc:5697] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 20 to 25, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 25 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.560468  7556 catalog_manager.cc:5697] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.562121  7556 catalog_manager.cc:5697] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 25 to 33, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 33 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.572997  7556 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:46.577459  7556 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 21 to 29, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 29 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:46.603515  7556 catalog_manager.cc:5697] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 21 to 28, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 28 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.612434  7556 catalog_manager.cc:5697] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 18 to 28. New cstate: current_term: 28 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.614902  7556 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 14 to 19, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 19 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.622820  7556 catalog_manager.cc:5697] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 26. New cstate: current_term: 26 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.630312  7556 catalog_manager.cc:5697] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 24, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
I20260709 12:29:46.641335  7556 catalog_manager.cc:5697] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:46.643723  7556 catalog_manager.cc:5697] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 22 to 28. New cstate: current_term: 28 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:46.651582  7556 catalog_manager.cc:5697] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 10 to 23. New cstate: current_term: 23 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } }
W20260709 12:29:46.659783  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:46.696468 17313 raft_consensus.cc:493] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:46.697448 17313 raft_consensus.cc:515] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:46.702455  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:46.702139  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddca3b06c0a84859adac6776a38c3518" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.717829 17313 leader_election.cc:290] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:46.722723  7626 leader_election.cc:304] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:46.727329 17313 raft_consensus.cc:2749] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:46.760076  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:46.902384 17313 raft_consensus.cc:493] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.903180 17313 raft_consensus.cc:515] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:46.907373  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:46.915542 17313 leader_election.cc:290] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:46.918540  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8987ec43c27c4d578d5ad1e143553334" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:46.920856  7626 leader_election.cc:304] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:46.924048 17313 raft_consensus.cc:2749] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:47.104903  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:47.145571  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:47.243868 17313 raft_consensus.cc:493] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:47.244599 17313 raft_consensus.cc:515] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:47.249742  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.252846  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16eac2fd2da54bec8a02bb35cc376087" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:47.256919  7627 leader_election.cc:304] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:47.258293 17209 raft_consensus.cc:2749] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:47.261158 17313 leader_election.cc:290] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:47.283898  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:47.329555  6451 thread.cc:527] Waited for 14000ms trying to join with heartbeat (tid 7687)
I20260709 12:29:47.338217  7550 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 23 to 24, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 24 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:47.352483  7550 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 15 to 19, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 19 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:47.365562  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 47061.3us
I20260709 12:29:47.402963 17313 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:47.403951 17313 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:47.406719 17313 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:47.408890  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.421804  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:47.426676  7624 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:47.427963 17411 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:47.429088 17411 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:47.433095  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.449616 17411 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:47.460173 17209 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:47.463874  7702 consensus_peers.cc:597] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:47.510340  7624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 73326us
W20260709 12:29:47.510231  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 45190.8us
I20260709 12:29:47.542783  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:47.549849  7624 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:47.555039 17209 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:47.561357  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:47.558984  7550 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 25 to 26, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193). New cstate: current_term: 26 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:47.565611  7550 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:47.571877  7556 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60410 (request call id 275) took 2292 ms. Trace:
I20260709 12:29:47.591112 17209 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:47.592093 17209 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:47.598445 17209 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:47.601052  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.574930  7556 rpcz_store.cc:276] 0709 12:29:45.261895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:45.262091 (+   196us) service_pool.cc:224] Handling call
0709 12:29:46.703485 (+1441394us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b5400234a28. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc006e4baa1 00007fc006e729b4 00007fc0059563dc 00007fc005917e0b 00007fc005917be4 00007fc007e17fa1 00007fc007e187ff 00007fc007cd7a1b 00007fc007ddf90e 00007fc0062a02a4 00007fc0062a0232
0709 12:29:47.553771 (+850286us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:46.751066 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:46.751742 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:46.751792 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:29:46.751813 (+    21us) tablet.cc:662] Decoding operations
0709 12:29:46.754478 (+  2665us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:46.754699 (+   221us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:46.754736 (+    37us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:29:46.781541 (+ 26805us) tablet.cc:701] Row locks acquired
0709 12:29:46.781564 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:29:46.781779 (+   215us) write_op.cc:270] Start()
0709 12:29:46.781929 (+   150us) write_op.cc:276] Timestamp: P: 1783600186781711 usec, L: 0
0709 12:29:46.781957 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:29:46.782929 (+   972us) log.cc:844] Serialized 11312 byte log entry
0709 12:29:46.852871 (+ 69942us) op_driver.cc:464] REPLICATION: finished
0709 12:29:46.900127 (+ 47256us) write_op.cc:301] APPLY: starting
0709 12:29:46.900223 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:47.019508 (+119285us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:47.019662 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:47.177642 (+157980us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:47.178085 (+   443us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:29:47.178134 (+    49us) write_op.cc:312] APPLY: finished
0709 12:29:47.178749 (+   615us) log.cc:844] Serialized 252 byte log entry
0709 12:29:47.179006 (+   257us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.197610 (+ 18604us) write_op.cc:454] Released schema lock
0709 12:29:47.197697 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:47.198202 (+   505us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.198234 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4562,"reported_tablets":41,"spinlock_wait_cycles":19020544,"child_traces":[["op",{"apply.queue_time_us":38840,"apply.run_cpu_time_us":42036,"apply.run_wall_time_us":298049,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125347,"num_ops":29,"prepare.queue_time_us":46763,"prepare.run_cpu_time_us":8393,"prepare.run_wall_time_us":67755,"replication_time_us":70703,"thread_start_us":82903,"threads_started":2}]]}
I20260709 12:29:47.613394  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:47.627142  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:47.640393  7626 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:47.644541  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.648164  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.650794  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:47.655167 17209 raft_consensus.cc:2749] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:47.666715  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:47.667971 17415 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:47.668838 17415 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:47.672441 17209 raft_consensus.cc:493] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:47.673391 17209 raft_consensus.cc:515] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:47.677582 17209 leader_election.cc:290] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:47.679179  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:47.681861  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b71af551cbdf4c63a9a0da6165082874" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:47.685151  7626 leader_election.cc:304] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:47.691697 17209 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:47.695422 17209 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:47.698138 17209 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:47.699246  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.700003 17209 raft_consensus.cc:2749] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:47.709141  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:47.715296 17209 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:47.718334 17209 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:47.722796  7626 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:47.723964 17313 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:47.725381  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:47.732838 17415 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:47.734226  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:47.740391 17209 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:47.743499  7702 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:47.744221  7702 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:47.747268  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:47.757579 17415 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:47.757727  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:47.761592  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:47.763744  7626 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:47.764763 17313 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:47.838917  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.840448  7550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 289) took 36454 ms (client timeout 15000 ms). Trace:
W20260709 12:29:47.842360  7550 rpcz_store.cc:269] 0709 12:29:11.386464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:24.709045 (+13322581us) service_pool.cc:224] Handling call
0709 12:29:47.840295 (+23131250us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:47.663346 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:47.664062 (+   716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:47.664112 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:29:47.664137 (+    25us) tablet.cc:662] Decoding operations
0709 12:29:47.664874 (+   737us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:47.664952 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:47.664985 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:29:47.692916 (+ 27931us) tablet.cc:701] Row locks acquired
0709 12:29:47.692938 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:29:47.693139 (+   201us) write_op.cc:270] Start()
0709 12:29:47.693328 (+   189us) write_op.cc:276] Timestamp: P: 1783600187693071 usec, L: 0
0709 12:29:47.693356 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:29:47.694262 (+   906us) log.cc:844] Serialized 1674 byte log entry
0709 12:29:47.714111 (+ 19849us) op_driver.cc:464] REPLICATION: finished
0709 12:29:47.733626 (+ 19515us) write_op.cc:301] APPLY: starting
0709 12:29:47.733745 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:47.751986 (+ 18241us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800471e20. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007d0a281 00007fc001aca4bc 00007fbffee1848b 00007fc007c729a9 00007fbffee3469b 00007fbffee3293f 00007fbffee34c02 00007fc005958ef0 00007fc005959489 00007fc00595b71a
0709 12:29:47.754128 (+  2142us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:47.754169 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:47.757105 (+  2936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:47.761675 (+  4570us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:29:47.761726 (+    51us) write_op.cc:312] APPLY: finished
0709 12:29:47.762002 (+   276us) log.cc:844] Serialized 49 byte log entry
0709 12:29:47.764200 (+  2198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.764323 (+   123us) write_op.cc:454] Released schema lock
0709 12:29:47.764381 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:47.774603 (+ 10222us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.774639 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":175950,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":19177,"apply.run_cpu_time_us":7945,"apply.run_wall_time_us":40934,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20847,"num_ops":4,"prepare.queue_time_us":25648,"prepare.run_cpu_time_us":4040,"prepare.run_wall_time_us":31346,"replication_time_us":20534,"spinlock_wait_cycles":27480960,"thread_start_us":49576,"threads_started":2}]]}
W20260709 12:29:47.849361  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.850083  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:47.848661  7553 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:47.868604  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:47.871373  7553 catalog_manager.cc:5697] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 26 to 27, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 27 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:47.893570  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.928574  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:47.903251  7553 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:47.982184  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:47.985064  7553 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 187) took 20919 ms (client timeout 15000 ms). Trace:
W20260709 12:29:47.986348  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:47.985684  7553 rpcz_store.cc:269] 0709 12:29:27.066166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:31.716682 (+4650516us) service_pool.cc:224] Handling call
0709 12:29:47.984932 (+16268250us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:47.939323 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:47.939864 (+   541us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:47.939899 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:29:47.939916 (+    17us) tablet.cc:662] Decoding operations
0709 12:29:47.940470 (+   554us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:47.940531 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:47.940559 (+    28us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:29:47.940886 (+   327us) tablet.cc:701] Row locks acquired
0709 12:29:47.940904 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:29:47.941048 (+   144us) write_op.cc:270] Start()
0709 12:29:47.941172 (+   124us) write_op.cc:276] Timestamp: P: 1783600187940997 usec, L: 0
0709 12:29:47.941195 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:29:47.941885 (+   690us) log.cc:844] Serialized 1293 byte log entry
0709 12:29:47.944435 (+  2550us) op_driver.cc:464] REPLICATION: finished
0709 12:29:47.946209 (+  1774us) write_op.cc:301] APPLY: starting
0709 12:29:47.946307 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:47.948410 (+  2103us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:47.948533 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:47.950706 (+  2173us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:47.950951 (+   245us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:29:47.951019 (+    68us) write_op.cc:312] APPLY: finished
0709 12:29:47.952291 (+  1272us) log.cc:844] Serialized 41 byte log entry
0709 12:29:47.953526 (+  1235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.953688 (+   162us) write_op.cc:454] Released schema lock
0709 12:29:47.953745 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:47.954248 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:47.954284 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1486,"apply.run_cpu_time_us":6246,"apply.run_wall_time_us":7979,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12540,"num_ops":3,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":2918,"prepare.run_wall_time_us":2918,"replication_time_us":3077}]]}
I20260709 12:29:48.012606  7555 catalog_manager.cc:5697] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 19 to 23, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 23 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:48.015689  7555 catalog_manager.cc:5697] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:48.131843 17313 raft_consensus.cc:493] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:48.132753 17313 raft_consensus.cc:515] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:48.133121  7552 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 19 to 20, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.136184 17313 leader_election.cc:290] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:48.137003  7552 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 24 to 28, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 28 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.137748  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:48.140450  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e2e2ecc360e4304b9a841fbd6b8287e" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:48.159902  7552 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 24 to 26, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 26 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.165087  7552 catalog_manager.cc:5697] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 20 to 25, leader changed from 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 25 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:48.174366  7555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60404 (request call id 290) took 14677 ms (client timeout 15000 ms). Trace:
W20260709 12:29:48.199507  7555 rpcz_store.cc:269] 0709 12:29:33.497001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:40.257465 (+6760464us) service_pool.cc:224] Handling call
0709 12:29:48.048983 (+7791518us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.049025 (+    42us) write_op.cc:454] Released schema lock
0709 12:29:48.174254 (+125229us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:48.023328 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:48.023923 (+   595us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:48.023962 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:29:48.023981 (+    19us) tablet.cc:662] Decoding operations
0709 12:29:48.024407 (+   426us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:48.024468 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:48.024504 (+    36us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:29:48.024850 (+   346us) tablet.cc:701] Row locks acquired
0709 12:29:48.024867 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:29:48.025033 (+   166us) write_op.cc:270] Start()
0709 12:29:48.025155 (+   122us) write_op.cc:276] Timestamp: P: 1783600188024975 usec, L: 0
0709 12:29:48.025173 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:29:48.025929 (+   756us) log.cc:844] Serialized 913 byte log entry
0709 12:29:48.028261 (+  2332us) op_driver.cc:464] REPLICATION: finished
0709 12:29:48.028864 (+   603us) write_op.cc:301] APPLY: starting
0709 12:29:48.028969 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:48.030669 (+  1700us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:48.030801 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:48.032589 (+  1788us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:48.032815 (+   226us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:29:48.032900 (+    85us) write_op.cc:312] APPLY: finished
0709 12:29:48.033204 (+   304us) log.cc:844] Serialized 33 byte log entry
0709 12:29:48.037985 (+  4781us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.038131 (+   146us) write_op.cc:454] Released schema lock
0709 12:29:48.038189 (+    58us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":290,"apply.run_cpu_time_us":5215,"apply.run_wall_time_us":9712,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8360,"num_ops":2,"prepare.queue_time_us":157,"prepare.run_cpu_time_us":2950,"prepare.run_wall_time_us":2950,"replication_time_us":2916}]]}
I20260709 12:29:48.177769  7627 leader_election.cc:304] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:48.174469  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.211469 17313 raft_consensus.cc:2749] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:48.193858  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:48.225862  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:48.234643  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:48.239989  6451 tablet_replica.cc:333] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.242470  6451 raft_consensus.cc:2243] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:29:48.253202  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:48.255623  6451 raft_consensus.cc:2272] T 575c307765244f48890311159c0ee934 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:48.260218  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.264573  7558 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:48.266505  7558 catalog_manager.cc:5697] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 26 to 28, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 28 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.284512  6451 tablet_replica.cc:333] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.286952  6451 raft_consensus.cc:2243] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.288369  6451 raft_consensus.cc:2272] T b3b8edd0c3f4472bb922dba093edf25a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.292678  7558 catalog_manager.cc:5697] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 23 to 26, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 26 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.296353  7558 catalog_manager.cc:5697] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 28 to 29, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 29 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.298360  6451 tablet_replica.cc:333] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.302161  6451 raft_consensus.cc:2243] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.305701  6451 raft_consensus.cc:2272] T 8987ec43c27c4d578d5ad1e143553334 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.298552  7558 catalog_manager.cc:5697] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 20 to 21, leader changed from d72176ccaed049958ac68f706d4f052d (127.6.76.195) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 21 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.318526  7558 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:48.317857  6451 tablet_replica.cc:333] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.321795  6451 raft_consensus.cc:2243] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.323473  6451 raft_consensus.cc:2272] T 08777b07fa9d4993a24c05bb4e2db3c8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.330097 17313 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:48.332988 17313 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:48.338647  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:48.340652  7702 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.345420 17209 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:48.346148 17209 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:48.346580 17313 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:48.350267  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:48.351361  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31230d31601445449fab8f0f9943db57" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:48.355268  7624 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:48.357366 17313 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:48.367367  7552 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 309) took 21290 ms (client timeout 15000 ms). Trace:
W20260709 12:29:48.367959  7552 rpcz_store.cc:269] 0709 12:29:27.077122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:36.944445 (+9867323us) service_pool.cc:224] Handling call
0709 12:29:48.231756 (+11287311us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.231786 (+    30us) write_op.cc:454] Released schema lock
0709 12:29:48.339205 (+107419us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:48.216400 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:48.216983 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:48.217053 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:29:48.217071 (+    18us) tablet.cc:662] Decoding operations
0709 12:29:48.217907 (+   836us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:48.217978 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:48.218008 (+    30us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:29:48.218462 (+   454us) tablet.cc:701] Row locks acquired
0709 12:29:48.218481 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:29:48.218651 (+   170us) write_op.cc:270] Start()
0709 12:29:48.218782 (+   131us) write_op.cc:276] Timestamp: P: 1783600188218600 usec, L: 0
0709 12:29:48.218809 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:29:48.219603 (+   794us) log.cc:844] Serialized 1674 byte log entry
0709 12:29:48.221938 (+  2335us) op_driver.cc:464] REPLICATION: finished
0709 12:29:48.222607 (+   669us) write_op.cc:301] APPLY: starting
0709 12:29:48.222694 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:48.225086 (+  2392us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:48.225196 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:48.229452 (+  4256us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:48.229657 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:29:48.229707 (+    50us) write_op.cc:312] APPLY: finished
0709 12:29:48.229959 (+   252us) log.cc:844] Serialized 49 byte log entry
0709 12:29:48.230838 (+   879us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.231036 (+   198us) write_op.cc:454] Released schema lock
0709 12:29:48.231130 (+    94us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":10126,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":427,"apply.run_cpu_time_us":6736,"apply.run_wall_time_us":8894,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16720,"num_ops":4,"prepare.queue_time_us":302,"prepare.run_cpu_time_us":3526,"prepare.run_wall_time_us":10976,"replication_time_us":2980}]]}
W20260709 12:29:48.380304  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.382388  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d64be480d9430189f5e290bab7c3b8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:48.385007  7627 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:48.386714 17313 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:48.388207 17209 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:48.403357 17313 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:48.404129 17313 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:48.419395  6451 tablet_replica.cc:333] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.425707  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:48.426999  6451 raft_consensus.cc:2243] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.428777  6451 pending_rounds.cc:70] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:48.429207  6451 pending_rounds.cc:77] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 23 index: 13 } timestamp: 7305626307320676352 op_type: NO_OP noop_request { }
I20260709 12:29:48.429786  6451 raft_consensus.cc:2889] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:48.430291  6451 raft_consensus.cc:2272] T 38bf041bf77540baa50b5319733c5e9e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:48.434028  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.451872  6451 tablet_replica.cc:333] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.473366  6451 raft_consensus.cc:2243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:48.479375  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 44570.7us
I20260709 12:29:48.500108 17313 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:48.511957  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:48.512187  6451 raft_consensus.cc:2272] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.517870  6451 tablet_replica.cc:333] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.524775  7626 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:48.541893  7558 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 188) took 6006 ms. Trace:
I20260709 12:29:48.542799 17209 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:48.557299  6451 raft_consensus.cc:2243] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.542385  7558 rpcz_store.cc:276] 0709 12:29:42.543843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:43.839214 (+1295371us) service_pool.cc:224] Handling call
0709 12:29:48.541821 (+4702607us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:48.343209 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:48.343895 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:48.343937 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:29:48.343961 (+    24us) tablet.cc:662] Decoding operations
0709 12:29:48.344871 (+   910us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:48.344945 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:48.344983 (+    38us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:29:48.345642 (+   659us) tablet.cc:701] Row locks acquired
0709 12:29:48.345665 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:29:48.345853 (+   188us) write_op.cc:270] Start()
0709 12:29:48.345994 (+   141us) write_op.cc:276] Timestamp: P: 1783600188345792 usec, L: 0
0709 12:29:48.346033 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:29:48.346861 (+   828us) log.cc:844] Serialized 2434 byte log entry
0709 12:29:48.350966 (+  4105us) op_driver.cc:464] REPLICATION: finished
0709 12:29:48.351511 (+   545us) write_op.cc:301] APPLY: starting
0709 12:29:48.351603 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:48.355894 (+  4291us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:48.356206 (+   312us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:48.363588 (+  7382us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:48.363790 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 12:29:48.363835 (+    45us) write_op.cc:312] APPLY: finished
0709 12:29:48.364106 (+   271us) log.cc:844] Serialized 65 byte log entry
0709 12:29:48.400399 (+ 36293us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.400529 (+   130us) write_op.cc:454] Released schema lock
0709 12:29:48.400578 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:48.412986 (+ 12408us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:48.413044 (+    58us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18365,"reported_tablets":31,"spinlock_wait_cycles":69888,"child_traces":[["op",{"apply.queue_time_us":309,"apply.run_cpu_time_us":10451,"apply.run_wall_time_us":61418,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25080,"num_ops":6,"prepare.queue_time_us":160,"prepare.run_cpu_time_us":3996,"prepare.run_wall_time_us":7320,"replication_time_us":4773}]]}
I20260709 12:29:48.588518  6451 raft_consensus.cc:2272] T 402d5cf4f7d24ac59dee1221fc2c7a3f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.593752  6451 tablet_replica.cc:333] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.595620  6451 raft_consensus.cc:2243] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.596580  6451 raft_consensus.cc:2272] T 0a2203ee13714c8bacacb6f83ec31034 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.597519 17422 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
W20260709 12:29:48.595881  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.623689 17422 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:48.626462 17422 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:48.627614  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:48.633025  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:48.634303  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:48.654074  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:48.654807  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:48.655966  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:48.657867  7627 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:48.659209 17313 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:48.686795  6451 tablet_replica.cc:333] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.698971  6451 raft_consensus.cc:2243] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.699754  6451 raft_consensus.cc:2272] T 189237b70e7f43458ac90a5773fd551f P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.704635  6451 tablet_replica.cc:333] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.705991  6451 raft_consensus.cc:2243] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:29:48.708137  6451 raft_consensus.cc:2272] T 7112151d65374f6690a117439cc2dc33 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:48.712146  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:48.713013  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:48.719683  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.724020  7558 catalog_manager.cc:5697] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: config changed from index 8 to 17, VOTER fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) evicted. New cstate: current_term: 24 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:48.748119  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.756592  6451 tablet_replica.cc:333] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.768206  6451 raft_consensus.cc:2243] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.776185  6451 raft_consensus.cc:2272] T 756aa7b7913e49aaba2a2164e45553fe P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.781075  6451 tablet_replica.cc:333] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.782186  6451 raft_consensus.cc:2243] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.783931  6451 raft_consensus.cc:2272] T 35934889fce04bc093f07a0f03c7165c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.788075  6451 tablet_replica.cc:333] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
W20260709 12:29:48.799160  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:48.803790  6451 raft_consensus.cc:2243] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.815536  6451 raft_consensus.cc:2272] T 248d73a0462c48fdad073c597b243806 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.838390  6451 tablet_replica.cc:333] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
W20260709 12:29:48.838603  7554 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35204 (request call id 184) took 96151 ms (client timeout 15000 ms). Trace:
W20260709 12:29:48.853835  7554 rpcz_store.cc:269] 0709 12:28:12.687197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.695482 (+  8285us) service_pool.cc:224] Handling call
0709 12:29:48.838368 (+96142886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48227,"reported_tablets":31}
I20260709 12:29:48.847085  6451 raft_consensus.cc:2243] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.857364  6451 raft_consensus.cc:2272] T 16eac2fd2da54bec8a02bb35cc376087 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.861801  6451 tablet_replica.cc:333] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.862957  6451 raft_consensus.cc:2243] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.903885  6451 pending_rounds.cc:70] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:48.904253  6451 pending_rounds.cc:77] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 25 index: 13 } timestamp: 7305626307166597120 op_type: NO_OP noop_request { }
I20260709 12:29:48.904738  6451 raft_consensus.cc:2889] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:48.905155  6451 raft_consensus.cc:2272] T 100d9b95a9cf4fab8305b579485c2332 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.908655  7723 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f766c3d13d4360bcdeb1d5f0f318a8 with delete_type TABLET_DATA_TOMBSTONED (TS fc963afbbf4c4b34b161448ef79bd2ed not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:40334
I20260709 12:29:48.926473  6451 tablet_replica.cc:333] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.928005  6451 raft_consensus.cc:2243] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.929071  6451 pending_rounds.cc:70] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 2 pending ops.
I20260709 12:29:48.929368  6451 pending_rounds.cc:77] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 27 index: 17 } timestamp: 7305626283172995072 op_type: NO_OP noop_request { }
I20260709 12:29:48.929862  6451 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:48.930274  6451 pending_rounds.cc:77] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 25 index: 16 } timestamp: 7305626245584834560 op_type: NO_OP noop_request { }
I20260709 12:29:48.930752  6451 raft_consensus.cc:2889] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:48.931296  6451 raft_consensus.cc:2272] T b71af551cbdf4c63a9a0da6165082874 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.935141  6451 tablet_replica.cc:333] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.937366  6451 raft_consensus.cc:2243] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.938174  6451 raft_consensus.cc:2272] T d997d33a05304f31b50341cc8c4bbf9b P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.953364  6451 tablet_replica.cc:333] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.957039  6451 raft_consensus.cc:2243] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:48.962971  6451 raft_consensus.cc:2272] T bae510633fdb47c1a109530c37a265b0 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.976280  6451 tablet_replica.cc:333] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:48.977460  6451 raft_consensus.cc:2243] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.978639  6451 raft_consensus.cc:2272] T 1e2e2ecc360e4304b9a841fbd6b8287e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.980301 17430 tablet_replica.cc:333] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:48.991747 17430 raft_consensus.cc:2243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:48.992890 17430 raft_consensus.cc:2272] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:48.997203  7559 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 310) took 5708 ms. Trace:
I20260709 12:29:49.000391  7559 rpcz_store.cc:276] 0709 12:29:43.288708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:44.197906 (+909198us) service_pool.cc:224] Handling call
0709 12:29:48.957275 (+4759369us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b68000a4360. stack: 00007fbffee03343 00007fbffee03240 00007fbffe7921d4 00007fc0084d0304 00007fc0084d0299 00007fc007cfdab1 00007fc007cdad9a 00007fc007cd94d8 00007fc007cd7b10 00007fc007ddf90e 00007fc0062a02a4 00007fc0062a0232 00007fc0062a0161 00007fc0062a00dc 00007fc00629f392 00007fc00025d48c
0709 12:29:48.996984 (+ 39709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21683,"reported_tablets":49,"spinlock_wait_cycles":11523840}
I20260709 12:29:49.008641  6451 tablet_replica.cc:333] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.017467  6451 raft_consensus.cc:2243] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.023524  6451 raft_consensus.cc:2272] T 674d58a219c247d091f8a10ca4b75af1 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.019184 17422 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:49.025024 17422 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:49.028795  6451 tablet_replica.cc:333] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.029160  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:49.030155  6451 raft_consensus.cc:2243] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.033856  6451 pending_rounds.cc:70] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:49.034404  6451 pending_rounds.cc:77] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 21 index: 17 } timestamp: 7305626278626164736 op_type: NO_OP noop_request { }
I20260709 12:29:49.036616  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:49.036979  6451 raft_consensus.cc:2889] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.037515  6451 raft_consensus.cc:2272] T cbbae2856e41420d91e3ccfbf507eacf P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:49.033919  7702 consensus_peers.cc:597] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:49.039728 17422 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:49.041214  7626 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:49.042377 17422 raft_consensus.cc:2749] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:49.041939  6451 tablet_replica.cc:333] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.044009  6451 raft_consensus.cc:2243] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:49.045444  6451 raft_consensus.cc:2272] T d353e70329004cc6b0c1a6bcda38e5c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.069628 17430 ts_tablet_manager.cc:1916] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:29:49.071724  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:49.071817  7557 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60420 (request call id 307) took 83272 ms (client timeout 15000 ms). Trace:
W20260709 12:29:49.072577  7557 rpcz_store.cc:269] 0709 12:28:25.798920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.799205 (+   285us) service_pool.cc:224] Handling call
0709 12:29:49.071454 (+83272249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31363,"reported_tablets":49}
I20260709 12:29:49.089049  6451 tablet_replica.cc:333] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.090372  6451 raft_consensus.cc:2243] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.091173  6451 pending_rounds.cc:70] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:49.102417  6451 pending_rounds.cc:77] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 23 index: 17 } timestamp: 7305626274722885632 op_type: NO_OP noop_request { }
I20260709 12:29:49.102880  6451 raft_consensus.cc:2889] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.103266  6451 raft_consensus.cc:2272] T 0bbccb309817428d8d84eeecd4ac4410 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.107256  6451 tablet_replica.cc:333] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.111183  6451 raft_consensus.cc:2243] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.112592  6451 raft_consensus.cc:2272] T 268874518d1d4544a8500be048628b67 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.125450  6451 tablet_replica.cc:333] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.147284  6451 raft_consensus.cc:2243] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:29:49.148917  6451 raft_consensus.cc:2272] T e78ad8f196c145088ee9ca42482e70ce P be9c043c6d424ba1ab8def55dc9bb3e2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.164063  6451 tablet_replica.cc:333] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.176779  6451 raft_consensus.cc:2243] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.178211  6451 raft_consensus.cc:2272] T fd33cd795fd74820b3adafb3cdbc07c9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.182217 17430 ts_tablet_manager.cc:1929] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.16
I20260709 12:29:49.182957  6451 tablet_replica.cc:333] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.184073  6451 raft_consensus.cc:2243] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.185078  6451 pending_rounds.cc:70] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:49.185341  6451 pending_rounds.cc:77] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305626241953239040 op_type: NO_OP noop_request { }
I20260709 12:29:49.185742  6451 raft_consensus.cc:2889] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.186100  6451 raft_consensus.cc:2272] T b69d30f987e646bfbea975a362ad70ec P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.182946 17430 log.cc:1199] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed: Deleting WAL directory at /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/wals/59f766c3d13d4360bcdeb1d5f0f318a8
I20260709 12:29:49.191397  6451 tablet_replica.cc:333] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.191322  7546 catalog_manager.cc:5028] TS fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): tablet 59f766c3d13d4360bcdeb1d5f0f318a8 (table test-workload [id=07c684bbd79840cf825767ed5d335938]) successfully deleted
I20260709 12:29:49.192598  6451 raft_consensus.cc:2243] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.193862  6451 pending_rounds.cc:70] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 2 pending ops.
I20260709 12:29:49.194134  6451 pending_rounds.cc:77] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 26 index: 21 } timestamp: 7305626241957314560 op_type: NO_OP noop_request { }
I20260709 12:29:49.194833  6451 raft_consensus.cc:2889] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.195173  6451 pending_rounds.cc:77] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 24 index: 20 } timestamp: 7305626204958285824 op_type: NO_OP noop_request { }
I20260709 12:29:49.196026  6451 raft_consensus.cc:2889] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.196466  6451 raft_consensus.cc:2272] T 24790f3be4e8465581029a2e0d526117 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.208046  6451 tablet_replica.cc:333] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.217037  6451 raft_consensus.cc:2243] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.217788  6451 raft_consensus.cc:2272] T 79849a645ae64c3787630b03e7da6307 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.224251  6451 tablet_replica.cc:333] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.225159  6451 raft_consensus.cc:2243] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.225875  6451 raft_consensus.cc:2272] T cee2ab66e8604e5bbd1f08acbef110d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:49.251754  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:49.258479  6451 tablet_replica.cc:333] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
W20260709 12:29:49.260136  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:49.259694  6451 raft_consensus.cc:2243] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:49.262266  6451 raft_consensus.cc:2272] T 629e345c827545598250d02a9f39e2fb P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.273967  6451 tablet_replica.cc:333] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.275727  6451 raft_consensus.cc:2243] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:29:49.276881 17422 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:49.277556  6451 raft_consensus.cc:2272] T 539d703b24874183847fa9a8889111e9 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.269426 17426 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:49.277729 17422 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:49.279062 17426 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:49.282663  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:49.283042  6451 tablet_replica.cc:333] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.283955  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:49.282967 17422 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:49.288244  7626 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: be9c043c6d424ba1ab8def55dc9bb3e2; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:49.289191  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:49.290251 17426 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:49.290445 17422 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:49.290830  6451 raft_consensus.cc:2243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.292061  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:49.292420  6451 raft_consensus.cc:2272] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.294217  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:49.295409 17407 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:49.296110  6451 tablet_replica.cc:333] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.289031  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f766c3d13d4360bcdeb1d5f0f318a8" candidate_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:49.296833  7736 raft_consensus.cc:1803] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: voting while tombstoned based on last-logged opid 24.16
I20260709 12:29:49.297086  6451 raft_consensus.cc:2243] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.298045  7736 raft_consensus.cc:2410] T 59f766c3d13d4360bcdeb1d5f0f318a8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9c043c6d424ba1ab8def55dc9bb3e2 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:29:49.298143  6451 raft_consensus.cc:2272] T 41a5afed00934a1d9878dd900b236263 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.302059  6451 tablet_replica.cc:333] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.303121  6451 raft_consensus.cc:2243] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.304519  6451 raft_consensus.cc:2272] T d9c7a755001445e2895ad66ede4c8f60 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.310623  6451 tablet_replica.cc:333] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.311718  6451 raft_consensus.cc:2243] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.312464  6451 raft_consensus.cc:2272] T db9652df6fb248539296e0b903cc4d1c P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.322670  6451 tablet_replica.cc:333] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.329139  6451 raft_consensus.cc:2243] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.330863  6451 raft_consensus.cc:2272] T 0733d7d23d9041e09a5ade18807c383e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.347781  6451 tablet_replica.cc:333] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.348728  6451 raft_consensus.cc:2243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.349618  6451 pending_rounds.cc:70] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:49.349912  6451 pending_rounds.cc:77] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305626222108635136 op_type: NO_OP noop_request { }
I20260709 12:29:49.350315  6451 raft_consensus.cc:2889] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.350706  6451 raft_consensus.cc:2272] T 59f766c3d13d4360bcdeb1d5f0f318a8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.354447  6451 tablet_replica.cc:333] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.355634  6451 raft_consensus.cc:2243] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.356531  6451 raft_consensus.cc:2272] T 40860756ca234260bb6147416168cd6a P be9c043c6d424ba1ab8def55dc9bb3e2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.360389  6451 tablet_replica.cc:333] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.361354  6451 raft_consensus.cc:2243] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.362170  6451 pending_rounds.cc:70] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Trying to abort 1 pending ops.
I20260709 12:29:49.362460  6451 pending_rounds.cc:77] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2: Aborting op as it isn't in flight: id { term: 24 index: 13 } timestamp: 7305626266909872128 op_type: NO_OP noop_request { }
I20260709 12:29:49.362854  6451 raft_consensus.cc:2889] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:49.372961 17416 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:49.374241 17416 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:49.381213 17416 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:49.384174  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:49.385382  7702 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:49.394348  6451 raft_consensus.cc:2272] T 3960664d9eb541f185c9eac4a231e5c3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.385366  7812 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 23.
I20260709 12:29:49.397002  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:49.398187 17416 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:49.398708 17416 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:49.399519 17416 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:49.403877  6451 tablet_replica.cc:333] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.405133  6451 raft_consensus.cc:2243] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.406313  6451 raft_consensus.cc:2272] T 691f0d885861432f91375d5f288af312 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.416998  6451 tablet_replica.cc:333] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.418020  6451 raft_consensus.cc:2243] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.418790  6451 raft_consensus.cc:2272] T ddca3b06c0a84859adac6776a38c3518 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:49.421332  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:49.428776 17416 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:49.444787  6451 tablet_replica.cc:333] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.445777  6451 raft_consensus.cc:2243] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.446641  6451 raft_consensus.cc:2272] T 7e34ef3d78ea4d53847d21b92b07fab2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.450508  6451 tablet_replica.cc:333] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.459896 17416 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:49.461277  6451 raft_consensus.cc:2243] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.462603  6451 raft_consensus.cc:2272] T 9b24ad6f078449b6a6418a331b6099f2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.462939  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575c307765244f48890311159c0ee934" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:49.463739  7812 raft_consensus.cc:3060] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:49.467636  7702 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:49.478192  6451 tablet_replica.cc:333] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.479815  6451 raft_consensus.cc:2243] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.481261  6451 raft_consensus.cc:2272] T 62d1255be1b348739b8310c8f1a903d2 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.501387  6451 tablet_replica.cc:333] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.502476  6451 raft_consensus.cc:2243] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.503544  6451 raft_consensus.cc:2272] T a0adec428658479f82eeacfa4eb81d5e P be9c043c6d424ba1ab8def55dc9bb3e2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.511785  6451 tablet_replica.cc:333] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.512665  6451 raft_consensus.cc:2243] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.513427  6451 raft_consensus.cc:2272] T a3d64be480d9430189f5e290bab7c3b8 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.519441  6451 tablet_replica.cc:333] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.520401  6451 raft_consensus.cc:2243] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:49.521103  6451 raft_consensus.cc:2272] T 31230d31601445449fab8f0f9943db57 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.525135  6451 tablet_replica.cc:333] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2: stopping tablet replica
I20260709 12:29:49.526213  6451 raft_consensus.cc:2243] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:49.527781  6451 raft_consensus.cc:2272] T 033af988cc8f496ca24e4443c21ea5d3 P be9c043c6d424ba1ab8def55dc9bb3e2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:49.529227  7812 raft_consensus.cc:2468] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:49.541482  7702 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:49.542529 17431 raft_consensus.cc:2804] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:49.543381 17431 raft_consensus.cc:697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:49.544672 17431 consensus_queue.cc:237] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:49.566447  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:49.601197  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:49.620376  7557 catalog_manager.cc:5697] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 23 to 24, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 24 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:49.653589  7702 consensus_peers.cc:597] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:49.866750  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:49.893436 17400 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:49.894150 17400 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:49.898094  7737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:49.900252 17400 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:49.927974  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:49.931493  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:49.937960 17400 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:29:49.942569  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:50.034098  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:50.038957  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.062688 17416 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
W20260709 12:29:50.063805  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.063463 17416 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.066295 17416 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:50.067035  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.069484  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:50.070350  8414 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 19.
I20260709 12:29:50.072057  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:50.073539 17416 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:29:50.074158 17416 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.074870 17416 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:29:50.094862  7702 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.104008  7737 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:50.107838 17416 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:50.121915  7702 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.123935 17416 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:50.144644  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7112151d65374f6690a117439cc2dc33" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501"
I20260709 12:29:50.145609  8414 raft_consensus.cc:3060] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:29:50.161374  7803 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:50.176043 17416 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:29:50.199422  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.201851  7812 raft_consensus.cc:1275] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:29:50.210909  7702 consensus_peers.cc:597] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.213336  8414 raft_consensus.cc:2468] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 20.
I20260709 12:29:50.225004 17416 consensus_queue.cc:1048] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:29:50.227924  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.235004  7700 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:50.236722 17431 raft_consensus.cc:2804] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:29:50.237286 17431 raft_consensus.cc:697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:50.238381 17431 consensus_queue.cc:237] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:50.270236  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:50.297210 17443 raft_consensus.cc:2955] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:50.302961 17426 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.299078  7803 raft_consensus.cc:2955] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:50.306176 17426 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.310004  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:50.309389  7559 catalog_manager.cc:5697] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 19 to 20, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 20 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:50.313068 17431 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.315312 17431 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.316653 17426 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:50.319105 17431 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:50.319198  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.326129  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:50.342989  7557 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 16, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 24 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:29:50.352267  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.352919  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:50.354039 17426 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:50.359437  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 80617.2us
W20260709 12:29:50.361097  7700 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b1e442a1beb4dc0bb3b6aef6d503ea5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.411783 17371 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.412490 17371 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.414901 17371 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:50.417367  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:50.438735  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 73722.9us
W20260709 12:29:50.439766  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:50.440330  7702 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.440764  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:50.441697 17431 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:50.428627  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.457521 17212 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.458261 17212 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.460543 17212 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:50.469107  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:50.469951  7803 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 25.
W20260709 12:29:50.470472  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:50.471508  7777 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.474728  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
W20260709 12:29:50.484129  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.485980 17212 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:50.486510 17212 raft_consensus.cc:493] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.486967 17212 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:50.505656 17431 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.506436 17431 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.510836 17431 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:50.517854  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:50.532174 17431 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.532968 17431 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.535296  7812 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:50.536123 17431 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:50.476979  7777 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:50.541566 17425 raft_consensus.cc:2749] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:50.542780  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 71320.2us
I20260709 12:29:50.544363  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:50.545113  7803 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 24.
I20260709 12:29:50.548159  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:50.549734 17431 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:50.550312 17431 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.550750 17431 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:50.564508 17431 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.569126 17431 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:50.576403  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d353e70329004cc6b0c1a6bcda38e5c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:50.577539  7803 raft_consensus.cc:3060] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:50.608608 17212 raft_consensus.cc:515] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.609052 17371 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.613024 17212 leader_election.cc:290] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:50.616940 17400 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.609730 17371 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.617779 17400 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.621838 17371 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:50.621467  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:50.628715 17400 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:50.629937  7803 raft_consensus.cc:2468] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc963afbbf4c4b34b161448ef79bd2ed in term 25.
I20260709 12:29:50.631456 17431 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.632246 17451 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.632200 17431 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.633034 17451 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.635176 17431 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:50.638052 17451 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:50.638247  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:50.640411  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:50.639026  7731 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 18.
I20260709 12:29:50.641309  7736 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 26.
I20260709 12:29:50.644366  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:50.641918  7702 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:50.645386  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.648205 17431 raft_consensus.cc:2804] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:50.648861 17425 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:50.649349 17425 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.649711 17425 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:50.644790  8380 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
W20260709 12:29:50.659037  7775 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b24ad6f078449b6a6418a331b6099f2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.664587  7731 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:29:50.658107 17461 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:29:50.670326 17461 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.670840 17461 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:50.673241  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae510633fdb47c1a109530c37a265b0" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:50.674001  7812 raft_consensus.cc:3060] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:50.655292 17431 raft_consensus.cc:697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 LEADER]: Becoming Leader. State: Replica: fc963afbbf4c4b34b161448ef79bd2ed, State: Running, Role: LEADER
I20260709 12:29:50.678238 17431 consensus_queue.cc:237] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.689250 17425 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:50.694196  7777 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.695118 17371 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:29:50.694885  7812 raft_consensus.cc:2468] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 26.
I20260709 12:29:50.707681  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:50.709017  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:50.709784  7777 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.710253  7777 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:50.711084 17371 raft_consensus.cc:2749] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:50.713055  8414 raft_consensus.cc:1275] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:29:50.725430  8379 leader_election.cc:304] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:50.728185 17462 raft_consensus.cc:2804] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:50.729452 17462 raft_consensus.cc:697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
W20260709 12:29:50.726303  7777 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.718467 17461 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:50.732324 17451 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.735766 17451 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:50.737143  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:50.738343 17451 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:50.740152  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.738287  7736 raft_consensus.cc:3060] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:50.743531  8412 raft_consensus.cc:2393] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc963afbbf4c4b34b161448ef79bd2ed in current term 19: Already voted for candidate 992d6df95ef74f7f8b12534534a45501 in this term.
I20260709 12:29:50.745066  7729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" is_pre_election: true
I20260709 12:29:50.745980  7729 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 22.
W20260709 12:29:50.747515  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:50.748008  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:50.749063 17371 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:50.749569 17371 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:50.750427 17371 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:50.739787 17425 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:50.759819  7736 raft_consensus.cc:2468] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 27.
I20260709 12:29:50.759006  7559 catalog_manager.cc:5697] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: term changed from 24 to 25, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194). New cstate: current_term: 25 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:50.761456  7778 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:50.768697 17461 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:50.770841 17431 consensus_queue.cc:1048] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:50.772732 17455 raft_consensus.cc:2955] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:29:50.780507  8379 leader_election.cc:336] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:50.785140  8379 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.763463 17425 raft_consensus.cc:2804] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:50.782030  7736 raft_consensus.cc:2955] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:50.787696 17425 raft_consensus.cc:697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
I20260709 12:29:50.788892 17425 consensus_queue.cc:237] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:50.789793  7777 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.795492  7731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:50.797593  7731 raft_consensus.cc:3060] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:29:50.749439 17462 consensus_queue.cc:237] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:50.812600  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:50.819031  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:50.822868  7731 raft_consensus.cc:2468] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 19.
I20260709 12:29:50.824419  8380 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed; no voters: 
I20260709 12:29:50.826360 17461 raft_consensus.cc:2804] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:29:50.828498 17461 raft_consensus.cc:697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:50.828714  7803 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:50.830435 17461 consensus_queue.cc:237] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:50.833040  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.839826 17371 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:50.860316  7777 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.863500  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fc963afbbf4c4b34b161448ef79bd2ed"
I20260709 12:29:50.864893  7736 raft_consensus.cc:3060] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:50.865811 17371 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:50.902004  8379 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:50.916611  7731 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:29:50.922473 17371 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:50.924445  7736 raft_consensus.cc:2468] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 23.
I20260709 12:29:50.918704  7559 catalog_manager.cc:5697] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 22 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:50.939785  7778 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:50.941241 17371 raft_consensus.cc:2804] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:50.942703 17371 raft_consensus.cc:697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Becoming Leader. State: Replica: d72176ccaed049958ac68f706d4f052d, State: Running, Role: LEADER
W20260709 12:29:50.949043  7775 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0adec428658479f82eeacfa4eb81d5e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.979123 17466 ts_tablet_manager.cc:933] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:50.980490  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:50.988128 17371 consensus_queue.cc:237] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:51.010113  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:51.010769  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.023366  7557 catalog_manager.cc:5697] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 26 to 27, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 27 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:51.025027  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:51.020054  7554 catalog_manager.cc:5697] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 25 to 26, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 26 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:51.042178  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:51.045737  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:50.991335 17462 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:51.046880 17462 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:51.051687  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.068703 17466 tablet_copy_client.cc:323] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
I20260709 12:29:51.084134  7747 tablet_copy_service.cc:140] P fc963afbbf4c4b34b161448ef79bd2ed: Received BeginTabletCopySession request for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:53564)
I20260709 12:29:51.085003  7747 tablet_copy_service.cc:161] P fc963afbbf4c4b34b161448ef79bd2ed: Beginning new tablet copy session on tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:53564: session id = 992d6df95ef74f7f8b12534534a45501-1b1e442a1beb4dc0bb3b6aef6d503ea5
W20260709 12:29:51.086784  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:51.087519  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:51.100625  7731 raft_consensus.cc:2955] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:51.104797 17462 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:51.117729  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:51.118628  7803 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 29.
W20260709 12:29:51.118907  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.120072  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:51.078038 17425 raft_consensus.cc:2955] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:51.140919  7812 raft_consensus.cc:1275] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:51.148159 17431 consensus_queue.cc:1048] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:51.135535 17461 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:51.156108 17461 raft_consensus.cc:493] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting leader election (detected failure of leader be9c043c6d424ba1ab8def55dc9bb3e2)
I20260709 12:29:51.156450  7747 tablet_copy_source_session.cc:215] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:51.156754 17461 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:29:51.157223  8379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111) [suppressed 417 similar messages]
I20260709 12:29:51.166239 17466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:51.160398  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:51.188309  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.187616  7557 catalog_manager.cc:5697] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 18 to 19, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 19 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:51.190303 17479 ts_tablet_manager.cc:933] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:51.261752  7803 raft_consensus.cc:1275] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:29:51.268308 17212 consensus_queue.cc:1048] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:51.258679  7559 catalog_manager.cc:5697] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed reported cstate change: config changed from index -1 to 12, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 20 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:51.276719  7729 raft_consensus.cc:1275] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 19 index: 18. (index mismatch)
I20260709 12:29:51.283202 17212 consensus_queue.cc:1048] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:51.278474 17461 raft_consensus.cc:515] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:51.298635 17461 leader_election.cc:290] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: Requested vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:51.302547  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
W20260709 12:29:51.305614  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:51.309067  7554 catalog_manager.cc:5697] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d reported cstate change: term changed from 22 to 23, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to d72176ccaed049958ac68f706d4f052d (127.6.76.195). New cstate: current_term: 23 leader_uuid: "d72176ccaed049958ac68f706d4f052d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:51.318614  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033af988cc8f496ca24e4443c21ea5d3" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:51.319600  7805 raft_consensus.cc:3060] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:51.328481  7736 consensus_queue.cc:237] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:51.346922  7733 raft_consensus.cc:1275] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:29:51.350625 17371 consensus_queue.cc:1048] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:29:51.358707  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:51.364198  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.364357  7805 raft_consensus.cc:2468] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 30.
I20260709 12:29:51.334930 17479 tablet_copy_client.cc:323] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
I20260709 12:29:51.378253 17466 tablet_copy_client.cc:806] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:51.379165 17466 tablet_copy_client.cc:670] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
W20260709 12:29:51.381479  8379 leader_election.cc:336] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:51.384769  8379 leader_election.cc:304] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:51.386178 17448 raft_consensus.cc:2804] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:51.386832 17448 raft_consensus.cc:697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 30 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:51.387904 17448 consensus_queue.cc:237] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:51.391994 17466 tablet_copy_client.cc:538] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:29:51.405258  7702 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:51.406139  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:51.438040  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.479521  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet 9b24ad6f078449b6a6418a331b6099f2 from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:29:51.480288  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet 9b24ad6f078449b6a6418a331b6099f2 from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-9b24ad6f078449b6a6418a331b6099f2
W20260709 12:29:51.484329  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.498533 17486 ts_tablet_manager.cc:933] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:51.511005  7822 tablet_copy_source_session.cc:215] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:51.524590  7731 raft_consensus.cc:1275] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:51.494997 17466 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:29:51.547454 17479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b24ad6f078449b6a6418a331b6099f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.551349 17451 consensus_queue.cc:1048] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:29:51.559249  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.564284  7554 catalog_manager.cc:5697] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 29 to 30, leader changed from be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 30 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:51.606320 17486 tablet_copy_client.cc:323] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.195:37801
W20260709 12:29:51.608420  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 131269us
I20260709 12:29:51.630097  7803 raft_consensus.cc:1275] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:29:51.649283 17431 consensus_queue.cc:1048] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:29:51.716591  7702 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:51.717509  7702 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:51.718070  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:51.720417  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:51.726231  7700 consensus_peers.cc:597] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487): Couldn't send request to peer 992d6df95ef74f7f8b12534534a45501. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08777b07fa9d4993a24c05bb4e2db3c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.728912 17479 tablet_copy_client.cc:806] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:51.729821  7822 tablet_copy_service.cc:140] P d72176ccaed049958ac68f706d4f052d: Received BeginTabletCopySession request for tablet a0adec428658479f82eeacfa4eb81d5e from peer 992d6df95ef74f7f8b12534534a45501 ({username='slave'} at 127.0.0.1:40506)
I20260709 12:29:51.758932  7822 tablet_copy_service.cc:161] P d72176ccaed049958ac68f706d4f052d: Beginning new tablet copy session on tablet a0adec428658479f82eeacfa4eb81d5e from peer 992d6df95ef74f7f8b12534534a45501 at {username='slave'} at 127.0.0.1:40506: session id = 992d6df95ef74f7f8b12534534a45501-a0adec428658479f82eeacfa4eb81d5e
I20260709 12:29:51.758962 17479 tablet_copy_client.cc:670] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:51.768953  7822 tablet_copy_source_session.cc:215] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:51.787990 17431 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:51.796821  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 48303.1us
W20260709 12:29:51.803544  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 86055.9us
W20260709 12:29:51.804369  7702 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:51.805282  7702 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:51.806731  7702 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:51.832705  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
W20260709 12:29:51.822389  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:51.859912 17486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0adec428658479f82eeacfa4eb81d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.871506 17431 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:29:51.888700  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 68107us
W20260709 12:29:51.889658  7702 consensus_peers.cc:597] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:51.890931  7702 consensus_peers.cc:597] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.944705  7805 raft_consensus.cc:1275] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:29:51.944008 17431 raft_consensus.cc:2955] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
I20260709 12:29:51.949179  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
W20260709 12:29:51.964707  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:51.970286  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:51.978497  7805 raft_consensus.cc:2955] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:29:51.991142  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.000924  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.007544  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.037883 17479 tablet_copy_client.cc:538] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:52.044325 17448 consensus_queue.cc:1048] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.013s
I20260709 12:29:52.074419 17486 tablet_copy_client.cc:806] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:52.094806  6451 tablet_copy_service.cc:362] P fc963afbbf4c4b34b161448ef79bd2ed: Destroying tablet copy session 992d6df95ef74f7f8b12534534a45501-1b1e442a1beb4dc0bb3b6aef6d503ea5 due to service shutdown
I20260709 12:29:52.095381  6451 tablet_copy_service.cc:434] P fc963afbbf4c4b34b161448ef79bd2ed: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-1b1e442a1beb4dc0bb3b6aef6d503ea5 on tablet 1b1e442a1beb4dc0bb3b6aef6d503ea5 with peer 992d6df95ef74f7f8b12534534a45501
W20260709 12:29:52.099920  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.106371 17486 tablet_copy_client.cc:670] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:52.131770  7559 catalog_manager.cc:5697] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index -1 to 14, NON_VOTER 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) added. New cstate: current_term: 21 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } }
W20260709 12:29:52.164604  7702 consensus_peers.cc:597] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:52.181516  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.184998 17498 ts_tablet_manager.cc:933] T 08777b07fa9d4993a24c05bb4e2db3c8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:52.196771 17479 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:29:52.197640 17466 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=1 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.198824 17466 tablet_bootstrap.cc:492] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
W20260709 12:29:52.214840  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.226917  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.217312 17466 ts_tablet_manager.cc:1403] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.723s	user 0.061s	sys 0.017s
W20260709 12:29:52.230393  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:52.235881 17486 tablet_copy_client.cc:538] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:29:52.245447  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:52.280777 17498 tablet_copy_client.cc:323] T 08777b07fa9d4993a24c05bb4e2db3c8 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Beginning tablet copy session from remote peer at address 127.6.76.194:35283
W20260709 12:29:52.287952  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.305514  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.242691 17466 raft_consensus.cc:359] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:52.312136 17486 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Bootstrap starting.
I20260709 12:29:52.309075 17466 raft_consensus.cc:740] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
W20260709 12:29:52.320441  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.343207  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.344412  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.345067  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.346508  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.338531 17466 consensus_queue.cc:260] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:52.354720 17466 ts_tablet_manager.cc:1434] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.002s
W20260709 12:29:52.347114  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.365569  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:52.377064  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.378196  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.378932  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.383426  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.388846  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.395741  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.398023  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:52.415791  8414 raft_consensus.cc:1217] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.15->[24.16-24.16]   Dedup: 24.16->[]
W20260709 12:29:52.460001  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.460834  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.483026  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.491576  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:52.484216  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.495930  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.506515  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.507537  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.508147  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.516075  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.547732  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:52.568920  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.586215  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.588436  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.592097  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.636668  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:52.665488  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:52.752471 17431 raft_consensus.cc:493] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:52.761336 17431 raft_consensus.cc:515] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:52.764226 17431 leader_election.cc:290] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:52.771739  7702 leader_election.cc:336] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:52.775817  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674d58a219c247d091f8a10ca4b75af1" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:52.829454  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:52.834375  6451 tablet_replica.cc:333] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:52.835644  6451 raft_consensus.cc:2243] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Raft consensus shutting down.
W20260709 12:29:52.842588  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:52.844106  7702 consensus_peers.cc:597] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:52.870642  6451 raft_consensus.cc:2272] T 575c307765244f48890311159c0ee934 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:52.874867  7700 leader_election.cc:304] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:52.880493 17456 raft_consensus.cc:2749] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:52.887063  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:52.900592 17496 raft_consensus.cc:493] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:52.901432 17496 raft_consensus.cc:515] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:52.905443 17496 leader_election.cc:290] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:52.922508  7702 leader_election.cc:336] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:52.930842 17496 raft_consensus.cc:493] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:52.931566 17496 raft_consensus.cc:515] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:52.935199  6451 tablet_replica.cc:333] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:52.936396  6451 raft_consensus.cc:2243] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:29:52.937793  6451 raft_consensus.cc:2272] T b3b8edd0c3f4472bb922dba093edf25a P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:52.949791  7702 leader_election.cc:336] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:52.951678  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:52.952768 17496 leader_election.cc:290] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:52.953059  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e34ef3d78ea4d53847d21b92b07fab2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:52.961720  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:52.966466  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:52.967157  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:52.980093  6451 tablet_replica.cc:333] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:52.981467  6451 raft_consensus.cc:2243] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:29:52.988865 17479 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.990103 17479 tablet_bootstrap.cc:492] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:29:52.991088 17479 ts_tablet_manager.cc:1403] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.795s	user 0.053s	sys 0.028s
I20260709 12:29:52.995815 17479 raft_consensus.cc:359] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:52.996961 17479 raft_consensus.cc:740] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:29:52.999651  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbbae2856e41420d91e3ccfbf507eacf" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.000676  6451 raft_consensus.cc:2272] T 08777b07fa9d4993a24c05bb4e2db3c8 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.011462 17479 consensus_queue.cc:260] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:53.018785 17479 ts_tablet_manager.cc:1434] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 12:29:53.020882  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-9b24ad6f078449b6a6418a331b6099f2 received from {username='slave'} at 127.0.0.1:40506
I20260709 12:29:53.021430  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-9b24ad6f078449b6a6418a331b6099f2 on tablet 9b24ad6f078449b6a6418a331b6099f2 with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:53.023617  6451 tablet_replica.cc:333] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.024684  6451 raft_consensus.cc:2243] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.030952 17431 raft_consensus.cc:493] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.031708 17431 raft_consensus.cc:515] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.034432 17431 leader_election.cc:290] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.035058 17431 raft_consensus.cc:493] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.043555 17431 raft_consensus.cc:515] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.045888 17431 leader_election.cc:290] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:53.061525  7700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 57422.7us
I20260709 12:29:53.062669  7700 leader_election.cc:304] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:53.049141 17496 raft_consensus.cc:1064] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:29:53.048115  6451 raft_consensus.cc:2272] T 8987ec43c27c4d578d5ad1e143553334 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.066699 17496 consensus_queue.cc:237] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.068528  6451 tablet_replica.cc:333] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.072600  6451 raft_consensus.cc:2243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.073662  6451 raft_consensus.cc:2272] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.078652  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d1255be1b348739b8310c8f1a903d2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.081202 17456 raft_consensus.cc:493] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.082204 17456 raft_consensus.cc:515] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.084878 17456 leader_election.cc:290] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:29:53.087083  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 59424.3us
I20260709 12:29:53.090071  6451 tablet_replica.cc:333] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.091540  6451 raft_consensus.cc:2243] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.092903  6451 raft_consensus.cc:2272] T 248d73a0462c48fdad073c597b243806 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.096505  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268874518d1d4544a8500be048628b67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.098795 17456 raft_consensus.cc:2749] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:53.101012  7700 leader_election.cc:304] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:53.102003 17456 raft_consensus.cc:2749] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:53.103700 17456 raft_consensus.cc:493] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.104352 17456 raft_consensus.cc:515] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.105283  6451 tablet_replica.cc:333] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.106283  6451 raft_consensus.cc:2243] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.106762 17456 leader_election.cc:290] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.107134  6451 raft_consensus.cc:2272] T 16eac2fd2da54bec8a02bb35cc376087 P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.110466  6451 tablet_replica.cc:333] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.116508 17486 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:53.123528  6451 raft_consensus.cc:2243] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.123631  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3960664d9eb541f185c9eac4a231e5c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.143345 17514 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.146176 17514 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.144820 17486 tablet_bootstrap.cc:492] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Bootstrap complete.
I20260709 12:29:53.150040  6451 raft_consensus.cc:2272] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.151664  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d9b95a9cf4fab8305b579485c2332" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.153302 17486 ts_tablet_manager.cc:1403] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Time spent bootstrapping tablet: real 0.850s	user 0.079s	sys 0.034s
I20260709 12:29:53.157394 17515 raft_consensus.cc:493] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.158188 17515 raft_consensus.cc:515] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.158859  6451 tablet_replica.cc:333] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.159909  6451 raft_consensus.cc:2243] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.160600  6451 raft_consensus.cc:2272] T 35934889fce04bc093f07a0f03c7165c P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.162390 17514 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.172024 17486 raft_consensus.cc:359] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:53.175179  6451 tablet_replica.cc:333] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.176465  6451 raft_consensus.cc:2243] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:53.166906  7702 leader_election.cc:336] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:53.170531  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:53.179687  7702 leader_election.cc:304] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.184194  7702 leader_election.cc:336] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.185138 17514 raft_consensus.cc:2749] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:53.203509  7702 leader_election.cc:304] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.204494 17416 raft_consensus.cc:2749] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:53.173548 17486 raft_consensus.cc:740] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Initialized, Role: LEARNER
I20260709 12:29:53.205780 17486 consensus_queue.cc:260] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:53.176952 17515 leader_election.cc:290] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.195725  6451 raft_consensus.cc:2272] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.232378  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.234530 17486 ts_tablet_manager.cc:1434] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Time spent starting tablet: real 0.081s	user 0.009s	sys 0.000s
I20260709 12:29:53.238067 17514 raft_consensus.cc:493] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.238750 17514 raft_consensus.cc:515] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.238083 17416 raft_consensus.cc:493] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.240706 17416 raft_consensus.cc:515] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.241269 17514 leader_election.cc:290] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.241964 17514 raft_consensus.cc:493] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.242511 17514 raft_consensus.cc:515] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.250540 17416 leader_election.cc:290] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.253268  7822 tablet_copy_service.cc:342] P d72176ccaed049958ac68f706d4f052d: Request end of tablet copy session 992d6df95ef74f7f8b12534534a45501-a0adec428658479f82eeacfa4eb81d5e received from {username='slave'} at 127.0.0.1:40506
I20260709 12:29:53.253805  7822 tablet_copy_service.cc:434] P d72176ccaed049958ac68f706d4f052d: ending tablet copy session 992d6df95ef74f7f8b12534534a45501-a0adec428658479f82eeacfa4eb81d5e on tablet a0adec428658479f82eeacfa4eb81d5e with peer 992d6df95ef74f7f8b12534534a45501
I20260709 12:29:53.255578 17514 leader_election.cc:290] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.255671  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741ccb73f0c4316be6682db332fd98e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.257582  8412 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 LEARNER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:29:53.260902  7805 raft_consensus.cc:1275] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Refusing update from remote peer fc963afbbf4c4b34b161448ef79bd2ed: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:29:53.270726  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756aa7b7913e49aaba2a2164e45553fe" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.277911  6451 tablet_replica.cc:333] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.279197  6451 raft_consensus.cc:2243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.280881  6451 raft_consensus.cc:2272] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.288744  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:53.304453  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b69d30f987e646bfbea975a362ad70ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.337568  6451 tablet_replica.cc:333] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.339074  6451 raft_consensus.cc:2243] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.341136  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bf041bf77540baa50b5319733c5e9e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:53.344041  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:53.344244  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.347612  8414 raft_consensus.cc:1217] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:29:53.345264 17515 raft_consensus.cc:493] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.349200 17515 raft_consensus.cc:515] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.351807 17515 leader_election.cc:290] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.345009 17514 raft_consensus.cc:493] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.352675 17514 raft_consensus.cc:515] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.360386 17514 leader_election.cc:290] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.357292 17431 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.361172 17416 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.364588 17431 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:53.364691 17416 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.366438 17431 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.361704 17515 raft_consensus.cc:493] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.376829 17515 raft_consensus.cc:515] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.362185 17496 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.379735 17496 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.382478 17496 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:53.374401 17456 raft_consensus.cc:670] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:29:53.390767  6451 raft_consensus.cc:2272] T 1e2e2ecc360e4304b9a841fbd6b8287e P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.396556 17416 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:53.397635  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.403112  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9652df6fb248539296e0b903cc4d1c" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.405320  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.405784  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.411168 17431 raft_consensus.cc:493] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.412256 17431 raft_consensus.cc:515] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.412532 17416 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:53.389118 17514 raft_consensus.cc:493] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.414986 17431 leader_election.cc:290] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.415700 17514 raft_consensus.cc:515] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:53.418015  7702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 44210.9us
I20260709 12:29:53.402848 17515 leader_election.cc:290] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.420476 17456 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.423784 17456 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.426317 17456 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.428963 17515 raft_consensus.cc:493] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.429932 17416 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.429773 17515 raft_consensus.cc:515] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.430478 17416 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.432282 17515 leader_election.cc:290] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.429140 17514 leader_election.cc:290] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.435858 17514 raft_consensus.cc:493] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.437229 17514 raft_consensus.cc:515] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.421182  7702 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.440477 17456 raft_consensus.cc:493] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.441063 17456 raft_consensus.cc:515] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.438205 17515 raft_consensus.cc:493] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.442619 17515 raft_consensus.cc:515] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.445091  6451 tablet_replica.cc:333] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.432674 17416 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.438735 17496 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.446184  6451 raft_consensus.cc:2243] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.445797 17518 consensus_queue.cc:1048] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:53.447336  6451 raft_consensus.cc:2272] T 674d58a219c247d091f8a10ca4b75af1 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.446022 17496 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:53.443980 17431 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:53.443480 17456 leader_election.cc:290] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:53.454752  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:53.436533  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:53.456966  6451 tablet_replica.cc:333] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.458007  6451 raft_consensus.cc:2243] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.458706  6451 raft_consensus.cc:2272] T 0741ccb73f0c4316be6682db332fd98e P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.453612  7702 leader_election.cc:336] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.456771 17515 leader_election.cc:290] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.460037  7702 leader_election.cc:304] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.465852  6451 tablet_replica.cc:333] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.466881  6451 raft_consensus.cc:2243] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.469964 17496 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.470798 17514 leader_election.cc:290] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:53.472858  7702 leader_election.cc:336] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.473449  7702 leader_election.cc:304] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.475483 17514 raft_consensus.cc:2749] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:53.486256  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.487628  8409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0733d7d23d9041e09a5ade18807c383e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.489408  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:53.493402  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.485644  6451 raft_consensus.cc:2272] T cbbae2856e41420d91e3ccfbf507eacf P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.496374  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:53.497534  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:53.504612  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:53.509577  7702 raft_consensus.cc:2686] T 100d9b95a9cf4fab8305b579485c2332 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.512749  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.519369  6451 tablet_replica.cc:333] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.520525  6451 raft_consensus.cc:2243] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.521441  6451 raft_consensus.cc:2272] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.548022  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691f0d885861432f91375d5f288af312" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.551853  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "539d703b24874183847fa9a8889111e9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.553946  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0adec428658479f82eeacfa4eb81d5e" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.558494  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d96c92c58a4ad683334c5afc8e9a67" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.558910  6451 tablet_replica.cc:333] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:53.559536  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.560173  6451 raft_consensus.cc:2243] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.561230  6451 raft_consensus.cc:2272] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.562626  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c84f7f7e484afbae3c398efd5924b4" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.564934  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629e345c827545598250d02a9f39e2fb" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.566850  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a5afed00934a1d9878dd900b236263" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.567189  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6563fdb577f64184ac4e28f7c43e8c30" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.567720 17517 raft_consensus.cc:493] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.570277  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33cd795fd74820b3adafb3cdbc07c9" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.570963 17517 raft_consensus.cc:515] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.573783 17517 leader_election.cc:290] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.576287  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.581350  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116499cb3f0a4b4eba7b2ace2817bb25" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
W20260709 12:29:53.604768  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.617322 17518 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.618278 17518 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:53.620617 17518 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:53.624821  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.627063  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd837ed3845462283f72c3010de59c3" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.627388  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5cde7dcc814e5d90f0b8e433f8f452" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.617266 17514 raft_consensus.cc:493] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:29:53.628648 17514 raft_consensus.cc:515] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:53.632431  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a2203ee13714c8bacacb6f83ec31034" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.640091  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b24ad6f078449b6a6418a331b6099f2" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.643563  8414 raft_consensus.cc:1217] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
I20260709 12:29:53.644972  6451 tablet_replica.cc:333] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.646986  6451 raft_consensus.cc:2243] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.656700  6451 raft_consensus.cc:2272] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.658993  7702 consensus_peers.cc:597] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:53.666884  7702 leader_election.cc:336] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.667598  7702 leader_election.cc:304] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.668430  7702 raft_consensus.cc:2686] T 756aa7b7913e49aaba2a2164e45553fe P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.669925  6451 tablet_replica.cc:333] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.673931  6451 raft_consensus.cc:2243] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.697944 17517 raft_consensus.cc:493] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 992d6df95ef74f7f8b12534534a45501)
I20260709 12:29:53.698907 17517 raft_consensus.cc:515] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:53.672550  7702 leader_election.cc:336] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.656783 17514 leader_election.cc:290] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:53.701683  7702 leader_election.cc:304] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.702586 17514 raft_consensus.cc:2749] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:53.703074 17517 leader_election.cc:290] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:53.703783  7702 leader_election.cc:336] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.704816  7702 leader_election.cc:304] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:53.704375  6451 raft_consensus.cc:2272] T d353e70329004cc6b0c1a6bcda38e5c9 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.710461 17517 raft_consensus.cc:2749] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:53.714731  7702 leader_election.cc:336] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.718324  6451 tablet_replica.cc:333] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:53.720853  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.716684  7702 leader_election.cc:304] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.721851  7702 raft_consensus.cc:2686] T 38bf041bf77540baa50b5319733c5e9e P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.724444  6451 raft_consensus.cc:2243] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.732633  6451 raft_consensus.cc:2272] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.748198  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b27e8abd04f2ea19a70a13d1958ec" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.753211  6451 tablet_replica.cc:333] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.760500  6451 raft_consensus.cc:2243] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.765161  6451 raft_consensus.cc:2272] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.768958  6451 tablet_replica.cc:333] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.758893  7702 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.769812  8414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ad8f196c145088ee9ca42482e70ce" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:53.770277 17517 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:53.773394  7702 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.774297 17517 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:53.775905  7702 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.776648  7702 raft_consensus.cc:2686] T 39c84f7f7e484afbae3c398efd5924b4 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.777844  7702 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.779204  6451 raft_consensus.cc:2243] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.779865 17517 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:53.781703  6451 raft_consensus.cc:2272] T 3960664d9eb541f185c9eac4a231e5c3 P fc963afbbf4c4b34b161448ef79bd2ed [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.784819  7702 leader_election.cc:304] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.785785  7702 raft_consensus.cc:2686] T ccd837ed3845462283f72c3010de59c3 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.788944  6451 tablet_replica.cc:333] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:53.789819  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.790431  7702 leader_election.cc:304] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.791465 17517 raft_consensus.cc:2749] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:53.794714  6451 raft_consensus.cc:2243] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.799422 17456 raft_consensus.cc:2955] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
W20260709 12:29:53.805091  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:53.806207  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:53.803960  7702 leader_election.cc:304] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.816051  6451 raft_consensus.cc:2272] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.817471  7702 raft_consensus.cc:2686] T 3e5b27e8abd04f2ea19a70a13d1958ec P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:53.819824  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463898dea61f4d7ab42c74b60beb15e6" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.829975  6451 tablet_replica.cc:333] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.834435  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189237b70e7f43458ac90a5773fd551f" candidate_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:53.851716  6451 raft_consensus.cc:2243] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.856526  7812 raft_consensus.cc:2955] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:53.858382  6451 raft_consensus.cc:2272] T 7e34ef3d78ea4d53847d21b92b07fab2 P fc963afbbf4c4b34b161448ef79bd2ed [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.863611  7702 leader_election.cc:336] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.865064  7702 leader_election.cc:304] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.868537  7702 raft_consensus.cc:2686] T 691f0d885861432f91375d5f288af312 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:29:53.869491  7702 leader_election.cc:336] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.870083  7702 leader_election.cc:304] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.867591  6451 tablet_replica.cc:333] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.871709  6451 raft_consensus.cc:2243] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.872274 17456 raft_consensus.cc:2749] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:53.874905  6451 raft_consensus.cc:2272] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.873116  7702 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.876011  7702 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.876997  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:53.877696  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:53.882881 17456 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:53.884837  7702 leader_election.cc:336] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.889518  7702 leader_election.cc:304] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.904635 17456 raft_consensus.cc:2749] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:53.915378  6451 tablet_replica.cc:333] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.931883  6451 raft_consensus.cc:2243] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.933056  6451 raft_consensus.cc:2272] T 539d703b24874183847fa9a8889111e9 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:53.921387  7702 consensus_peers.cc:597] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:53.935704  7702 leader_election.cc:336] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.936285  7702 leader_election.cc:304] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.937384 17456 raft_consensus.cc:2749] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:53.938918  7702 leader_election.cc:336] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.939805  7702 leader_election.cc:304] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:53.940927 17456 raft_consensus.cc:2749] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:53.945021  6451 tablet_replica.cc:333] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:53.949760  6451 raft_consensus.cc:2243] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:53.948350  7702 leader_election.cc:336] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.950528  7702 leader_election.cc:304] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.952785  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 57780.6us
W20260709 12:29:53.953851  7702 raft_consensus.cc:2686] T fd33cd795fd74820b3adafb3cdbc07c9 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:29:53.954831  7702 leader_election.cc:336] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.955442  7702 leader_election.cc:304] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.956817 17456 raft_consensus.cc:2749] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:53.959744  6451 raft_consensus.cc:2272] T 629e345c827545598250d02a9f39e2fb P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.963454  6451 tablet_replica.cc:333] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:53.964007  7702 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.964954  7702 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
W20260709 12:29:53.965747  7702 raft_consensus.cc:2686] T 9b24ad6f078449b6a6418a331b6099f2 P fc963afbbf4c4b34b161448ef79bd2ed: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:29:53.966593  7702 consensus_peers.cc:597] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:53.970768  7702 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.964490  6451 raft_consensus.cc:2243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.972504 17456 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:53.975239  6451 raft_consensus.cc:2272] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.982012  6451 tablet_replica.cc:333] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:53.991926  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:53.990235  7702 leader_election.cc:336] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.993750  7702 leader_election.cc:304] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:29:53.995002 17456 raft_consensus.cc:2749] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:53.996459  7702 leader_election.cc:336] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:53.996992  7702 leader_election.cc:304] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc963afbbf4c4b34b161448ef79bd2ed; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:53.991675  6451 raft_consensus.cc:2243] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.998356 17456 raft_consensus.cc:2749] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:53.998450  7554 catalog_manager.cc:5697] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d reported cstate change: config changed from index 16 to 17, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:54.000212  6451 raft_consensus.cc:2272] T e8d96c92c58a4ad683334c5afc8e9a67 P fc963afbbf4c4b34b161448ef79bd2ed [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.008651  6451 tablet_replica.cc:333] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.009690  6451 raft_consensus.cc:2243] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.012817  6451 raft_consensus.cc:2272] T 41a5afed00934a1d9878dd900b236263 P fc963afbbf4c4b34b161448ef79bd2ed [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.015139  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:29:54.017699  8414 raft_consensus.cc:2955] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 992d6df95ef74f7f8b12534534a45501 (127.6.76.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } }
I20260709 12:29:54.045193 17509 raft_consensus.cc:1064] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:29:54.048833 17509 consensus_queue.cc:237] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:54.063201  6451 tablet_replica.cc:333] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
W20260709 12:29:54.064015  7702 consensus_peers.cc:597] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.064754  6451 raft_consensus.cc:2243] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.074391  6451 raft_consensus.cc:2272] T d9c7a755001445e2895ad66ede4c8f60 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.078641  6451 tablet_replica.cc:333] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.079627  6451 raft_consensus.cc:2243] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.080371  6451 raft_consensus.cc:2272] T db9652df6fb248539296e0b903cc4d1c P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.085006  6451 tablet_replica.cc:333] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.085852  6451 raft_consensus.cc:2243] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.086568  6451 raft_consensus.cc:2272] T 0733d7d23d9041e09a5ade18807c383e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.087400  8379 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:37801, user_credentials={real_user=slave}} blocked reactor thread for 60424.8us
I20260709 12:29:54.089788  6451 tablet_replica.cc:333] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.090751  6451 raft_consensus.cc:2243] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.091792  6451 raft_consensus.cc:2272] T 463898dea61f4d7ab42c74b60beb15e6 P fc963afbbf4c4b34b161448ef79bd2ed [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.095089  6451 tablet_replica.cc:333] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.096352  6451 raft_consensus.cc:2243] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.097707  6451 raft_consensus.cc:2272] T 0a2203ee13714c8bacacb6f83ec31034 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.102511  6451 tablet_replica.cc:333] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.103554  6451 raft_consensus.cc:2243] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.105242  6451 raft_consensus.cc:2272] T 402d5cf4f7d24ac59dee1221fc2c7a3f P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.108770  6451 tablet_replica.cc:333] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.109874  6451 raft_consensus.cc:2243] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.110667  6451 raft_consensus.cc:2272] T b69d30f987e646bfbea975a362ad70ec P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.114599  6451 tablet_replica.cc:333] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.115882  6451 raft_consensus.cc:2243] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:29:54.117450  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.117877  6451 raft_consensus.cc:2272] T 24790f3be4e8465581029a2e0d526117 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.118988  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:54.121580  6451 tablet_replica.cc:333] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.124941  6451 raft_consensus.cc:2243] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.126299  6451 raft_consensus.cc:2272] T 6b5cde7dcc814e5d90f0b8e433f8f452 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.131145  8412 raft_consensus.cc:1275] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:29:54.134662  6451 tablet_replica.cc:333] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.135821  6451 raft_consensus.cc:2243] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.136953  6451 raft_consensus.cc:2272] T 268874518d1d4544a8500be048628b67 P fc963afbbf4c4b34b161448ef79bd2ed [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.140138 17513 consensus_queue.cc:1048] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:54.151165  6451 tablet_replica.cc:333] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.152798  6451 raft_consensus.cc:2243] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.164330  6451 raft_consensus.cc:2272] T e78ad8f196c145088ee9ca42482e70ce P fc963afbbf4c4b34b161448ef79bd2ed [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.166023  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.168334  6451 tablet_replica.cc:333] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.169437  6451 raft_consensus.cc:2243] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.170449  6451 raft_consensus.cc:2272] T 116499cb3f0a4b4eba7b2ace2817bb25 P fc963afbbf4c4b34b161448ef79bd2ed [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.180038  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 54220.5us
I20260709 12:29:54.175117  6451 tablet_replica.cc:333] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.185838  6451 raft_consensus.cc:2243] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.186848  6451 raft_consensus.cc:2272] T a0adec428658479f82eeacfa4eb81d5e P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.205477  6451 tablet_replica.cc:333] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.206579  6451 raft_consensus.cc:2243] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.208024  6451 raft_consensus.cc:2272] T a3d64be480d9430189f5e290bab7c3b8 P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.211133  6451 tablet_replica.cc:333] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.211994  6451 raft_consensus.cc:2243] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.213677  6451 raft_consensus.cc:2272] T 7112151d65374f6690a117439cc2dc33 P fc963afbbf4c4b34b161448ef79bd2ed [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.217510  6451 tablet_replica.cc:333] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.218431  6451 raft_consensus.cc:2243] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.219453  6451 raft_consensus.cc:2272] T 189237b70e7f43458ac90a5773fd551f P fc963afbbf4c4b34b161448ef79bd2ed [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.222859  6451 tablet_replica.cc:333] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.223744  6451 raft_consensus.cc:2243] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.224547  6451 raft_consensus.cc:2272] T 6563fdb577f64184ac4e28f7c43e8c30 P fc963afbbf4c4b34b161448ef79bd2ed [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.231446  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:54.252198  7775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:36487, user_credentials={real_user=slave}} blocked reactor thread for 51287.9us
W20260709 12:29:54.267547  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:54.268292  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.271265  6451 tablet_replica.cc:333] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed: stopping tablet replica
I20260709 12:29:54.272176  6451 raft_consensus.cc:2243] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.273026  6451 raft_consensus.cc:2272] T 62d1255be1b348739b8310c8f1a903d2 P fc963afbbf4c4b34b161448ef79bd2ed [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.281268  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.282003 17513 raft_consensus.cc:1064] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: attempting to promote NON_VOTER 992d6df95ef74f7f8b12534534a45501 to VOTER
I20260709 12:29:54.284960 17513 consensus_queue.cc:237] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:54.282074  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:54.321269  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.323450 17451 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.324119 17451 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:54.326655 17451 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:54.338119  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:54.351070 17451 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.351872 17451 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:54.354622 17451 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:54.364596  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:54.378990  8413 raft_consensus.cc:1275] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: Refusing update from remote peer d72176ccaed049958ac68f706d4f052d: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:29:54.385129 17529 consensus_queue.cc:1048] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [LEADER]: Connected to new peer: Peer: permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:29:54.402019  7778 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:54.402612  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:54.403612 17513 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:54.412312  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.409711  7778 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:54.422374  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:54.423389 17451 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:54.441712  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.464113  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.466650  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.474901 17513 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.475721 17513 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:54.478317 17513 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:54.483882  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.486856  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:54.490705  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:54.491371  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:54.492295 17451 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:54.513707  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.541705  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:54.557603  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.586068  7701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:54.586985  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.596427  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.640301  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.661880 17451 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.662699 17451 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:54.665519 17451 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
I20260709 12:29:54.672209  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:29:54.674904  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.693385  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.704528  7778 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:54.705631  7778 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:54.707940 17451 raft_consensus.cc:2749] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:54.718227  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.741641  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.742381  7778 connection.cc:573] client connection to 127.6.76.194:35283 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:29:54.748927  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.755168 17522 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.756053 17522 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:54.765059  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.769166  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:54.770044 17522 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:54.770063  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 29.
I20260709 12:29:54.772586  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: 
I20260709 12:29:54.774009 17522 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:54.774699 17522 raft_consensus.cc:493] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting leader election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:54.775194 17522 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:29:54.781703  8380 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:54.794371  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.805449  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:54.808555 17522 raft_consensus.cc:515] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:54.812489 17522 leader_election.cc:290] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: Requested vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:54.812696  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f6a2ef5ca4422b882b27d3d2ad1f36" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d"
I20260709 12:29:54.813516  7803 raft_consensus.cc:3060] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:54.815652 17498 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40608 (request call id 1391) took 2735 ms. Trace:
I20260709 12:29:54.816188 17498 rpcz_store.cc:276] 0709 12:29:52.079862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:52.080042 (+   180us) service_pool.cc:224] Handling call
0709 12:29:52.185470 (+105428us) ts_tablet_manager.cc:934] T 08777b07fa9d4993a24c05bb4e2db3c8 P 992d6df95ef74f7f8b12534534a45501: Initiating tablet copy from peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
0709 12:29:54.815543 (+2630073us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":321,"tablet-copy.queue_time_us":93165,"thread_start_us":156178,"threads_started":2}
W20260709 12:29:54.818295  8379 connection.cc:466] server connection from 127.0.0.1:40608 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40608 (request call id 1391) could send its response
W20260709 12:29:54.824179  8380 leader_election.cc:336] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:54.830036  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.832055  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.836380  7803 raft_consensus.cc:2468] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 30.
I20260709 12:29:54.838270  8379 leader_election.cc:304] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:54.839723 17522 raft_consensus.cc:2804] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:54.852955 17522 raft_consensus.cc:697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 30 LEADER]: Becoming Leader. State: Replica: 992d6df95ef74f7f8b12534534a45501, State: Running, Role: LEADER
I20260709 12:29:54.855760 17522 consensus_queue.cc:237] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:54.866763  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.883162  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.890472  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.891003  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:54.892216  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:54.906013  7554 catalog_manager.cc:5697] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 reported cstate change: term changed from 29 to 30, leader changed from fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194) to 992d6df95ef74f7f8b12534534a45501 (127.6.76.196). New cstate: current_term: 30 leader_uuid: "992d6df95ef74f7f8b12534534a45501" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:54.925814  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.948947  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.954639  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.977528  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.978184  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.984863  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.985510  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:54.995733  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:55.002475  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.003165  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.047312  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:55.053974  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.065346  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:55.075129 17423 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.075970 17423 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:55.078774 17423 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:55.085029  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.089852  8380 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:55.090859  8379 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:55.091383  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.094161 17423 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:55.101006  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:55.138335 17529 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.138929 17529 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:55.141276 17529 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:55.154551  7778 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:55.158527  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:55.159056  7777 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.166050 17541 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:55.202193  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:55.317415  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:55.318516  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:55.351181 17541 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.351869 17541 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:55.354321 17541 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:55.364885  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:55.367076  7778 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:55.367602  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.368517 17513 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.369079 17513 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:55.373454  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:29:55.375286 17529 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:55.385646 17513 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:29:55.398314  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:55.398742  7812 raft_consensus.cc:1275] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Refusing update from remote peer 992d6df95ef74f7f8b12534534a45501: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:29:55.401031 17423 consensus_queue.cc:1048] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:29:55.403052  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:55.403650  7777 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: 992d6df95ef74f7f8b12534534a45501, be9c043c6d424ba1ab8def55dc9bb3e2
I20260709 12:29:55.405707 17541 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:55.407570  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:55.408751  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:55.466068  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:55.488294 17466 tablet_copy_client.cc:1131] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:55.518350 17423 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.519024 17423 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:55.521800 17423 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:55.530954  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:55.531930  7812 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:29:55.531268  8380 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:55.537997  8379 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:55.538548  8379 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.542670 17423 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:55.564847 17529 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.565443 17529 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:55.568085 17529 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:55.582476  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.605770  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:55.606416  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:55.607115  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:55.629812  7778 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:55.630599 17423 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.632279 17423 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:55.634663 17423 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:29:55.635545  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.643901 17513 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:55.645381  8380 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:55.648100  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:55.648581  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.649571 17522 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:55.655090  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:55.659060  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:55.669294 17513 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:55.669982 17513 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:55.673023 17513 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:55.732816  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:55.734287  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:55.732956  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:55.755862  7778 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:55.758342  7778 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:55.759191 17513 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:55.793169  6451 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 7758)
W20260709 12:29:55.760911  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:56.002868 17513 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:56.004498 17513 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:56.023049 17513 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:56.040359  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:56.060873  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:56.063959  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:56.079820 17541 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:56.075701  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.110260  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.125447  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:56.142350  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.162508  8379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111) [suppressed 777 similar messages]
W20260709 12:29:56.192812  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:56.211788 17451 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:56.212788 17451 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:56.215647 17451 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:56.224339  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:56.233309  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:56.254630  7778 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:56.264349  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:56.259256  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:56.272001 17451 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:56.323120  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:56.382915  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.387427  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:56.456018  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:56.539640  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:56.619087 17451 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:56.620072 17451 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:56.623093 17451 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:56.632956  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:56.690793  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.710726  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.755084  7778 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:56.758849  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:56.756708  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:56.766256  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:56.772267  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:56.788445 17451 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:56.795285  6451 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 7758)
W20260709 12:29:56.822402  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:56.879863  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:56.858834  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:29:56.897603 17451 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:56.900328 17451 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:56.898715  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:56.907775 17451 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:56.911531  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:56.927482  7778 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:56.928035  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:56.928905 17548 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:56.958076  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:56.968161  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:56.998478  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.003772  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:57.007995  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:57.009987  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.068401  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.078652  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.096858  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.132566  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:57.143944  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.154410  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.184113  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.190717  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.205070  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.219923 17423 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:57.221048 17423 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:57.229053  8380 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:57.233260 17423 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:57.242813  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.246347 17529 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:57.247242 17529 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:29:57.256074  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:57.257759 17529 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:57.259514  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.261502 17545 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
W20260709 12:29:57.262887  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.262542 17545 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:57.263778 17529 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:57.264473 17529 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:57.266578 17545 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:29:57.278120 17529 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:57.279590  8380 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:57.280117  8379 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:57.281626  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.283003 17545 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:57.283912  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:57.288543  7778 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:57.289100  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.289919 17529 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:57.290592  7812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:29:57.291745  7812 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:29:57.305260  8379 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:57.305892  8379 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.308926 17423 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:57.310315  7778 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:57.310899  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.311859 17529 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:57.320372  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.339179  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.339766  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.340215  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.345566 17423 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:57.346387 17423 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:57.354388  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.355165  8380 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:57.357565  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:57.361004 17423 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:57.362198  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:57.363166  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.364670 17423 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:29:57.398675  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.403281  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.409806  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.437752  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.444324  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:57.463507  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:57.477032 17451 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:57.477777 17451 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:57.480034 17451 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:57.482107  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.486608  7778 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:57.504127  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:57.504637  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:57.505460 17451 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:57.519995  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.535094  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.552644  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.600674  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.611152  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.611865  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.621403  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:57.678164  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.728291  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:57.740702  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:57.760882  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:57.768947  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.777305  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:57.785681  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.796792  6451 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 7758)
W20260709 12:29:57.804816  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:57.826390  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.002099  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:58.035549  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.081326  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.113600  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:58.201193  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:58.234917  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:58.299541  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.430550  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:58.434361  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.464371  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:58.478473 17548 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:58.479346 17548 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:29:58.485351 17548 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:58.503190  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:58.505498  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.506912  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:58.507588  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:58.510890 17529 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:58.595399 17529 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:58.597674 17529 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:58.609488 17529 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:58.621217  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.650810  7778 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:58.685371  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:58.686177  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:58.688427  7777 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:58.691001 17529 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:58.696045  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.785718  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:58.797392  6451 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 7758)
W20260709 12:29:58.802860  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:58.799819  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.811460  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:58.829603  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:58.895351  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:58.919052  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:58.922853  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:59.066737  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:59.157987  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:59.202111 17545 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:59.202732 17545 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:29:59.207602 17545 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:59.211874  8380 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:59.214020  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:59.214915  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:59.234653 17529 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:59.235406 17529 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:59.235567 17545 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:29:59.237752 17529 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:59.269481  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:59.275099  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.307292  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:59.325331  7778 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:59.333079  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:59.347396 17552 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:59.353117 17551 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:59.353948 17551 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:29:59.356719  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:59.364153 17551 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:59.373055  8380 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:29:59.377588  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.396859  8379 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:59.408725  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:59.409762 17551 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:59.416878 17552 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:59.417649 17552 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:59.420401 17552 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:59.424132  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:59.444986  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:29:59.445642 17552 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:29:59.448365 17552 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:29:59.451048 17552 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:29:59.447480  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:59.454468  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.473184  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:29:59.475618  7778 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:59.476956  7778 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:29:59.484859 17552 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:29:59.490278  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:59.504172  7778 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:29:59.539188  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:29:59.527838  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:59.549872 17548 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:59.564692  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:59.587170  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 83056.4us
W20260709 12:29:59.640094  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.640869  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.643606  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.663058  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.684706  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.685300  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.685741  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.702018  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.704500  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.704907  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.747447  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:59.798122  6451 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 7758)
W20260709 12:29:59.810613  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.814605  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.827872  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.839963  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.866472  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.870873  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.873610  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.902172  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.903642  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.933430  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.936604  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:59.969558  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:59.985627  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:30:00.020413 17548 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:00.021101 17548 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:00.023705 17548 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:00.035671  7778 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:00.039759  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:00.054512  7777 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:00.055565 17552 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:30:00.066668  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:00.077464  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.079361  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.089484  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:00.101796  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.103186  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.102567  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:00.149160  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.155941  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.159971  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:30:00.164299  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.173167  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.182564  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:30:00.192741 17529 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:00.193390 17529 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:00.195704 17529 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:00.207841  7778 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:00.217106  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.232329  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:00.233022  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:00.237250 17554 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:00.251510  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:30:00.393653 17552 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:00.394591 17552 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:00.404848  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:30:00.405771  8413 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
W20260709 12:30:00.407335  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:00.423352 17552 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:30:00.424664  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.426661  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.429518  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.445142  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:00.445838  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:00.446761 17552 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:00.448225  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.468568  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.495635  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.511695  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.527513  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:00.578333  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:00.670847  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.692142  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:00.799175  6451 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 7758)
W20260709 12:30:00.878682  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.886374  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:00.914959  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:30:00.929200  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:30:00.978681 17554 raft_consensus.cc:493] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:00.979453 17554 raft_consensus.cc:515] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:30:00.986958  7777 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:00.988991 17554 leader_election.cc:290] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:00.996408  7778 leader_election.cc:336] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:01.007491  7778 leader_election.cc:304] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:01.008688 17554 raft_consensus.cc:2749] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:30:01.041390  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.135154  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:01.145996  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.162657  7777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111) [suppressed 804 similar messages]
W20260709 12:30:01.233081  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.265249  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:01.318524  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:01.326713  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:01.425591  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.471403  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.473721  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:01.508291  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.514190  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:01.533977  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:01.710233  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:01.727167  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:30:01.766029 17554 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:01.766600 17554 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:01.769039 17554 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:01.789149  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:01.791735  7778 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:01.792361  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:01.793270 17554 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:30:01.799824  6451 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 7758)
W20260709 12:30:01.838195  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:30:01.868139 17554 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:01.868965 17554 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:30:01.871781 17554 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:01.873621  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:01.898883  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:01.913919  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:01.917703  7778 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:01.920607 17554 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:30:01.922425  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.940687  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:01.950606  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:01.951705  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:01.950196  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.028484  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:02.034642  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:02.047559  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.068318  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.068697  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:30:02.104261 17557 raft_consensus.cc:493] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:02.105489 17557 raft_consensus.cc:515] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:02.109871 17557 leader_election.cc:290] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:02.086526  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.130316  8380 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:02.142782  7778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:35283, user_credentials={real_user=slave}} blocked reactor thread for 56343.9us
W20260709 12:30:02.133275  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.170313  8379 leader_election.cc:336] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:02.171165  8379 leader_election.cc:304] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:02.172319 17557 raft_consensus.cc:2749] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:30:02.258088 17556 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:02.258821 17556 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:02.263898 17548 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:02.266330 17548 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:02.274371 17548 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:30:02.269289  8379 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:02.295784  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:30:02.317940  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
W20260709 12:30:02.318408  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:30:02.318765  8413 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
W20260709 12:30:02.328701  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:02.329612  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:02.324856  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.335882  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.337216  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:02.338963  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:02.340772 17559 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:02.346937  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.362792  8380 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:02.364766  8380 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:02.366252 17545 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:30:02.368078 17556 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:02.368780  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.370139  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.380532  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.400298  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:30:02.404248  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "992d6df95ef74f7f8b12534534a45501" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d72176ccaed049958ac68f706d4f052d" is_pre_election: true
I20260709 12:30:02.405063  7805 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992d6df95ef74f7f8b12534534a45501 in term 24.
W20260709 12:30:02.406037  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.406579  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.414543  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.427423  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.457180  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:02.467260  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.501765  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.520748  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:02.565447  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.624454  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.627449  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:02.631541  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:02.649329  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:02.659992  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:30:02.675381  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.685617  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.688609  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.707101  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.720539  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:02.728453  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:02.768399  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.785756  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:02.800467  6451 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 7758)
W20260709 12:30:02.811076  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.857319  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:02.879336  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.893187  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:02.898520  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.920205  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:02.985574  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:03.091328  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:03.091988  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:03.153388  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.248720  7777 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.281669  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:03.357685  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:03.394089  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080020fc00 after lost signal to thread 6454
W20260709 12:30:03.395834  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080029a320 after lost signal to thread 7593
W20260709 12:30:03.397807  7833 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7f00 after lost signal to thread 7836
W20260709 12:30:03.407750  7833 debug-util.cc:398] Leaking SignalData structure 0x7b080019a800 after lost signal to thread 8439
W20260709 12:30:03.603262  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.670720  7777 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.688129  7777 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:30:03.692565  7777 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:03.691535  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:30:03.702801 17567 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:03.703649 17567 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:03.706583 17567 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:03.712962  8380 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:03.718673  8379 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:03.724874  8379 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:03.726455 17567 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:30:03.735975  7777 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.760778  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:03.801144  6451 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 7758)
W20260709 12:30:03.838349  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:03.869488  7778 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:03.886605  7777 consensus_peers.cc:597] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:30:03.955686  7777 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:03.958741  7777 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:04.019008  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:30:04.048641 17565 raft_consensus.cc:493] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.049484 17565 raft_consensus.cc:515] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:04.063371 17565 leader_election.cc:290] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:04.034474  7777 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:04.079810  7777 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:43519, user_credentials={real_user=slave}} blocked reactor thread for 45451us
W20260709 12:30:04.075841  7778 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:04.089664  7777 leader_election.cc:336] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:04.091609  7777 leader_election.cc:304] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:04.103734 17559 raft_consensus.cc:2749] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:04.131639  7777 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:30:04.244328 17565 raft_consensus.cc:493] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.245095 17565 raft_consensus.cc:515] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:04.248348 17565 leader_election.cc:290] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:04.290980  7778 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:04.303964  7778 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.304546  7777 leader_election.cc:336] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:04.312999  7777 leader_election.cc:304] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:04.316241 17559 raft_consensus.cc:2749] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:30:04.331694  7777 consensus_peers.cc:597] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:04.338929  7777 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:04.339550  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:04.346810  7777 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:30:04.383394 17559 raft_consensus.cc:493] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.384101 17559 raft_consensus.cc:515] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:04.386780 17559 leader_election.cc:290] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:04.393013  7777 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:04.412184  7778 leader_election.cc:336] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:04.413966  7778 leader_election.cc:304] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:30:04.413398  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:04.416429 17559 raft_consensus.cc:2749] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:30:04.432781 17565 raft_consensus.cc:493] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.433555 17565 raft_consensus.cc:515] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
W20260709 12:30:04.444283  7777 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:04.446530 17565 leader_election.cc:290] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:30:04.449699 17558 raft_consensus.cc:493] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.453338 17558 raft_consensus.cc:515] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } }
I20260709 12:30:04.455840 17558 leader_election.cc:290] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:04.465068  7777 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:04.515285  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.525791  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.528733  7777 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:04.532114  7778 consensus_peers.cc:597] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.532913  7778 leader_election.cc:336] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:04.533442  7778 leader_election.cc:304] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:04.534435 17558 raft_consensus.cc:2749] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:04.537182  7778 leader_election.cc:336] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:04.546011  7778 leader_election.cc:304] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:04.547155 17558 raft_consensus.cc:2749] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:30:04.547531  7778 consensus_peers.cc:597] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.577942  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.611937  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:04.626137  7778 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.626917  7778 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.629528  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.656872  7777 consensus_peers.cc:597] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:04.702387  7778 consensus_peers.cc:597] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:04.711197  7778 consensus_peers.cc:597] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.717585  7778 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.730957  7778 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.783396  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:04.788564 17558 raft_consensus.cc:493] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:04.789505 17558 raft_consensus.cc:515] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } }
I20260709 12:30:04.803177 17558 leader_election.cc:290] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
W20260709 12:30:04.808765  7778 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:04.819051  7778 consensus_peers.cc:597] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.835045  7777 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.836063  7778 consensus_peers.cc:597] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.841490  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:04.872586  7777 consensus_peers.cc:597] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:04.897301  7777 leader_election.cc:336] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:04.909657  7777 leader_election.cc:304] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
W20260709 12:30:04.908775  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.907454  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:30:04.918972 17558 raft_consensus.cc:2749] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:30:04.965167  7778 consensus_peers.cc:597] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:04.981645  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:30:04.994661  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:05.023365  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.024111  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.042117  7778 consensus_peers.cc:597] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.137625  7778 consensus_peers.cc:597] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.129559  7777 consensus_peers.cc:597] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:05.160825  7778 consensus_peers.cc:597] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.174571  7778 consensus_peers.cc:597] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:05.168406  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
W20260709 12:30:05.176389  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.184180  7777 consensus_peers.cc:597] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:05.195546  7778 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.210381  7777 consensus_peers.cc:597] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:05.214159  7778 consensus_peers.cc:597] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.218679  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:05.223709  7778 consensus_peers.cc:597] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.244829  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.254952  7778 consensus_peers.cc:597] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.269647  7777 consensus_peers.cc:597] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:30:05.340590  8380 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.387044  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:05.404350  8380 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.491714  7778 consensus_peers.cc:597] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:05.507613 17559 raft_consensus.cc:493] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:05.508591 17559 raft_consensus.cc:515] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } attrs { replace: true } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:05.518924  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b1e442a1beb4dc0bb3b6aef6d503ea5" candidate_uuid: "d72176ccaed049958ac68f706d4f052d" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "992d6df95ef74f7f8b12534534a45501" is_pre_election: true
I20260709 12:30:05.519814  8413 raft_consensus.cc:2468] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72176ccaed049958ac68f706d4f052d in term 24.
I20260709 12:30:05.520778 17559 leader_election.cc:290] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), 992d6df95ef74f7f8b12534534a45501 (127.6.76.196:36487)
W20260709 12:30:05.527940  7777 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:05.536938  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
W20260709 12:30:05.555632  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:30:05.560467  7778 leader_election.cc:336] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:05.561201  7778 leader_election.cc:304] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501, d72176ccaed049958ac68f706d4f052d; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:05.562194 17559 raft_consensus.cc:2749] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:30:05.588267  7778 consensus_peers.cc:597] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.610605  8379 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:05.633868  8379 consensus_peers.cc:597] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:05.639451  7778 consensus_peers.cc:597] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.676980  8379 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.679212  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:05.685526  6451 tablet_replica.cc:333] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.686952  6451 raft_consensus.cc:2243] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.689033  6451 raft_consensus.cc:2272] T 575c307765244f48890311159c0ee934 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:05.690253  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:05.705785  8379 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:05.713292  7778 consensus_peers.cc:597] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:30:05.719862  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.723836  6451 tablet_replica.cc:333] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.725103  6451 raft_consensus.cc:2243] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.726054  6451 raft_consensus.cc:2272] T b3b8edd0c3f4472bb922dba093edf25a P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.744024  6451 tablet_replica.cc:333] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.745811  6451 raft_consensus.cc:2243] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.747121  6451 raft_consensus.cc:2272] T 08777b07fa9d4993a24c05bb4e2db3c8 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.757907  6451 tablet_replica.cc:333] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.759179  6451 raft_consensus.cc:2243] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.762960  6451 raft_consensus.cc:2272] T 38bf041bf77540baa50b5319733c5e9e P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.766639  6451 tablet_replica.cc:333] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.767884  6451 raft_consensus.cc:2243] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.769680  6451 raft_consensus.cc:2272] T 8987ec43c27c4d578d5ad1e143553334 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.773308  6451 tablet_replica.cc:333] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.774559  6451 raft_consensus.cc:2243] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.775593  6451 raft_consensus.cc:2272] T 402d5cf4f7d24ac59dee1221fc2c7a3f P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.781131  6451 tablet_replica.cc:333] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.782780  6451 raft_consensus.cc:2243] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.785660  6451 raft_consensus.cc:2272] T 0a2203ee13714c8bacacb6f83ec31034 P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.789577  6451 tablet_replica.cc:333] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.790738  6451 raft_consensus.cc:2243] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.792450  6451 raft_consensus.cc:2272] T 248d73a0462c48fdad073c597b243806 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.803344  6451 tablet_replica.cc:333] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.807425  6451 raft_consensus.cc:2243] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.809226  6451 raft_consensus.cc:2272] T 16eac2fd2da54bec8a02bb35cc376087 P d72176ccaed049958ac68f706d4f052d [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.816897  6451 tablet_replica.cc:333] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.818498  6451 raft_consensus.cc:2243] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.820278  6451 raft_consensus.cc:2272] T 100d9b95a9cf4fab8305b579485c2332 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.829831  6451 tablet_replica.cc:333] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
W20260709 12:30:05.831825  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.840843  6451 raft_consensus.cc:2243] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.846556  6451 raft_consensus.cc:2272] T 756aa7b7913e49aaba2a2164e45553fe P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:05.844498  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:05.854941  7778 consensus_peers.cc:597] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:05.871093  6451 tablet_replica.cc:333] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.872334  6451 raft_consensus.cc:2243] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.873940  6451 raft_consensus.cc:2272] T 35934889fce04bc093f07a0f03c7165c P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:05.874217  7777 consensus_peers.cc:597] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:30:05.877205  6451 tablet_replica.cc:333] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.878196  6451 raft_consensus.cc:2243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.878968  6451 raft_consensus.cc:2272] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.882175  6451 tablet_replica.cc:333] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.883265  6451 raft_consensus.cc:2243] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Raft consensus shutting down.
W20260709 12:30:05.884678  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.884718  6451 raft_consensus.cc:2272] T 0741ccb73f0c4316be6682db332fd98e P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:05.889748  8379 consensus_peers.cc:597] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.890782  6451 tablet_replica.cc:333] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.891934  6451 raft_consensus.cc:2243] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.892993  6451 raft_consensus.cc:2272] T b71af551cbdf4c63a9a0da6165082874 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.938812  6451 tablet_replica.cc:333] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.945835  6451 raft_consensus.cc:2243] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:05.948693  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.951748  6451 raft_consensus.cc:2272] T bae510633fdb47c1a109530c37a265b0 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.958186  6451 tablet_replica.cc:333] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.961627  6451 raft_consensus.cc:2243] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.963028  6451 raft_consensus.cc:2272] T d997d33a05304f31b50341cc8c4bbf9b P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:05.967756  8379 consensus_peers.cc:597] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:30:05.969020  6451 tablet_replica.cc:333] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.971946  6451 raft_consensus.cc:2243] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:05.972133  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.974042  6451 raft_consensus.cc:2272] T ccd837ed3845462283f72c3010de59c3 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.979802  6451 tablet_replica.cc:333] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.981819  6451 raft_consensus.cc:2243] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:30:05.985746  8379 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:05.987520 17567 raft_consensus.cc:493] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fc963afbbf4c4b34b161448ef79bd2ed)
I20260709 12:30:05.987520  6451 raft_consensus.cc:2272] T fd33cd795fd74820b3adafb3cdbc07c9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.993713 17567 raft_consensus.cc:515] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:05.994906  6451 tablet_replica.cc:333] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:05.998432  6451 raft_consensus.cc:2243] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.000533  6451 raft_consensus.cc:2272] T 3e5b27e8abd04f2ea19a70a13d1958ec P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:06.004529  8379 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:06.006143 17567 leader_election.cc:290] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283)
I20260709 12:30:06.007761  6451 tablet_replica.cc:333] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.011076  6451 raft_consensus.cc:2243] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 LEADER]: Raft consensus shutting down.
W20260709 12:30:06.012228  8380 leader_election.cc:336] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:06.013201  8380 leader_election.cc:304] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:06.014539 17567 raft_consensus.cc:2749] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:30:06.017568  6451 raft_consensus.cc:2272] T 39c84f7f7e484afbae3c398efd5924b4 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.021720  6451 tablet_replica.cc:333] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.023909  6451 raft_consensus.cc:2243] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.025712  6451 raft_consensus.cc:2272] T 79849a645ae64c3787630b03e7da6307 P d72176ccaed049958ac68f706d4f052d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.030989  6451 tablet_replica.cc:333] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.032133  6451 raft_consensus.cc:2243] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.033675  6451 raft_consensus.cc:2272] T cee2ab66e8604e5bbd1f08acbef110d2 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:06.036731  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Couldn't send request to peer d72176ccaed049958ac68f706d4f052d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:06.037864  6451 tablet_replica.cc:333] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.039599  6451 raft_consensus.cc:2243] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.043282  6451 raft_consensus.cc:2272] T 629e345c827545598250d02a9f39e2fb P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.047716  6451 tablet_replica.cc:333] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.049145  6451 raft_consensus.cc:2243] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.054538  6451 raft_consensus.cc:2272] T 539d703b24874183847fa9a8889111e9 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.060530  6451 tablet_replica.cc:333] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.061625  6451 raft_consensus.cc:2243] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.065940  6451 pending_rounds.cc:70] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Trying to abort 1 pending ops.
I20260709 12:30:06.066495  6451 pending_rounds.cc:77] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305626394814861312 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "9b24ad6f078449b6a6418a331b6099f2" old_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } } new_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } } }
I20260709 12:30:06.068758  6451 raft_consensus.cc:2917] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Aborting config change with OpId 22.15: Aborted: Op aborted
I20260709 12:30:06.069756  6451 raft_consensus.cc:2889] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:30:06.070302  6451 raft_consensus.cc:2272] T 9b24ad6f078449b6a6418a331b6099f2 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.075294  6451 tablet_replica.cc:333] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.076821  6451 raft_consensus.cc:2243] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.079358  6451 raft_consensus.cc:2272] T 3960664d9eb541f185c9eac4a231e5c3 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.083442  6451 tablet_replica.cc:333] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.084791  6451 raft_consensus.cc:2243] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.088008  6451 raft_consensus.cc:2272] T 691f0d885861432f91375d5f288af312 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.092973  6451 tablet_replica.cc:333] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.094201  6451 raft_consensus.cc:2243] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.096628  6451 raft_consensus.cc:2272] T ddca3b06c0a84859adac6776a38c3518 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:06.099628  8379 consensus_peers.cc:597] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:30:06.101620  6451 tablet_replica.cc:333] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.109861  6451 raft_consensus.cc:2243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.113109  6451 raft_consensus.cc:2272] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.117573  6451 tablet_replica.cc:333] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.118942  6451 raft_consensus.cc:2243] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.121471  6451 raft_consensus.cc:2272] T e8d96c92c58a4ad683334c5afc8e9a67 P d72176ccaed049958ac68f706d4f052d [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.125823  6451 tablet_replica.cc:333] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.127099  6451 raft_consensus.cc:2243] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:30:06.108884  8379 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:30:06.130714  6451 raft_consensus.cc:2272] T 41a5afed00934a1d9878dd900b236263 P d72176ccaed049958ac68f706d4f052d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.140017  6451 tablet_replica.cc:333] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.141197  6451 raft_consensus.cc:2243] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.142146  6451 raft_consensus.cc:2272] T d9c7a755001445e2895ad66ede4c8f60 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.146369  6451 tablet_replica.cc:333] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.148438  6451 raft_consensus.cc:2243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Raft consensus shutting down.
W20260709 12:30:06.150286  8379 consensus_peers.cc:597] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:30:06.150339  6451 raft_consensus.cc:2272] T 59f766c3d13d4360bcdeb1d5f0f318a8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.156337  6451 tablet_replica.cc:333] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.157575  6451 raft_consensus.cc:2243] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.158618  6451 raft_consensus.cc:2272] T 40860756ca234260bb6147416168cd6a P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.162863  6451 tablet_replica.cc:333] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.163861  6451 raft_consensus.cc:2243] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.166564  6451 raft_consensus.cc:2272] T 268874518d1d4544a8500be048628b67 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:06.167997  8379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111) [suppressed 794 similar messages]
I20260709 12:30:06.171242  6451 tablet_replica.cc:333] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.172587  6451 raft_consensus.cc:2243] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.175529  6451 raft_consensus.cc:2272] T b69d30f987e646bfbea975a362ad70ec P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.180334  6451 tablet_replica.cc:333] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.181680  6451 raft_consensus.cc:2243] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.182951  6451 raft_consensus.cc:2272] T 6b5cde7dcc814e5d90f0b8e433f8f452 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.187438  6451 tablet_replica.cc:333] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.188759  6451 raft_consensus.cc:2243] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.190040  6451 raft_consensus.cc:2272] T d353e70329004cc6b0c1a6bcda38e5c9 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.193162  6451 tablet_replica.cc:333] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.193894  6451 raft_consensus.cc:2243] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.194849  6451 raft_consensus.cc:2272] T 0bbccb309817428d8d84eeecd4ac4410 P d72176ccaed049958ac68f706d4f052d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.198055  6451 tablet_replica.cc:333] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.198946  6451 raft_consensus.cc:2243] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.201547  6451 raft_consensus.cc:2272] T 31230d31601445449fab8f0f9943db57 P d72176ccaed049958ac68f706d4f052d [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.205700  6451 tablet_replica.cc:333] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.206871  6451 raft_consensus.cc:2243] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.208250  6451 raft_consensus.cc:2272] T 033af988cc8f496ca24e4443c21ea5d3 P d72176ccaed049958ac68f706d4f052d [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.212100  6451 tablet_replica.cc:333] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.213230  6451 raft_consensus.cc:2243] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.215613  6451 raft_consensus.cc:2272] T 463898dea61f4d7ab42c74b60beb15e6 P d72176ccaed049958ac68f706d4f052d [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.219609  6451 tablet_replica.cc:333] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.220741  6451 raft_consensus.cc:2243] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.223945  6451 raft_consensus.cc:2272] T 189237b70e7f43458ac90a5773fd551f P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.226974  6451 tablet_replica.cc:333] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.227726  6451 raft_consensus.cc:2243] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.229290  6451 raft_consensus.cc:2272] T 6563fdb577f64184ac4e28f7c43e8c30 P d72176ccaed049958ac68f706d4f052d [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.232649  6451 tablet_replica.cc:333] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.233390  6451 raft_consensus.cc:2243] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.234660  6451 raft_consensus.cc:2272] T 116499cb3f0a4b4eba7b2ace2817bb25 P d72176ccaed049958ac68f706d4f052d [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.237684  6451 tablet_replica.cc:333] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.238476  6451 raft_consensus.cc:2243] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.240988  6451 pending_rounds.cc:70] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Trying to abort 1 pending ops.
I20260709 12:30:06.241284  6451 pending_rounds.cc:77] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305626395784491008 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "a0adec428658479f82eeacfa4eb81d5e" old_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } } new_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } } }
I20260709 12:30:06.242115  6451 raft_consensus.cc:2917] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: Aborting config change with OpId 20.13: Aborted: Op aborted
I20260709 12:30:06.242625  6451 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:30:06.242973  6451 raft_consensus.cc:2272] T a0adec428658479f82eeacfa4eb81d5e P d72176ccaed049958ac68f706d4f052d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.246181  6451 tablet_replica.cc:333] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.246904  6451 raft_consensus.cc:2243] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:06.247771  6451 raft_consensus.cc:2272] T a3d64be480d9430189f5e290bab7c3b8 P d72176ccaed049958ac68f706d4f052d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.251420  6451 tablet_replica.cc:333] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d: stopping tablet replica
I20260709 12:30:06.252256  6451 raft_consensus.cc:2243] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:30:06.252558  8379 consensus_peers.cc:597] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:30:06.254148  6451 raft_consensus.cc:2272] T 62d1255be1b348739b8310c8f1a903d2 P d72176ccaed049958ac68f706d4f052d [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:06.289441  8379 consensus_peers.cc:597] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:06.757989  8380 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:06.800272  8379 consensus_peers.cc:597] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:06.937526  8380 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.039815  8380 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.058015  8379 consensus_peers.cc:597] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:30:07.115113  8379 consensus_peers.cc:597] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:30:07.191685 17571 raft_consensus.cc:493] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.192631 17571 raft_consensus.cc:515] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.196171 17571 leader_election.cc:290] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.205314  8379 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:07.206190  8379 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.206714  8379 leader_election.cc:304] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:30:07.207700 17571 raft_consensus.cc:2749] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:30:07.217429  8380 leader_election.cc:336] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:07.219817 17569 raft_consensus.cc:493] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.220758 17569 raft_consensus.cc:515] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.223569 17569 leader_election.cc:290] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.231302  8379 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
W20260709 12:30:07.231951  8380 leader_election.cc:336] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:07.232510  8380 leader_election.cc:304] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.233434 17569 raft_consensus.cc:2749] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:30:07.235917 17569 raft_consensus.cc:493] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.236799 17569 raft_consensus.cc:515] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.239691 17569 leader_election.cc:290] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.246601  8380 leader_election.cc:336] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:07.247347  8379 leader_election.cc:336] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.247855  8379 leader_election.cc:304] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.248792 17569 raft_consensus.cc:2749] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:30:07.275704 17569 raft_consensus.cc:493] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.276578 17569 raft_consensus.cc:515] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.280068 17569 leader_election.cc:290] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.282805  8379 consensus_peers.cc:597] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:30:07.287988  8380 leader_election.cc:336] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:07.288636  8379 leader_election.cc:336] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.289170  8379 leader_election.cc:304] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.290048 17569 raft_consensus.cc:2749] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:30:07.315549  8379 consensus_peers.cc:597] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:30:07.316629 17569 raft_consensus.cc:493] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.317490 17569 raft_consensus.cc:515] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.327385 17569 leader_election.cc:290] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.343492  8380 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:07.352962  8379 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
I20260709 12:30:07.353658  8379 leader_election.cc:304] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.354010 17575 raft_consensus.cc:493] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.354625 17569 raft_consensus.cc:2749] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:30:07.354960 17575 raft_consensus.cc:515] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:30:07.355783  8379 consensus_peers.cc:597] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.356564  8379 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:30:07.357796  8379 leader_election.cc:336] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.358616 17575 leader_election.cc:290] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
I20260709 12:30:07.361464 17575 raft_consensus.cc:493] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.363582 17575 raft_consensus.cc:515] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
W20260709 12:30:07.366117  8380 leader_election.cc:336] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:07.366171 17575 leader_election.cc:290] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.372329  8379 leader_election.cc:336] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.372990  8379 leader_election.cc:304] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.373849 17575 raft_consensus.cc:2749] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:30:07.376905  8379 leader_election.cc:336] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:07.377833  8379 leader_election.cc:336] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.378341  8379 leader_election.cc:304] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:30:07.379541 17575 raft_consensus.cc:2749] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:30:07.410172  8380 consensus_peers.cc:597] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:30:07.411972  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:30:07.441779 17575 raft_consensus.cc:493] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.442773 17575 raft_consensus.cc:515] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.446331 17575 leader_election.cc:290] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.454311  8380 leader_election.cc:336] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
W20260709 12:30:07.456609  8379 leader_election.cc:336] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.457515  8379 leader_election.cc:304] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.458968 17575 raft_consensus.cc:2749] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:30:07.492130  8380 consensus_peers.cc:597] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.518868  8379 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:30:07.552994 17569 raft_consensus.cc:493] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.553845 17569 raft_consensus.cc:515] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.556911 17569 leader_election.cc:290] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.565909  8379 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111)
W20260709 12:30:07.566854  8379 leader_election.cc:336] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.567453  8379 leader_election.cc:304] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: be9c043c6d424ba1ab8def55dc9bb3e2, d72176ccaed049958ac68f706d4f052d
I20260709 12:30:07.568441 17569 raft_consensus.cc:2749] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:30:07.569607  6451 tablet_server.cc:179] TabletServer@127.6.76.196:0 shutting down...
W20260709 12:30:07.581463  8380 consensus_peers.cc:597] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.582374  8379 consensus_peers.cc:597] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 -> Peer be9c043c6d424ba1ab8def55dc9bb3e2 (127.6.76.193:43519): Couldn't send request to peer be9c043c6d424ba1ab8def55dc9bb3e2. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:43519: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:30:07.618947  8380 consensus_peers.cc:597] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:30:07.638713  8380 consensus_peers.cc:597] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:30:07.649066 17575 raft_consensus.cc:493] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d72176ccaed049958ac68f706d4f052d)
I20260709 12:30:07.649745 17575 raft_consensus.cc:515] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } }
I20260709 12:30:07.655997 17575 leader_election.cc:290] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283), d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801)
W20260709 12:30:07.659778  8380 leader_election.cc:336] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111)
I20260709 12:30:07.661510  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:07.662724  6451 tablet_replica.cc:333] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.664400  6451 raft_consensus.cc:2243] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.665135  6451 pending_rounds.cc:70] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Trying to abort 1 pending ops.
I20260709 12:30:07.665462  6451 pending_rounds.cc:77] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305626395784491008 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "a0adec428658479f82eeacfa4eb81d5e" old_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } } new_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } attrs { replace: true } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } } }
I20260709 12:30:07.667058  6451 raft_consensus.cc:2917] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Aborting config change with OpId 20.13: Aborted: Op aborted
I20260709 12:30:07.667716  6451 raft_consensus.cc:2889] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:30:07.668156  6451 raft_consensus.cc:2272] T a0adec428658479f82eeacfa4eb81d5e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEARNER]: Raft consensus is shut down!
W20260709 12:30:07.669315  8379 leader_election.cc:336] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d72176ccaed049958ac68f706d4f052d (127.6.76.195:37801): Network error: Client connection negotiation failed: client connection to 127.6.76.195:37801: connect: Connection refused (error 111)
I20260709 12:30:07.669947  8379 leader_election.cc:304] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992d6df95ef74f7f8b12534534a45501; no voters: d72176ccaed049958ac68f706d4f052d, fc963afbbf4c4b34b161448ef79bd2ed
I20260709 12:30:07.670887 17575 raft_consensus.cc:2749] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:30:07.672627  6451 tablet_replica.cc:333] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.675453  6451 raft_consensus.cc:2243] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.676879  6451 raft_consensus.cc:2272] T d997d33a05304f31b50341cc8c4bbf9b P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.680183  6451 tablet_replica.cc:333] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.681063  6451 raft_consensus.cc:2243] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.682487  6451 raft_consensus.cc:2272] T 40860756ca234260bb6147416168cd6a P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.686338  6451 tablet_replica.cc:333] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.687525  6451 raft_consensus.cc:2243] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.688449  6451 raft_consensus.cc:2272] T 39c84f7f7e484afbae3c398efd5924b4 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.692830  6451 tablet_replica.cc:333] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
W20260709 12:30:07.693149  8380 consensus_peers.cc:597] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:30:07.693895  6451 raft_consensus.cc:2243] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.695278  6451 raft_consensus.cc:2272] T 3e5b27e8abd04f2ea19a70a13d1958ec P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.698505  6451 tablet_replica.cc:333] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.699613  6451 raft_consensus.cc:2243] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.701258  6451 raft_consensus.cc:2272] T 0733d7d23d9041e09a5ade18807c383e P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.704463  6451 tablet_replica.cc:333] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.705430  6451 raft_consensus.cc:2243] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.706550  6451 raft_consensus.cc:2272] T 24790f3be4e8465581029a2e0d526117 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.710575  6451 tablet_replica.cc:333] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.711548  6451 raft_consensus.cc:2243] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:30:07.712097  8380 consensus_peers.cc:597] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 -> Peer fc963afbbf4c4b34b161448ef79bd2ed (127.6.76.194:35283): Couldn't send request to peer fc963afbbf4c4b34b161448ef79bd2ed. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:35283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:30:07.713038  6451 raft_consensus.cc:2272] T 6b5cde7dcc814e5d90f0b8e433f8f452 P 992d6df95ef74f7f8b12534534a45501 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.716382  6451 tablet_replica.cc:333] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.717252  6451 raft_consensus.cc:2243] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.718112  6451 raft_consensus.cc:2272] T 0741ccb73f0c4316be6682db332fd98e P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.721761  6451 tablet_replica.cc:333] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.722510  6451 raft_consensus.cc:2243] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.723841  6451 raft_consensus.cc:2272] T 674d58a219c247d091f8a10ca4b75af1 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.726532  6451 tablet_replica.cc:333] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.727432  6451 raft_consensus.cc:2243] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.728659  6451 raft_consensus.cc:2272] T cee2ab66e8604e5bbd1f08acbef110d2 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.732069  6451 tablet_replica.cc:333] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.733110  6451 raft_consensus.cc:2243] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.734659  6451 raft_consensus.cc:2272] T 79849a645ae64c3787630b03e7da6307 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.737825  6451 tablet_replica.cc:333] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.738651  6451 raft_consensus.cc:2243] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.739418  6451 raft_consensus.cc:2272] T 6563fdb577f64184ac4e28f7c43e8c30 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.742209  6451 tablet_replica.cc:333] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.742919  6451 raft_consensus.cc:2243] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.744192  6451 raft_consensus.cc:2272] T db9652df6fb248539296e0b903cc4d1c P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.747008  6451 tablet_replica.cc:333] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.747786  6451 raft_consensus.cc:2243] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.749079  6451 raft_consensus.cc:2272] T e78ad8f196c145088ee9ca42482e70ce P 992d6df95ef74f7f8b12534534a45501 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.752200  6451 tablet_replica.cc:333] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.752965  6451 raft_consensus.cc:2243] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.753670  6451 raft_consensus.cc:2272] T 31230d31601445449fab8f0f9943db57 P 992d6df95ef74f7f8b12534534a45501 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.756630  6451 tablet_replica.cc:333] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.757409  6451 raft_consensus.cc:2243] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.758598  6451 raft_consensus.cc:2272] T b71af551cbdf4c63a9a0da6165082874 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.761087  6451 tablet_replica.cc:333] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.761935  6451 raft_consensus.cc:2243] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.762641  6451 raft_consensus.cc:2272] T 7112151d65374f6690a117439cc2dc33 P 992d6df95ef74f7f8b12534534a45501 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.765177  6451 tablet_replica.cc:333] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.765902  6451 raft_consensus.cc:2243] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.767138  6451 raft_consensus.cc:2272] T 1e2e2ecc360e4304b9a841fbd6b8287e P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.770067  6451 tablet_replica.cc:333] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.770795  6451 raft_consensus.cc:2243] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.772257  6451 raft_consensus.cc:2272] T 0bbccb309817428d8d84eeecd4ac4410 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.775408  6451 tablet_replica.cc:333] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.776136  6451 raft_consensus.cc:2243] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.776934  6451 raft_consensus.cc:2272] T 463898dea61f4d7ab42c74b60beb15e6 P 992d6df95ef74f7f8b12534534a45501 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.779938  6451 tablet_replica.cc:333] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.780922  6451 raft_consensus.cc:2243] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.782371  6451 raft_consensus.cc:2272] T cbbae2856e41420d91e3ccfbf507eacf P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.785449  6451 tablet_replica.cc:333] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.786280  6451 raft_consensus.cc:2243] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.788154  6451 raft_consensus.cc:2272] T 033af988cc8f496ca24e4443c21ea5d3 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.791201  6451 tablet_replica.cc:333] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.792117  6451 raft_consensus.cc:2243] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.793391  6451 raft_consensus.cc:2272] T ccd837ed3845462283f72c3010de59c3 P 992d6df95ef74f7f8b12534534a45501 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.796329  6451 tablet_replica.cc:333] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.797096  6451 raft_consensus.cc:2243] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.798250  6451 raft_consensus.cc:2272] T 59f766c3d13d4360bcdeb1d5f0f318a8 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.802000  6451 tablet_replica.cc:333] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.802718  6451 raft_consensus.cc:2243] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.803948  6451 raft_consensus.cc:2272] T ddca3b06c0a84859adac6776a38c3518 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.806452  6451 tablet_replica.cc:333] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.807139  6451 raft_consensus.cc:2243] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.808347  6451 raft_consensus.cc:2272] T 73bea8a36a5046b5b9fb3b3c3a5386f9 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.810734  6451 tablet_replica.cc:333] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.811574  6451 raft_consensus.cc:2243] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.812173  6451 pending_rounds.cc:70] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Trying to abort 1 pending ops.
I20260709 12:30:07.812410  6451 pending_rounds.cc:77] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305626394814861312 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "9b24ad6f078449b6a6418a331b6099f2" old_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: NON_VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: true } } } new_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "be9c043c6d424ba1ab8def55dc9bb3e2" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 43519 } attrs { replace: true } } peers { permanent_uuid: "fc963afbbf4c4b34b161448ef79bd2ed" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 35283 } } peers { permanent_uuid: "d72176ccaed049958ac68f706d4f052d" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 37801 } } peers { permanent_uuid: "992d6df95ef74f7f8b12534534a45501" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 36487 } attrs { promote: false } } } }
I20260709 12:30:07.813390  6451 raft_consensus.cc:2917] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 FOLLOWER]: Aborting config change with OpId 22.15: Aborted: Op aborted
I20260709 12:30:07.813882  6451 raft_consensus.cc:2889] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:30:07.814216  6451 raft_consensus.cc:2272] T 9b24ad6f078449b6a6418a331b6099f2 P 992d6df95ef74f7f8b12534534a45501 [term 22 LEARNER]: Raft consensus is shut down!
I20260709 12:30:07.817637  6451 tablet_replica.cc:333] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.818374  6451 raft_consensus.cc:2243] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.819662  6451 raft_consensus.cc:2272] T 7e34ef3d78ea4d53847d21b92b07fab2 P 992d6df95ef74f7f8b12534534a45501 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.822729  6451 tablet_replica.cc:333] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.823948  6451 raft_consensus.cc:2243] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.825248  6451 raft_consensus.cc:2272] T 1b1e442a1beb4dc0bb3b6aef6d503ea5 P 992d6df95ef74f7f8b12534534a45501 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.830713  6451 tablet_replica.cc:333] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.831552  6451 raft_consensus.cc:2243] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.832954  6451 raft_consensus.cc:2272] T 35f6a2ef5ca4422b882b27d3d2ad1f36 P 992d6df95ef74f7f8b12534534a45501 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.835587  6451 tablet_replica.cc:333] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.836211  6451 raft_consensus.cc:2243] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.836922  6451 raft_consensus.cc:2272] T e8d96c92c58a4ad683334c5afc8e9a67 P 992d6df95ef74f7f8b12534534a45501 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.839475  6451 tablet_replica.cc:333] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.840098  6451 raft_consensus.cc:2243] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:30:07.841610  6451 raft_consensus.cc:2272] T bae510633fdb47c1a109530c37a265b0 P 992d6df95ef74f7f8b12534534a45501 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:07.844643  6451 tablet_replica.cc:333] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501: stopping tablet replica
I20260709 12:30:07.845269  6451 raft_consensus.cc:2243] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:07.845939  6451 raft_consensus.cc:2272] T 116499cb3f0a4b4eba7b2ace2817bb25 P 992d6df95ef74f7f8b12534534a45501 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:08.466845  6451 tablet_server.cc:196] TabletServer@127.6.76.196:0 shutdown complete.
I20260709 12:30:08.552752  6451 master.cc:562] Master@127.6.76.254:33941 shutting down...
W20260709 12:30:08.596101  7607 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:30:08.596486  7607 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:30:08.596745  7607 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 12:30:08.601542  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:08.602458  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:08.602866  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P 004cbdb090d14f8ca89ed535ed34a625: stopping tablet replica
I20260709 12:30:08.771515  6451 master.cc:584] Master@127.6.76.254:33941 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (326923 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:30:08.894378  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:33695
I20260709 12:30:08.895689  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:08.902841 17577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:08.903123 17576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:08.903872 17579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:08.904999  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:08.908779  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:08.909037  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:08.909277  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600208909254 us; error 0 us; skew 500 ppm
I20260709 12:30:08.909876  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:08.913059  6451 webserver.cc:533] Webserver started at http://127.6.76.254:45011/ using document root <none> and password file <none>
I20260709 12:30:08.913681  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:08.913902  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:08.914248  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:08.915808  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "d05bd585df814ae9904c4dc50a8943f2"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-n447"
I20260709 12:30:08.921386  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:30:08.926445 17584 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:08.927634  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:08.928093  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "d05bd585df814ae9904c4dc50a8943f2"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-n447"
I20260709 12:30:08.928547  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:08.949288  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:08.950888  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.001479  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:33695
I20260709 12:30:09.001606 17635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:33695 every 8 connection(s)
I20260709 12:30:09.006449 17636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.019435 17636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2: Bootstrap starting.
I20260709 12:30:09.024983 17636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:09.030709 17636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2: No bootstrap required, opened a new log
I20260709 12:30:09.033607 17636 raft_consensus.cc:359] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER }
I20260709 12:30:09.034066 17636 raft_consensus.cc:385] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:09.034281 17636 raft_consensus.cc:740] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05bd585df814ae9904c4dc50a8943f2, State: Initialized, Role: FOLLOWER
I20260709 12:30:09.034979 17636 consensus_queue.cc:260] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER }
I20260709 12:30:09.035467 17636 raft_consensus.cc:399] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:09.035682 17636 raft_consensus.cc:493] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:09.035915 17636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:09.041210 17636 raft_consensus.cc:515] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER }
I20260709 12:30:09.041774 17636 leader_election.cc:304] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d05bd585df814ae9904c4dc50a8943f2; no voters: 
I20260709 12:30:09.043028 17636 leader_election.cc:290] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:09.043643 17639 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:09.045249 17639 raft_consensus.cc:697] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 1 LEADER]: Becoming Leader. State: Replica: d05bd585df814ae9904c4dc50a8943f2, State: Running, Role: LEADER
I20260709 12:30:09.046214 17639 consensus_queue.cc:237] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER }
I20260709 12:30:09.046571 17636 sys_catalog.cc:565] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:09.052151 17640 sys_catalog.cc:455] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d05bd585df814ae9904c4dc50a8943f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER } }
I20260709 12:30:09.052994 17640 sys_catalog.cc:458] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:09.052933 17641 sys_catalog.cc:455] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d05bd585df814ae9904c4dc50a8943f2. Latest consensus state: current_term: 1 leader_uuid: "d05bd585df814ae9904c4dc50a8943f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d05bd585df814ae9904c4dc50a8943f2" member_type: VOTER } }
I20260709 12:30:09.053714 17641 sys_catalog.cc:458] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:09.059998 17645 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:09.068296 17645 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:09.069538  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:09.080842 17645 catalog_manager.cc:1383] Generated new cluster ID: e215920c78ab4468a57e03df37b080cd
I20260709 12:30:09.081287 17645 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:09.100843 17645 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:09.102983 17645 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:09.122021 17645 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2: Generated new TSK 0
I20260709 12:30:09.123044 17645 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:09.137961  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.145783 17657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.147873 17658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.154579  6451 server_base.cc:1061] running on GCE node
W20260709 12:30:09.154593 17660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.158978  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.159361  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.159505  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600209159492 us; error 0 us; skew 500 ppm
I20260709 12:30:09.160113  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.163753  6451 webserver.cc:533] Webserver started at http://127.6.76.193:37929/ using document root <none> and password file <none>
I20260709 12:30:09.164337  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.164534  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.164824  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.166199  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.171814  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:30:09.176851 17665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.177876  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 12:30:09.178309  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.178666  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.211319  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.212733  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.214504  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.217446  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.217737  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.218065  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.218297  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.271180  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:37959
I20260709 12:30:09.271382 17727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:37959 every 8 connection(s)
I20260709 12:30:09.277868  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.286146 17732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.288445 17733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.291687 17735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.291818  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:09.296056  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.296486  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.296747  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600209296724 us; error 0 us; skew 500 ppm
I20260709 12:30:09.297643  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.299743 17728 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33695
I20260709 12:30:09.300251 17728 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.300969  6451 webserver.cc:533] Webserver started at http://127.6.76.194:37535/ using document root <none> and password file <none>
I20260709 12:30:09.301415 17728 heartbeater.cc:507] Master 127.6.76.254:33695 requested a full tablet report, sending...
I20260709 12:30:09.301777  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.302089  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.302465  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.304585  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "753c9642e55f4b819d92519d568c2d94"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.304942 17601 ts_manager.cc:194] Registered new tserver with Master: b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:09.308086 17601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50444
I20260709 12:30:09.312417  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:09.317833 17740 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.319047  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:30:09.319593  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "753c9642e55f4b819d92519d568c2d94"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.319952  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.333681  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.335341  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.337160  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.341534  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.341925  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.342321  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.342585  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.400770  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:38991
I20260709 12:30:09.400906 17802 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:38991 every 8 connection(s)
I20260709 12:30:09.406422  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.414625 17806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.416853 17807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.420228  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:09.421446 17803 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33695
I20260709 12:30:09.422031 17803 heartbeater.cc:461] Registering TS with master...
W20260709 12:30:09.422027 17810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.423694 17803 heartbeater.cc:507] Master 127.6.76.254:33695 requested a full tablet report, sending...
I20260709 12:30:09.423827  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.424328  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.424595  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600209424576 us; error 0 us; skew 500 ppm
I20260709 12:30:09.425523  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.427417 17601 ts_manager.cc:194] Registered new tserver with Master: 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:09.429950 17601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50454
I20260709 12:30:09.430210  6451 webserver.cc:533] Webserver started at http://127.6.76.195:36075/ using document root <none> and password file <none>
I20260709 12:30:09.431013  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.431396  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.431744  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.433699  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "632ff234ebc74963b295e6b50ccb78e6"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.440400  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 12:30:09.445269 17814 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.446421  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 12:30:09.446923  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "632ff234ebc74963b295e6b50ccb78e6"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.447309  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.457835  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.459453  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.461117  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.464036  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.464331  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.464614  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.464807  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.517673  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:45767
I20260709 12:30:09.517800 17876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:45767 every 8 connection(s)
I20260709 12:30:09.523458  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.534019 17880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.535476 17881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.540194 17877 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33695
I20260709 12:30:09.540833  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:09.540928 17877 heartbeater.cc:461] Registering TS with master...
W20260709 12:30:09.540846 17883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.542862 17877 heartbeater.cc:507] Master 127.6.76.254:33695 requested a full tablet report, sending...
I20260709 12:30:09.542924  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.543419  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.543639  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600209543618 us; error 0 us; skew 500 ppm
I20260709 12:30:09.544538  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.546245 17601 ts_manager.cc:194] Registered new tserver with Master: 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:09.548256  6451 webserver.cc:533] Webserver started at http://127.6.76.196:33047/ using document root <none> and password file <none>
I20260709 12:30:09.548661 17601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50456
I20260709 12:30:09.549274  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.549626  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.550197  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.551774  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-3-root/instance:
uuid: "30d17e55b097491f895c84a3443cb250"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.557883  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:30:09.562094 17888 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.563205  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:30:09.563699  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
uuid: "30d17e55b097491f895c84a3443cb250"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:09.564069  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.576870  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.578295  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.579872  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.582609  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.582935  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.583261  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.583483  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:30:09.639020  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.196:46335
I20260709 12:30:09.639159 17950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.196:46335 every 8 connection(s)
I20260709 12:30:09.656467 17951 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33695
I20260709 12:30:09.657124 17951 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.658113 17951 heartbeater.cc:507] Master 127.6.76.254:33695 requested a full tablet report, sending...
I20260709 12:30:09.660702 17601 ts_manager.cc:194] Registered new tserver with Master: 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:09.660934  6451 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.018202455s
I20260709 12:30:09.662415 17601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50472
I20260709 12:30:09.695626 17601 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50478:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:30:09.988317 17916 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:09.990201 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.992846 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8bdb4d10d0433c909031a9340f6c1b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:09.996978 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8bdb4d10d0433c909031a9340f6c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.997143 17913 tablet_service.cc:1511] Processing CreateTablet for tablet edef181ff5d249908318aaf871e8b47e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:09.997983 17914 tablet_service.cc:1511] Processing CreateTablet for tablet deb7403698d84ba789b17c0ef904859f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:30:09.999207 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edef181ff5d249908318aaf871e8b47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.997139 17912 tablet_service.cc:1511] Processing CreateTablet for tablet 051771ffcb694816b9aff49434d710aa (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:10.001325 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 051771ffcb694816b9aff49434d710aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.002187 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7403698d84ba789b17c0ef904859f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.004142 17911 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8ca1913d34de8bdacce55ee0fa4da (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:10.006048 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8ca1913d34de8bdacce55ee0fa4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.012493 17693 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:10.014298 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.020251 17910 tablet_service.cc:1511] Processing CreateTablet for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:10.022642 17910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.026280 17690 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:10.029474 17689 tablet_service.cc:1511] Processing CreateTablet for tablet edef181ff5d249908318aaf871e8b47e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:10.031436 17687 tablet_service.cc:1511] Processing CreateTablet for tablet d7b12a0adfe741248970463752f3b30e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:10.032769 17692 tablet_service.cc:1511] Processing CreateTablet for tablet 20637ddae560402b81988ca96fc61b1e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:30:10.034135 17688 tablet_service.cc:1511] Processing CreateTablet for tablet 051771ffcb694816b9aff49434d710aa (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:10.035583 17691 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8bdb4d10d0433c909031a9340f6c1b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:10.058115 17687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b12a0adfe741248970463752f3b30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.064913 17765 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:10.066963 17765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.069044 17766 tablet_service.cc:1511] Processing CreateTablet for tablet deb7403698d84ba789b17c0ef904859f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:30:10.095314 17835 tablet_service.cc:1511] Processing CreateTablet for tablet c953b9330799457cb8d5c4ea252f5b30 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:10.073587 17914 tablet_service.cc:1511] Processing CreateTablet for tablet c953b9330799457cb8d5c4ea252f5b30 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:10.073365 17763 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8ca1913d34de8bdacce55ee0fa4da (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:10.096740 17837 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8ca1913d34de8bdacce55ee0fa4da (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:10.075361 17764 tablet_service.cc:1511] Processing CreateTablet for tablet edef181ff5d249908318aaf871e8b47e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:10.097813 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c953b9330799457cb8d5c4ea252f5b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.078178 17686 tablet_service.cc:1511] Processing CreateTablet for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:10.080363 17768 tablet_service.cc:1511] Processing CreateTablet for tablet 20637ddae560402b81988ca96fc61b1e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
W20260709 12:30:10.083052 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:10.100322 17762 tablet_service.cc:1511] Processing CreateTablet for tablet d7b12a0adfe741248970463752f3b30e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:10.102617 17762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b12a0adfe741248970463752f3b30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.103562 17763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8ca1913d34de8bdacce55ee0fa4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.105084 17761 tablet_service.cc:1511] Processing CreateTablet for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:10.106799 17764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edef181ff5d249908318aaf871e8b47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.093068 17836 tablet_service.cc:1511] Processing CreateTablet for tablet d7b12a0adfe741248970463752f3b30e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:10.109922 17768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20637ddae560402b81988ca96fc61b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.111033 17761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.093041 17840 tablet_service.cc:1511] Processing CreateTablet for tablet deb7403698d84ba789b17c0ef904859f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:30:10.115952 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7403698d84ba789b17c0ef904859f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.091571 17841 tablet_service.cc:1511] Processing CreateTablet for tablet 20637ddae560402b81988ca96fc61b1e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:30:10.097343 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c953b9330799457cb8d5c4ea252f5b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.095314 17838 tablet_service.cc:1511] Processing CreateTablet for tablet 051771ffcb694816b9aff49434d710aa (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:10.121642 17838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 051771ffcb694816b9aff49434d710aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.077952 17767 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8bdb4d10d0433c909031a9340f6c1b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:10.123867 17841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20637ddae560402b81988ca96fc61b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.124523 17767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8bdb4d10d0433c909031a9340f6c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.091490 17842 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:10.127527 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b12a0adfe741248970463752f3b30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.128588 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd0b912e8f4ed3a78dbc39c92aa5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.093922 17839 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:10.132550 17839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.135505 17760 tablet_service.cc:1511] Processing CreateTablet for tablet 7da5cd04765b43bdaec5b9bae211aae0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:10.137476 17760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da5cd04765b43bdaec5b9bae211aae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.139631 17837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8ca1913d34de8bdacce55ee0fa4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.140743 17834 tablet_service.cc:1511] Processing CreateTablet for tablet 7da5cd04765b43bdaec5b9bae211aae0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:10.142601 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da5cd04765b43bdaec5b9bae211aae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.145993 17685 tablet_service.cc:1511] Processing CreateTablet for tablet 85f591a6d82c496ab6a106341934b97d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:10.088176 17691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8bdb4d10d0433c909031a9340f6c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.148682 17686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a94780ceb44c84bbaa8ba8ce21b7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.149932 17840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7403698d84ba789b17c0ef904859f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.140857 17916 tablet_service.cc:1511] Processing CreateTablet for tablet 7da5cd04765b43bdaec5b9bae211aae0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:10.126511 17687 tablet_service.cc:1511] Processing CreateTablet for tablet c953b9330799457cb8d5c4ea252f5b30 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:10.150045 17689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edef181ff5d249908318aaf871e8b47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.161252 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f591a6d82c496ab6a106341934b97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.164019 17688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 051771ffcb694816b9aff49434d710aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.171854 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7db027e2c4f5186a0d0c50ed0d6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.177973 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20637ddae560402b81988ca96fc61b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.196020 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da5cd04765b43bdaec5b9bae211aae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.197403 17759 tablet_service.cc:1511] Processing CreateTablet for tablet 85f591a6d82c496ab6a106341934b97d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:10.198604 17759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f591a6d82c496ab6a106341934b97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.206399 17687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c953b9330799457cb8d5c4ea252f5b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.210093 17833 tablet_service.cc:1511] Processing CreateTablet for tablet 85f591a6d82c496ab6a106341934b97d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:10.216063 17833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f591a6d82c496ab6a106341934b97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.222613 17913 tablet_service.cc:1511] Processing CreateTablet for tablet 85189191344a46deb54f096d7ac2c06d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:10.224465 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85189191344a46deb54f096d7ac2c06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.225781 17693 tablet_service.cc:1511] Processing CreateTablet for tablet 85189191344a46deb54f096d7ac2c06d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:10.232138 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85189191344a46deb54f096d7ac2c06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.243345 17971 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:10.246639 17972 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:10.291882 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 79b1348052e745648f1cba4c11a7e863 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.293738 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b1348052e745648f1cba4c11a7e863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.306056 17973 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:10.312254 17728 heartbeater.cc:499] Master 127.6.76.254:33695 was elected leader, sending a full tablet report...
I20260709 12:30:10.313108 17835 tablet_service.cc:1511] Processing CreateTablet for tablet 79b1348052e745648f1cba4c11a7e863 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.314740 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b1348052e745648f1cba4c11a7e863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.334425 17972 tablet_bootstrap.cc:654] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.335541 17684 tablet_service.cc:1511] Processing CreateTablet for tablet 79b1348052e745648f1cba4c11a7e863 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.340530 17684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b1348052e745648f1cba4c11a7e863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.354912 17971 tablet_bootstrap.cc:654] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.375313 17835 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4f834976044f1be00f6e31249489e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:10.380035 17910 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4f834976044f1be00f6e31249489e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:10.396847 17765 tablet_service.cc:1511] Processing CreateTablet for tablet 85189191344a46deb54f096d7ac2c06d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:10.416548 17767 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff59477c0d846eaab2e41621e730061 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:10.406628 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff59477c0d846eaab2e41621e730061 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:10.337908 17973 tablet_bootstrap.cc:654] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.404357 17688 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff59477c0d846eaab2e41621e730061 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:10.434319 17803 heartbeater.cc:499] Master 127.6.76.254:33695 was elected leader, sending a full tablet report...
I20260709 12:30:10.454563 17688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff59477c0d846eaab2e41621e730061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.449594 17914 tablet_service.cc:1511] Processing CreateTablet for tablet f3120efae4a24a319065841f0dff065a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.452886 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4f834976044f1be00f6e31249489e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.465209 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff59477c0d846eaab2e41621e730061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.406622 17766 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4f834976044f1be00f6e31249489e (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:10.408414 17974 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:10.448544 17765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85189191344a46deb54f096d7ac2c06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.450296 17910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4f834976044f1be00f6e31249489e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.493012 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3120efae4a24a319065841f0dff065a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.498162 17685 tablet_service.cc:1511] Processing CreateTablet for tablet f3120efae4a24a319065841f0dff065a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.490586 17767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff59477c0d846eaab2e41621e730061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.499775 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3120efae4a24a319065841f0dff065a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.508365 17972 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:10.509033 17972 ts_tablet_manager.cc:1403] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.263s	user 0.010s	sys 0.007s
I20260709 12:30:10.514108 17971 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:10.514766 17971 ts_tablet_manager.cc:1403] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.272s	user 0.000s	sys 0.013s
I20260709 12:30:10.524381 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4f834976044f1be00f6e31249489e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.526911 17760 tablet_service.cc:1511] Processing CreateTablet for tablet f3120efae4a24a319065841f0dff065a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.528594 17760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3120efae4a24a319065841f0dff065a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.533443 17974 tablet_bootstrap.cc:654] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.535534 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 2933687c848f48e2b01b32ede660a4c9 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.537228 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2933687c848f48e2b01b32ede660a4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.548906 17973 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:10.549574 17973 ts_tablet_manager.cc:1403] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.003s
I20260709 12:30:10.550575 17759 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd5aff4923949cd9a68bdae18218da4 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.552372 17877 heartbeater.cc:499] Master 127.6.76.254:33695 was elected leader, sending a full tablet report...
I20260709 12:30:10.563033 17833 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd5aff4923949cd9a68bdae18218da4 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.563868 17913 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd5aff4923949cd9a68bdae18218da4 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.565757 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd5aff4923949cd9a68bdae18218da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.528617 17842 tablet_service.cc:1511] Processing CreateTablet for tablet 2933687c848f48e2b01b32ede660a4c9 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.543089 17972 raft_consensus.cc:359] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:10.567221 17972 raft_consensus.cc:385] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.567749 17972 raft_consensus.cc:740] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.568953 17972 consensus_queue.cc:260] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:10.576272 17693 tablet_service.cc:1511] Processing CreateTablet for tablet 2933687c848f48e2b01b32ede660a4c9 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.578184 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2933687c848f48e2b01b32ede660a4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.580994 17759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd5aff4923949cd9a68bdae18218da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.581367 17973 raft_consensus.cc:359] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.582299 17973 raft_consensus.cc:385] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.582624 17973 raft_consensus.cc:740] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.583994 17973 consensus_queue.cc:260] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.617143 17690 tablet_service.cc:1511] Processing CreateTablet for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.623920 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.630137 17971 raft_consensus.cc:359] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:10.631214 17971 raft_consensus.cc:385] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.631621 17971 raft_consensus.cc:740] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.631760 17972 ts_tablet_manager.cc:1434] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.122s	user 0.010s	sys 0.013s
I20260709 12:30:10.640323 17973 ts_tablet_manager.cc:1434] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.090s	user 0.014s	sys 0.022s
I20260709 12:30:10.646216 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.648046 17767 tablet_service.cc:1511] Processing CreateTablet for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.649828 17767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.656811 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2933687c848f48e2b01b32ede660a4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.632769 17971 consensus_queue.cc:260] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:10.663975 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2aef671e0349d6ab0d618271ae26a8 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.666033 17951 heartbeater.cc:499] Master 127.6.76.254:33695 was elected leader, sending a full tablet report...
I20260709 12:30:10.666368 17973 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:10.668427 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4872dc0c0f3c4bf69e5618fae1ac5f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.678535 17840 tablet_service.cc:1511] Processing CreateTablet for tablet 09de1ae34c854240a05f695427ac7c94 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.683089 17971 ts_tablet_manager.cc:1434] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.168s	user 0.014s	sys 0.000s
I20260709 12:30:10.684561 17916 tablet_service.cc:1511] Processing CreateTablet for tablet 09de1ae34c854240a05f695427ac7c94 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.680578 17840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09de1ae34c854240a05f695427ac7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.684688 17766 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2aef671e0349d6ab0d618271ae26a8 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.686616 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2aef671e0349d6ab0d618271ae26a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.684688 17691 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2aef671e0349d6ab0d618271ae26a8 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.692174 17833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd5aff4923949cd9a68bdae18218da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.693667 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09de1ae34c854240a05f695427ac7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.698407 17974 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:10.699085 17971 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:10.699110 17974 ts_tablet_manager.cc:1403] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.291s	user 0.015s	sys 0.003s
I20260709 12:30:10.706462 17972 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:10.707038 17974 raft_consensus.cc:359] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.707931 17974 raft_consensus.cc:385] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.708262 17974 raft_consensus.cc:740] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.709142 17974 consensus_queue.cc:260] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.709842 17691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2aef671e0349d6ab0d618271ae26a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.711679 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2aef671e0349d6ab0d618271ae26a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.714704 17972 tablet_bootstrap.cc:654] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.706480 17973 tablet_bootstrap.cc:654] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.722994 17835 tablet_service.cc:1511] Processing CreateTablet for tablet 57e8cc31758742738306cb81f8d38d38 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.724445 17914 tablet_service.cc:1511] Processing CreateTablet for tablet 57e8cc31758742738306cb81f8d38d38 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.725932 17685 tablet_service.cc:1511] Processing CreateTablet for tablet 09de1ae34c854240a05f695427ac7c94 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.726038 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e8cc31758742738306cb81f8d38d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.729025 17971 tablet_bootstrap.cc:654] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.733247 17693 tablet_service.cc:1511] Processing CreateTablet for tablet 6874bb3d07a84cc1b883375c3e83bcfe (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.735329 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874bb3d07a84cc1b883375c3e83bcfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.737704 17760 tablet_service.cc:1511] Processing CreateTablet for tablet 6874bb3d07a84cc1b883375c3e83bcfe (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.739494 17760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874bb3d07a84cc1b883375c3e83bcfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.739768 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e8cc31758742738306cb81f8d38d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.748143 17759 tablet_service.cc:1511] Processing CreateTablet for tablet 57e8cc31758742738306cb81f8d38d38 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.750018 17759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e8cc31758742738306cb81f8d38d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.755788 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09de1ae34c854240a05f695427ac7c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.767350 17834 tablet_service.cc:1511] Processing CreateTablet for tablet 6874bb3d07a84cc1b883375c3e83bcfe (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.769026 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874bb3d07a84cc1b883375c3e83bcfe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:10.777483 17878 tablet.cc:2406] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:10.767412 17974 ts_tablet_manager.cc:1434] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.064s	user 0.014s	sys 0.001s
I20260709 12:30:10.792537 17974 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:10.805485 17910 tablet_service.cc:1511] Processing CreateTablet for tablet ffeea622a1d642ffb92c2fe4b4dcaa57 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.807099 17910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeea622a1d642ffb92c2fe4b4dcaa57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.806808 17690 tablet_service.cc:1511] Processing CreateTablet for tablet ffeea622a1d642ffb92c2fe4b4dcaa57 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.808420 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeea622a1d642ffb92c2fe4b4dcaa57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.826449 17767 tablet_service.cc:1511] Processing CreateTablet for tablet ffeea622a1d642ffb92c2fe4b4dcaa57 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.828718 17767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeea622a1d642ffb92c2fe4b4dcaa57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.831112 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 59353fd7c83841ea9e358df44da38a6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.832758 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59353fd7c83841ea9e358df44da38a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.833247 17973 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:10.834023 17973 ts_tablet_manager.cc:1403] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.168s	user 0.018s	sys 0.025s
I20260709 12:30:10.834857 17691 tablet_service.cc:1511] Processing CreateTablet for tablet 59353fd7c83841ea9e358df44da38a6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.836644 17691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59353fd7c83841ea9e358df44da38a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.836845 17981 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.839141 17981 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:10.837410 17972 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:10.838056 17916 tablet_service.cc:1511] Processing CreateTablet for tablet 59353fd7c83841ea9e358df44da38a6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.846511 17981 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:10.853649 17972 ts_tablet_manager.cc:1403] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.008s
I20260709 12:30:10.867146 17842 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd790b3318a46d0a138b9649e5e447a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.868901 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59353fd7c83841ea9e358df44da38a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.871841 17972 raft_consensus.cc:359] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:10.869097 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd790b3318a46d0a138b9649e5e447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.881212 17971 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:10.882793 17971 ts_tablet_manager.cc:1403] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.184s	user 0.030s	sys 0.002s
I20260709 12:30:10.889523 17760 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd790b3318a46d0a138b9649e5e447a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.881733 17972 raft_consensus.cc:385] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.891004 17972 raft_consensus.cc:740] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.891726 17979 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.892091 17972 consensus_queue.cc:260] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:10.892565 17979 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.902415 17760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd790b3318a46d0a138b9649e5e447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.903779 17979 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:10.904742 17974 tablet_bootstrap.cc:654] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.927956 17978 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.930015 17913 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd790b3318a46d0a138b9649e5e447a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.930747 17764 tablet_service.cc:1511] Processing CreateTablet for tablet 952cd8f570354b1ea762c251df7bbc19 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:10.931666 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd790b3318a46d0a138b9649e5e447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.950333 17972 ts_tablet_manager.cc:1434] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.088s	user 0.005s	sys 0.004s
I20260709 12:30:10.951637 17972 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:10.953917 17973 raft_consensus.cc:359] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.954917 17973 raft_consensus.cc:385] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.928627 17978 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:10.955403 17973 raft_consensus.cc:740] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.956920 17973 consensus_queue.cc:260] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:10.959107 17972 tablet_bootstrap.cc:654] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.916813 17971 raft_consensus.cc:359] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:10.977626 17911 tablet_service.cc:1511] Processing CreateTablet for tablet 49f142d3ffe24e67b967c51897ba2e82 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:10.964061 17971 raft_consensus.cc:385] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.979166 17971 raft_consensus.cc:740] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.980360 17971 consensus_queue.cc:260] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:10.980564 17688 tablet_service.cc:1511] Processing CreateTablet for tablet 952cd8f570354b1ea762c251df7bbc19 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:10.982858 17688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cd8f570354b1ea762c251df7bbc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.983453 17764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cd8f570354b1ea762c251df7bbc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.983004 17971 ts_tablet_manager.cc:1434] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.001s
I20260709 12:30:11.001888 17973 ts_tablet_manager.cc:1434] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.168s	user 0.010s	sys 0.006s
I20260709 12:30:11.033206 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f142d3ffe24e67b967c51897ba2e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.035624 17982 raft_consensus.cc:493] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.037873 17982 raft_consensus.cc:515] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.037331 17973 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:11.055847 17978 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:11.085590 17973 tablet_bootstrap.cc:654] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.088464 17915 tablet_service.cc:1511] Processing CreateTablet for tablet b3592a718f974e5f9f26e16b1942834f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:11.090467 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3592a718f974e5f9f26e16b1942834f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.090572 17914 tablet_service.cc:1511] Processing CreateTablet for tablet ffede44de4554930b68441a774c0dd06 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:11.088719 17834 tablet_service.cc:1511] Processing CreateTablet for tablet 952cd8f570354b1ea762c251df7bbc19 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:11.092315 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffede44de4554930b68441a774c0dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.092448 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cd8f570354b1ea762c251df7bbc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.104784 17766 tablet_service.cc:1511] Processing CreateTablet for tablet b3592a718f974e5f9f26e16b1942834f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:11.105738 17765 tablet_service.cc:1511] Processing CreateTablet for tablet ea1fbecc92da4549828813c3a7185b1a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:11.107739 17982 leader_election.cc:290] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:11.112663 17759 tablet_service.cc:1511] Processing CreateTablet for tablet f211414d5e584164a01d70059b214a4b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 12:30:11.114861 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:11.106487 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3592a718f974e5f9f26e16b1942834f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.145884 17971 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:11.173216 17912 tablet_service.cc:1511] Processing CreateTablet for tablet d8101eccca0c484c9d5b29aaff2497db (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:11.174158 17763 tablet_service.cc:1511] Processing CreateTablet for tablet 67ea582b2cc3418bac58a2e7ceb32649 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:11.159144 17910 tablet_service.cc:1511] Processing CreateTablet for tablet ea1fbecc92da4549828813c3a7185b1a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:11.160955 17833 tablet_service.cc:1511] Processing CreateTablet for tablet ffede44de4554930b68441a774c0dd06 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:11.164263 17974 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:11.165148 17765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1fbecc92da4549828813c3a7185b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.164438 17767 tablet_service.cc:1511] Processing CreateTablet for tablet 49f142d3ffe24e67b967c51897ba2e82 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:11.196441 17761 tablet_service.cc:1511] Processing CreateTablet for tablet 21c2238a9361414181c3aff69d2f9dbd (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:11.175014 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8101eccca0c484c9d5b29aaff2497db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.185941 17908 tablet_service.cc:1511] Processing CreateTablet for tablet 21c2238a9361414181c3aff69d2f9dbd (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:11.187659 17690 tablet_service.cc:1511] Processing CreateTablet for tablet 49f142d3ffe24e67b967c51897ba2e82 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:11.188465 17907 tablet_service.cc:1511] Processing CreateTablet for tablet 16e7941692924de28d3fb53cf3cd833a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:11.189261 17972 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:11.194825 17685 tablet_service.cc:1511] Processing CreateTablet for tablet f211414d5e584164a01d70059b214a4b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:30:11.204212 17974 ts_tablet_manager.cc:1403] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.412s	user 0.016s	sys 0.015s
I20260709 12:30:11.175333 17909 tablet_service.cc:1511] Processing CreateTablet for tablet 67ea582b2cc3418bac58a2e7ceb32649 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:11.225642 17972 ts_tablet_manager.cc:1403] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.271s	user 0.022s	sys 0.029s
I20260709 12:30:11.223510 17981 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.232158 17833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffede44de4554930b68441a774c0dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.242200 17759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f211414d5e584164a01d70059b214a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.279951 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:11.248026 17979 raft_consensus.cc:493] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.281983 17979 raft_consensus.cc:515] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:11.282606 17742 leader_election.cc:343] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.250989 17981 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.283943 17767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f142d3ffe24e67b967c51897ba2e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.287484 17761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c2238a9361414181c3aff69d2f9dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.292673 17689 tablet_service.cc:1511] Processing CreateTablet for tablet fbb98822af0c4e85aa127325141dce47 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:11.266762 17837 tablet_service.cc:1511] Processing CreateTablet for tablet 16e7941692924de28d3fb53cf3cd833a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:11.295504 17686 tablet_service.cc:1511] Processing CreateTablet for tablet 21c2238a9361414181c3aff69d2f9dbd (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:11.298164 17687 tablet_service.cc:1511] Processing CreateTablet for tablet 16e7941692924de28d3fb53cf3cd833a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:11.299666 17763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ea582b2cc3418bac58a2e7ceb32649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.300172 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:11.302050 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:11.254375 17692 tablet_service.cc:1511] Processing CreateTablet for tablet d8101eccca0c484c9d5b29aaff2497db (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:11.264034 17693 tablet_service.cc:1511] Processing CreateTablet for tablet ffede44de4554930b68441a774c0dd06 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:11.254709 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:11.304148 17837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e7941692924de28d3fb53cf3cd833a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.259397 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f142d3ffe24e67b967c51897ba2e82. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.306015 17890 leader_election.cc:343] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.312765 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8101eccca0c484c9d5b29aaff2497db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.313674 17979 leader_election.cc:290] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:11.262742 17684 tablet_service.cc:1511] Processing CreateTablet for tablet ea1fbecc92da4549828813c3a7185b1a (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:11.263813 17764 tablet_service.cc:1511] Processing CreateTablet for tablet de32f68d2ee24bd09e87ceae49cd7e81 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:11.263984 17840 tablet_service.cc:1511] Processing CreateTablet for tablet f211414d5e584164a01d70059b214a4b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:30:11.265626 17838 tablet_service.cc:1511] Processing CreateTablet for tablet 67ea582b2cc3418bac58a2e7ceb32649 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:11.266069 17835 tablet_service.cc:1511] Processing CreateTablet for tablet b3592a718f974e5f9f26e16b1942834f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:11.265626 17841 tablet_service.cc:1511] Processing CreateTablet for tablet fbb98822af0c4e85aa127325141dce47 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:11.268297 17839 tablet_service.cc:1511] Processing CreateTablet for tablet d8101eccca0c484c9d5b29aaff2497db (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:11.269258 17762 tablet_service.cc:1511] Processing CreateTablet for tablet fbb98822af0c4e85aa127325141dce47 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:11.342202 17840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f211414d5e584164a01d70059b214a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.343036 17974 raft_consensus.cc:359] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:11.344092 17974 raft_consensus.cc:385] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.344432 17974 raft_consensus.cc:740] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.346773 17974 consensus_queue.cc:260] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:11.348908 17838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ea582b2cc3418bac58a2e7ceb32649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.356716 17839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8101eccca0c484c9d5b29aaff2497db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.360052 17841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb98822af0c4e85aa127325141dce47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.316231 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f211414d5e584164a01d70059b214a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.364149 17684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1fbecc92da4549828813c3a7185b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.364599 17974 ts_tablet_manager.cc:1434] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.141s	user 0.009s	sys 0.000s
W20260709 12:30:11.317822 17889 leader_election.cc:343] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.318689 17842 tablet_service.cc:1511] Processing CreateTablet for tablet de32f68d2ee24bd09e87ceae49cd7e81 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:11.372010 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32f68d2ee24bd09e87ceae49cd7e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.259133 17971 tablet_bootstrap.cc:654] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.274631 17972 raft_consensus.cc:359] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.379619 17972 raft_consensus.cc:385] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.380208 17972 raft_consensus.cc:740] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.381552 17972 consensus_queue.cc:260] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.261555 17910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1fbecc92da4549828813c3a7185b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.338569 17764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32f68d2ee24bd09e87ceae49cd7e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.369740 17889 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:11.328004 17981 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:11.397783 17978 raft_consensus.cc:493] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.397820 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3592a718f974e5f9f26e16b1942834f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.398324 17762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb98822af0c4e85aa127325141dce47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.402725 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffede44de4554930b68441a774c0dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.402678 17978 raft_consensus.cc:515] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:11.404785 17889 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 87258.7us
I20260709 12:30:11.408569 17768 tablet_service.cc:1511] Processing CreateTablet for tablet f292bf1109ec493d9111ad805d777330 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:11.410280 17768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f292bf1109ec493d9111ad805d777330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.409060 17686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c2238a9361414181c3aff69d2f9dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.415433 17978 leader_election.cc:290] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:11.415791 17689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb98822af0c4e85aa127325141dce47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.419876 17974 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:11.402729 17907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e7941692924de28d3fb53cf3cd833a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.425859 17909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ea582b2cc3418bac58a2e7ceb32649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.434623 17691 tablet_service.cc:1511] Processing CreateTablet for tablet f292bf1109ec493d9111ad805d777330 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:11.439322 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:11.440474 17972 ts_tablet_manager.cc:1434] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.197s	user 0.010s	sys 0.000s
I20260709 12:30:11.441701 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:11.444031 17687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e7941692924de28d3fb53cf3cd833a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.444844 17908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c2238a9361414181c3aff69d2f9dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.445839 17982 raft_consensus.cc:493] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.455070 17982 raft_consensus.cc:515] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:11.441833 17818 leader_election.cc:343] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.458439 17818 leader_election.cc:343] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.462001 17691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f292bf1109ec493d9111ad805d777330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.463199 17973 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:11.463972 17973 ts_tablet_manager.cc:1403] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.427s	user 0.072s	sys 0.081s
W20260709 12:30:11.462080 17889 leader_election.cc:343] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.462859 17911 tablet_service.cc:1511] Processing CreateTablet for tablet de32f68d2ee24bd09e87ceae49cd7e81 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:11.452833 17974 tablet_bootstrap.cc:654] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.502094 17916 tablet_service.cc:1511] Processing CreateTablet for tablet 173b1a41d3a1403e92f3345206f45510 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:11.428417 17981 raft_consensus.cc:2749] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.509521 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:11.503086 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32f68d2ee24bd09e87ceae49cd7e81. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.523718 17744 leader_election.cc:343] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.524324 17744 leader_election.cc:304] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:11.528417 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173b1a41d3a1403e92f3345206f45510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.547570 17972 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:11.563367 17837 tablet_service.cc:1511] Processing CreateTablet for tablet f292bf1109ec493d9111ad805d777330 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:11.565217 17837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f292bf1109ec493d9111ad805d777330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.565068 17973 raft_consensus.cc:359] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.566365 17973 raft_consensus.cc:385] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.567608 17973 raft_consensus.cc:740] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
W20260709 12:30:11.590705 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 67216.7us
I20260709 12:30:11.595520 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:11.597415 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:11.599541 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 9a387a07df404a3a8cdec9c49a791e6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.612565 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a387a07df404a3a8cdec9c49a791e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.602669 17978 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.614389 18008 raft_consensus.cc:2749] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.614799 17978 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.617026 17703 raft_consensus.cc:2468] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
W20260709 12:30:11.620229 17816 leader_election.cc:343] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.621088 17816 leader_election.cc:304] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:11.623039 17979 raft_consensus.cc:2749] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.604439 17684 tablet_service.cc:1511] Processing CreateTablet for tablet 173b1a41d3a1403e92f3345206f45510 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:11.595453 17765 tablet_service.cc:1511] Processing CreateTablet for tablet 173b1a41d3a1403e92f3345206f45510 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:11.606910 17982 leader_election.cc:290] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:11.607733 17972 tablet_bootstrap.cc:654] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.658972 17765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173b1a41d3a1403e92f3345206f45510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.625206 17816 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:11.608556 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:11.668371 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:11.668979 17914 tablet_service.cc:1511] Processing CreateTablet for tablet 9a387a07df404a3a8cdec9c49a791e6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.670588 17914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a387a07df404a3a8cdec9c49a791e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.670696 17979 raft_consensus.cc:2804] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.671177 17979 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.671625 17979 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:11.675704 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 50653us
W20260709 12:30:11.676733 17669 leader_election.cc:343] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.679077 17742 leader_election.cc:343] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.679899 17684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173b1a41d3a1403e92f3345206f45510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.702091 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:11.705030 17978 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:11.709295 17669 leader_election.cc:343] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.710846 17768 tablet_service.cc:1511] Processing CreateTablet for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.714174 17768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.728591 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:11.730053 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:11.732465 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:11.735633 17667 leader_election.cc:343] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.736516 17667 leader_election.cc:343] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.736955 17667 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:11.738993 17916 tablet_service.cc:1511] Processing CreateTablet for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.740561 17916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.740935 17837 tablet_service.cc:1511] Processing CreateTablet for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.742563 17837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc87abcb4d04cbb8196b5b8ecc06ec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.744962 17692 tablet_service.cc:1511] Processing CreateTablet for tablet 43b78182e2a746608309739058b74474 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:30:11.746688 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b78182e2a746608309739058b74474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.752343 17912 tablet_service.cc:1511] Processing CreateTablet for tablet 2f87c2ee9429444babb65d1c8a1ec7c1 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.753998 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f87c2ee9429444babb65d1c8a1ec7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.754315 17759 tablet_service.cc:1511] Processing CreateTablet for tablet 2f87c2ee9429444babb65d1c8a1ec7c1 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.590116 17973 consensus_queue.cc:260] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.756425 17978 raft_consensus.cc:2749] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.761168 17759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f87c2ee9429444babb65d1c8a1ec7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.761536 17913 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea37968d44c4ec5a4320301e4173d8b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.763212 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea37968d44c4ec5a4320301e4173d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.765443 17971 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:11.766332 17971 ts_tablet_manager.cc:1403] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.649s	user 0.015s	sys 0.015s
I20260709 12:30:11.770179 17971 raft_consensus.cc:359] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.772253 17685 tablet_service.cc:1511] Processing CreateTablet for tablet 9a387a07df404a3a8cdec9c49a791e6f (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.771092 17971 raft_consensus.cc:385] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.784443 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:11.777235 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a387a07df404a3a8cdec9c49a791e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.788409 17839 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea37968d44c4ec5a4320301e4173d8b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.793090 17839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea37968d44c4ec5a4320301e4173d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.786042 17688 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea37968d44c4ec5a4320301e4173d8b (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.794663 17688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea37968d44c4ec5a4320301e4173d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.759539 17979 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.797560 17850 raft_consensus.cc:2393] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:11.799746 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:11.800247 17667 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
W20260709 12:30:11.801678 17818 leader_election.cc:343] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.802307 17978 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:11.803028 17890 leader_election.cc:343] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.803625 17890 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:11.804986 17981 raft_consensus.cc:2749] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.808374 17763 tablet_service.cc:1511] Processing CreateTablet for tablet 90406ef14e034b2daea6aaf73e737745 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:30:11.810216 17763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90406ef14e034b2daea6aaf73e737745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.823335 17974 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:11.824085 17974 ts_tablet_manager.cc:1403] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.406s	user 0.011s	sys 0.006s
I20260709 12:30:11.825646 17834 tablet_service.cc:1511] Processing CreateTablet for tablet 90406ef14e034b2daea6aaf73e737745 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:30:11.828626 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90406ef14e034b2daea6aaf73e737745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.832589 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:11.787313 17971 raft_consensus.cc:740] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.835836 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 90406ef14e034b2daea6aaf73e737745 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:30:11.837620 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90406ef14e034b2daea6aaf73e737745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.839474 17979 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:11.843123 17693 tablet_service.cc:1511] Processing CreateTablet for tablet 2f87c2ee9429444babb65d1c8a1ec7c1 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.844959 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f87c2ee9429444babb65d1c8a1ec7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.772295 17766 tablet_service.cc:1511] Processing CreateTablet for tablet 43b78182e2a746608309739058b74474 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:30:11.847942 17972 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:11.848444 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b78182e2a746608309739058b74474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.848759 17972 ts_tablet_manager.cc:1403] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.307s	user 0.023s	sys 0.000s
I20260709 12:30:11.851594 17835 tablet_service.cc:1511] Processing CreateTablet for tablet 43b78182e2a746608309739058b74474 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:30:11.853765 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b78182e2a746608309739058b74474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.864411 17973 ts_tablet_manager.cc:1434] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.400s	user 0.008s	sys 0.007s
I20260709 12:30:11.868377 17973 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:11.875433 17971 consensus_queue.cc:260] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:11.878276 17971 ts_tablet_manager.cc:1434] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.112s	user 0.011s	sys 0.000s
I20260709 12:30:11.908622 17971 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:11.910126 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:11.912971 17666 leader_election.cc:343] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.913826 17666 leader_election.cc:304] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:11.885497 17972 raft_consensus.cc:359] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:11.915508 17972 raft_consensus.cc:385] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.915961 17972 raft_consensus.cc:740] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.917176 17972 consensus_queue.cc:260] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:11.924238 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:11.934626 17742 leader_election.cc:343] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.935438 17742 leader_election.cc:304] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:11.937191 17982 raft_consensus.cc:2749] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:11.921185 17586 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 83668.6us
I20260709 12:30:11.944290 18014 raft_consensus.cc:2749] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.949934 17974 raft_consensus.cc:359] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.991720 17974 raft_consensus.cc:385] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.992224 17974 raft_consensus.cc:740] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.993319 17974 consensus_queue.cc:260] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.969300 17841 tablet_service.cc:1511] Processing CreateTablet for tablet af2da24cf13b4824be213c5a60b39803 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:11.995187 17841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2da24cf13b4824be213c5a60b39803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.976044 17842 tablet_service.cc:1511] Processing CreateTablet for tablet bb2695d8605c4716960fd53bdc4eb899 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:30:11.997452 18012 raft_consensus.cc:493] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.998260 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2695d8605c4716960fd53bdc4eb899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.998054 18012 raft_consensus.cc:515] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:11.978046 17764 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0b2be2ba8b41579122c1d5d17a6b26 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:11.947912 17765 tablet_service.cc:1511] Processing CreateTablet for tablet af2da24cf13b4824be213c5a60b39803 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:12.000644 17764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0b2be2ba8b41579122c1d5d17a6b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.001696 17765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2da24cf13b4824be213c5a60b39803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.967725 17978 raft_consensus.cc:2749] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.000617 17973 tablet_bootstrap.cc:654] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.005142 17915 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0b2be2ba8b41579122c1d5d17a6b26 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:11.954432 17972 ts_tablet_manager.cc:1434] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.003s
I20260709 12:30:12.021353 17972 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:12.024979 17915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0b2be2ba8b41579122c1d5d17a6b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.049933 17971 tablet_bootstrap.cc:654] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.057200 17762 tablet_service.cc:1511] Processing CreateTablet for tablet 3e90ea3001724feca1d5658aef941cf0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:12.058934 17762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e90ea3001724feca1d5658aef941cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.063328 17907 tablet_service.cc:1511] Processing CreateTablet for tablet bb2695d8605c4716960fd53bdc4eb899 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:30:12.065092 17907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2695d8605c4716960fd53bdc4eb899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.067654 17701 raft_consensus.cc:2468] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:12.068492 17972 tablet_bootstrap.cc:654] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.070686 17816 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:12.072887 17979 raft_consensus.cc:2804] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.081732 17974 ts_tablet_manager.cc:1434] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.257s	user 0.011s	sys 0.006s
I20260709 12:30:12.083256 17974 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:12.047127 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 134444us
I20260709 12:30:12.111932 17692 tablet_service.cc:1511] Processing CreateTablet for tablet af2da24cf13b4824be213c5a60b39803 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:12.108611 17979 raft_consensus.cc:697] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:12.123823 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 3e90ea3001724feca1d5658aef941cf0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:12.124801 17913 tablet_service.cc:1511] Processing CreateTablet for tablet 3e90ea3001724feca1d5658aef941cf0 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:12.122026 18012 leader_election.cc:290] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.128883 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.129151 17689 tablet_service.cc:1511] Processing CreateTablet for tablet 1607aea0063f40f0a8d248cf7860749d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:30:12.140278 17768 tablet_service.cc:1511] Processing CreateTablet for tablet 1607aea0063f40f0a8d248cf7860749d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:30:12.143347 17768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1607aea0063f40f0a8d248cf7860749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.143998 17835 tablet_service.cc:1511] Processing CreateTablet for tablet 1607aea0063f40f0a8d248cf7860749d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 12:30:12.145328 17818 leader_election.cc:343] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.117128 17691 tablet_service.cc:1511] Processing CreateTablet for tablet bb2695d8605c4716960fd53bdc4eb899 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 12:30:12.127015 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:12.132707 17690 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0b2be2ba8b41579122c1d5d17a6b26 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:12.147718 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2da24cf13b4824be213c5a60b39803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.148566 17689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1607aea0063f40f0a8d248cf7860749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.149180 17913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e90ea3001724feca1d5658aef941cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.149986 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0b2be2ba8b41579122c1d5d17a6b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.152879 17691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2695d8605c4716960fd53bdc4eb899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.137511 17974 tablet_bootstrap.cc:654] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.157905 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e90ea3001724feca1d5658aef941cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.162725 17835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1607aea0063f40f0a8d248cf7860749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.177352 17912 tablet_service.cc:1511] Processing CreateTablet for tablet 4bda730805b64dda88c94c3072246211 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:12.179019 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bda730805b64dda88c94c3072246211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.181073 17979 consensus_queue.cc:237] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.185532 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:12.187604 17815 leader_election.cc:343] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.188181 17815 leader_election.cc:304] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:12.190110 17766 tablet_service.cc:1511] Processing CreateTablet for tablet 4bda730805b64dda88c94c3072246211 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:12.191843 17766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bda730805b64dda88c94c3072246211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.194450 17834 tablet_service.cc:1511] Processing CreateTablet for tablet d3be2bb16cb74f8fb343216495dc07a3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:12.195451 18012 raft_consensus.cc:2749] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.197808 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3be2bb16cb74f8fb343216495dc07a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.209306 17908 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34038 (request call id 27) took 1036 ms. Trace:
I20260709 12:30:12.210224 17973 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:12.210960 17973 ts_tablet_manager.cc:1403] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.343s	user 0.019s	sys 0.039s
I20260709 12:30:12.212832 17981 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.213557 17981 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.216066 17907 tablet_service.cc:1511] Processing CreateTablet for tablet d3be2bb16cb74f8fb343216495dc07a3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:12.218674 17907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3be2bb16cb74f8fb343216495dc07a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.210700 17687 tablet_service.cc:1511] Processing CreateTablet for tablet 4bda730805b64dda88c94c3072246211 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:12.228291 17687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bda730805b64dda88c94c3072246211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.254675 17981 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.255584 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:12.257642 17974 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
W20260709 12:30:12.257870 17890 leader_election.cc:343] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.258464 17974 ts_tablet_manager.cc:1403] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.176s	user 0.037s	sys 0.008s
I20260709 12:30:12.259796 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.272681 18013 raft_consensus.cc:493] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.273382 18013 raft_consensus.cc:515] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.274403 17972 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
W20260709 12:30:12.274900 17890 leader_election.cc:343] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.273962 17763 tablet_service.cc:1511] Processing CreateTablet for tablet 4460e907889c4b0d9b234ccf0f574eaf (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:12.275187 17972 ts_tablet_manager.cc:1403] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.254s	user 0.010s	sys 0.017s
I20260709 12:30:12.275532 17890 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:12.282980 17981 raft_consensus.cc:2749] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.209950 17908 rpcz_store.cc:276] 0709 12:30:11.172724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:30:11.179719 (+  6995us) service_pool.cc:224] Handling call
0709 12:30:11.268111 (+ 88392us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:30:11.268244 (+   133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:30:11.920297 (+652053us) tablet_metadata.cc:665] Metadata flushed
0709 12:30:12.208622 (+288325us) tablet_replica.cc:206] Creating consensus instance
0709 12:30:12.209222 (+   600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:30:12.276885 17842 tablet_service.cc:1511] Processing CreateTablet for tablet 4460e907889c4b0d9b234ccf0f574eaf (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:12.288964 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4460e907889c4b0d9b234ccf0f574eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.279265 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:12.307636 17669 leader_election.cc:343] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.279997 17763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4460e907889c4b0d9b234ccf0f574eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.276770 17693 tablet_service.cc:1511] Processing CreateTablet for tablet d3be2bb16cb74f8fb343216495dc07a3 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:12.286620 17972 raft_consensus.cc:359] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.278187 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:12.318796 17666 leader_election.cc:343] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.317982 17908 tablet_service.cc:1511] Processing CreateTablet for tablet 4460e907889c4b0d9b234ccf0f574eaf (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:12.319504 17666 leader_election.cc:304] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:12.320488 17978 raft_consensus.cc:2749] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.314254 17693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3be2bb16cb74f8fb343216495dc07a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.322997 17974 raft_consensus.cc:359] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.330854 17974 raft_consensus.cc:385] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.331406 17974 raft_consensus.cc:740] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.278257 18013 leader_election.cc:290] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:12.332623 17974 consensus_queue.cc:260] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.336010 17908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4460e907889c4b0d9b234ccf0f574eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.316449 17972 raft_consensus.cc:385] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.347072 17972 raft_consensus.cc:740] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.348181 17972 consensus_queue.cc:260] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.351459 17972 ts_tablet_manager.cc:1434] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.002s
I20260709 12:30:12.352782 17972 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:12.355640 17974 ts_tablet_manager.cc:1434] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.097s	user 0.010s	sys 0.000s
I20260709 12:30:12.290350 17971 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:12.288838 17973 raft_consensus.cc:359] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.363768 17973 raft_consensus.cc:385] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.363934 17971 ts_tablet_manager.cc:1403] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.456s	user 0.020s	sys 0.016s
I20260709 12:30:12.364213 17973 raft_consensus.cc:740] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.370978 17685 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc8a907276843128987dab0a2f52235 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:12.373250 17685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc8a907276843128987dab0a2f52235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.381699 17972 tablet_bootstrap.cc:654] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.383285 17762 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc8a907276843128987dab0a2f52235 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:12.386909 17762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc8a907276843128987dab0a2f52235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.389107 17834 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc8a907276843128987dab0a2f52235 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:12.395437 17834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc8a907276843128987dab0a2f52235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.399650 17971 raft_consensus.cc:359] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.403618 17982 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.404328 17971 raft_consensus.cc:385] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.404716 17971 raft_consensus.cc:740] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.404472 17982 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.408763 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:12.410723 17742 leader_election.cc:343] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.414705 17982 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.415676 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.416052 17974 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:12.416580 17925 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:12.418177 17744 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:12.419656 17907 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae4ca22644a4128bc2662c6bb502884 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:12.420681 17982 raft_consensus.cc:2804] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.421015 17907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae4ca22644a4128bc2662c6bb502884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.421177 17982 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.421620 17982 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.365352 17973 consensus_queue.cc:260] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.440562 17768 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae4ca22644a4128bc2662c6bb502884 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:12.442332 17768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae4ca22644a4128bc2662c6bb502884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.442433 17912 tablet_service.cc:1511] Processing CreateTablet for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:12.443461 17687 tablet_service.cc:1511] Processing CreateTablet for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:12.444226 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.445066 17687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.446487 17599 catalog_manager.cc:5697] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.448474 17836 tablet_service.cc:1511] Processing CreateTablet for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:12.450079 17836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b6bcf23aeb44b0b8aea5466500e7f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.461903 17982 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.414446 17971 consensus_queue.cc:260] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.484252 17971 ts_tablet_manager.cc:1434] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.119s	user 0.008s	sys 0.002s
I20260709 12:30:12.493153 17971 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:12.502993 17972 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:12.503749 17972 ts_tablet_manager.cc:1403] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.151s	user 0.007s	sys 0.022s
I20260709 12:30:12.504823 17974 tablet_bootstrap.cc:654] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.513238 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:12.513864 17979 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.513988 17982 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.514369 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:12.514837 17982 raft_consensus.cc:493] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.514490 17979 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.515106 17925 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:12.515213 17742 leader_election.cc:343] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.519924 17692 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae4ca22644a4128bc2662c6bb502884 (DEFAULT_TABLE table=test-workload [id=1fdf5cdfc0e54c8091501108a7b351d5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:12.521869 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae4ca22644a4128bc2662c6bb502884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.523536 17973 ts_tablet_manager.cc:1434] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.312s	user 0.022s	sys 0.001s
I20260709 12:30:12.523798 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.525089 17701 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:12.528899 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.529577 17816 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:12.530638 17973 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:12.525163 17979 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.530735 18012 raft_consensus.cc:2804] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.532138 18012 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.532594 18012 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.525266 17972 raft_consensus.cc:359] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.533540 17972 raft_consensus.cc:385] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.533928 17972 raft_consensus.cc:740] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
W20260709 12:30:12.531461 17818 leader_election.cc:343] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.536109 17972 consensus_queue.cc:260] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.550738 17971 tablet_bootstrap.cc:654] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.551919 17973 tablet_bootstrap.cc:654] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.553972 17974 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:12.554672 17972 ts_tablet_manager.cc:1434] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.005s
I20260709 12:30:12.515450 17982 raft_consensus.cc:515] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.568352 17982 leader_election.cc:290] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.569684 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.572456 17972 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:12.574636 17744 leader_election.cc:343] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.576753 17925 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:12.578357 18012 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.581916 17974 ts_tablet_manager.cc:1403] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.004s
I20260709 12:30:12.584915 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.585810 17701 raft_consensus.cc:2468] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:12.587352 17742 leader_election.cc:304] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:12.587989 17744 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:12.588886 17982 raft_consensus.cc:2804] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.589097 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:12.589413 17982 raft_consensus.cc:493] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.590860 17982 raft_consensus.cc:3060] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.589458 18023 raft_consensus.cc:2804] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:30:12.602346 17818 leader_election.cc:343] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.586845 17974 raft_consensus.cc:359] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.615636 18012 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.615873 17974 raft_consensus.cc:385] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.617133 17974 raft_consensus.cc:740] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.619445 17974 consensus_queue.cc:260] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.613319 17979 consensus_queue.cc:1048] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:12.622511 17973 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:12.623185 17973 ts_tablet_manager.cc:1403] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.098s	user 0.027s	sys 0.014s
I20260709 12:30:12.614482 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:12.624068 17701 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.626856 17973 raft_consensus.cc:359] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.628135 17973 raft_consensus.cc:385] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.628659 17973 raft_consensus.cc:740] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.630126 17973 consensus_queue.cc:260] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.615038 17982 raft_consensus.cc:515] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.621833 18023 raft_consensus.cc:697] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:12.635154 17973 ts_tablet_manager.cc:1434] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:30:12.636948 17971 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:12.640200 17971 ts_tablet_manager.cc:1403] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.147s	user 0.010s	sys 0.035s
I20260709 12:30:12.640813 17973 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:12.640398 18023 consensus_queue.cc:237] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.647205 17974 ts_tablet_manager.cc:1434] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.001s
I20260709 12:30:12.653702 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:12.656008 17744 leader_election.cc:343] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.615574 17972 tablet_bootstrap.cc:654] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.653688 17973 tablet_bootstrap.cc:654] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.657051 18013 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.657709 18013 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.666280 17974 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:12.666436 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:12.668078 17702 raft_consensus.cc:3060] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.673519 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:12.675535 17974 tablet_bootstrap.cc:654] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:12.676347 17667 leader_election.cc:343] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.667204 17600 catalog_manager.cc:5697] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.681882 17972 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:12.682551 17972 ts_tablet_manager.cc:1403] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.005s
I20260709 12:30:12.686820 17972 raft_consensus.cc:359] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.687716 17972 raft_consensus.cc:385] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.688062 17972 raft_consensus.cc:740] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.689162 17972 consensus_queue.cc:260] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.692400 17972 ts_tablet_manager.cc:1434] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:30:12.693807 17972 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:12.694015 17981 raft_consensus.cc:493] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.694948 17981 raft_consensus.cc:515] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.697697 18013 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.697842 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.698837 17925 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:12.700735 17669 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:12.702068 18013 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.702613 18013 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.703087 18013 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.703300 17972 tablet_bootstrap.cc:654] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.709296 17981 leader_election.cc:290] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.709810 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:12.666563 17982 leader_election.cc:290] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.666708 17701 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:12.711081 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.714988 17702 raft_consensus.cc:2468] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:12.716992 17816 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:12.718484 17742 leader_election.cc:304] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:12.719439 17979 raft_consensus.cc:2804] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.720039 17979 raft_consensus.cc:697] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:12.720890 17979 consensus_queue.cc:237] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:12.725232 17890 leader_election.cc:343] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.731489 17777 raft_consensus.cc:2468] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:12.733274 17889 leader_election.cc:304] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:12.734583 17981 raft_consensus.cc:2804] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.735030 17981 raft_consensus.cc:493] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.735386 17981 raft_consensus.cc:3060] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.739574 17982 raft_consensus.cc:2804] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.740649 17982 raft_consensus.cc:697] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:12.741726 17974 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:12.742432 17974 ts_tablet_manager.cc:1403] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.076s	user 0.005s	sys 0.008s
I20260709 12:30:12.742036 17982 consensus_queue.cc:237] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.752048 17981 raft_consensus.cc:515] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.754756 17978 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.754065 17599 catalog_manager.cc:5697] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:12.755396 17978 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.757632 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:12.758369 17777 raft_consensus.cc:3060] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.759177 17978 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:12.759658 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.760779 17925 raft_consensus.cc:2468] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:12.763048 17669 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:12.764227 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:12.764627 17978 raft_consensus.cc:2804] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.765337 17978 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.765734 17978 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:12.771323 17666 leader_election.cc:343] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.772573 17971 raft_consensus.cc:359] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.773463 17971 raft_consensus.cc:385] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.773773 17971 raft_consensus.cc:740] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.774737 17971 consensus_queue.cc:260] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:12.777063 17818 consensus_peers.cc:597] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:12.777522 17978 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.779280 17974 raft_consensus.cc:359] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.780223 17974 raft_consensus.cc:385] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.783047 17777 raft_consensus.cc:2468] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:12.786055 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:12.786289 17889 leader_election.cc:304] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
W20260709 12:30:12.789003 17890 leader_election.cc:343] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.784772 17981 leader_election.cc:290] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:12.797595 17981 raft_consensus.cc:2804] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.798184 17981 raft_consensus.cc:697] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:12.799145 17981 consensus_queue.cc:237] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.814740 17972 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:12.780586 17974 raft_consensus.cc:740] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.818264 17972 ts_tablet_manager.cc:1403] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.125s	user 0.014s	sys 0.004s
I20260709 12:30:12.819617 17982 raft_consensus.cc:493] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.820284 17982 raft_consensus.cc:515] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.821506 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:12.822279 17925 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.822556 17973 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:12.822865 17982 leader_election.cc:290] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:12.823273 17973 ts_tablet_manager.cc:1403] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.183s	user 0.058s	sys 0.045s
I20260709 12:30:12.826884 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.827308 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:12.827718 17974 consensus_queue.cc:260] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:12.829515 17742 leader_election.cc:343] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.827850 17973 raft_consensus.cc:359] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.830242 17973 raft_consensus.cc:385] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.830689 17973 raft_consensus.cc:740] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.800405 17599 catalog_manager.cc:5697] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:12.831266 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:12.831578 17978 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:12.833082 17666 leader_election.cc:343] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.803996 18013 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.833840 17973 consensus_queue.cc:260] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.835156 17850 raft_consensus.cc:2468] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:12.837153 18013 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.837555 17742 leader_election.cc:304] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:12.837877 17974 ts_tablet_manager.cc:1434] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.095s	user 0.012s	sys 0.000s
I20260709 12:30:12.839293 17974 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:12.839605 17982 raft_consensus.cc:2804] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.839743 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:12.840195 17982 raft_consensus.cc:493] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.840674 17982 raft_consensus.cc:3060] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.818269 17971 ts_tablet_manager.cc:1434] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.178s	user 0.007s	sys 0.000s
W20260709 12:30:12.844973 17667 leader_election.cc:343] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.845141 17971 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:12.838951 17925 raft_consensus.cc:2468] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:12.848147 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:12.849143 17926 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.849552 17669 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:12.851190 18013 raft_consensus.cc:2804] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.852146 18013 raft_consensus.cc:697] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:12.853556 18013 consensus_queue.cc:237] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:12.871896 17973 ts_tablet_manager.cc:1434] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.048s	user 0.015s	sys 0.005s
I20260709 12:30:12.873634 17973 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:12.882121 17973 tablet_bootstrap.cc:654] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.883191 17926 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:12.885015 17972 raft_consensus.cc:359] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.885982 17972 raft_consensus.cc:385] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.886337 17972 raft_consensus.cc:740] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.891713 17669 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:12.895283 17978 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.896437 17978 raft_consensus.cc:697] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:12.894762 17599 catalog_manager.cc:5697] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.897895 17978 consensus_queue.cc:237] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.903259 17972 consensus_queue.cc:260] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.906082 17972 ts_tablet_manager.cc:1434] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.087s	user 0.010s	sys 0.000s
I20260709 12:30:12.910287 17971 tablet_bootstrap.cc:654] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.916894 17974 tablet_bootstrap.cc:654] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.918721 17972 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:12.920763 17600 catalog_manager.cc:5697] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.894255 17982 raft_consensus.cc:515] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.929736 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:12.931351 17973 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
W20260709 12:30:12.931532 17742 leader_election.cc:343] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.932096 17973 ts_tablet_manager.cc:1403] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.059s	user 0.027s	sys 0.013s
I20260709 12:30:12.934648 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:12.935655 17852 raft_consensus.cc:3060] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.939579 17973 raft_consensus.cc:359] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.940472 17973 raft_consensus.cc:385] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.940800 17973 raft_consensus.cc:740] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.941710 17973 consensus_queue.cc:260] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:12.948580 18023 raft_consensus.cc:493] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.949381 18023 raft_consensus.cc:515] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.949908 17973 ts_tablet_manager.cc:1434] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:30:12.951505 17973 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:12.951722 17598 catalog_manager.cc:5697] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:12.955144 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:12.956691 17982 leader_election.cc:290] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:12.960249 17852 raft_consensus.cc:2468] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
W20260709 12:30:12.960615 17744 leader_election.cc:343] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.962309 18023 leader_election.cc:290] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:12.964298 17972 tablet_bootstrap.cc:654] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.964938 17742 leader_election.cc:304] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:12.965026 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:12.966361 17974 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:12.966533 18023 raft_consensus.cc:2804] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.967190 17974 ts_tablet_manager.cc:1403] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.128s	user 0.033s	sys 0.017s
I20260709 12:30:12.967267 18023 raft_consensus.cc:697] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:12.968559 18023 consensus_queue.cc:237] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.971692 17974 raft_consensus.cc:359] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.980336 17974 raft_consensus.cc:385] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:12.973462 17742 leader_election.cc:343] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.981544 17742 leader_election.cc:304] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:12.981968 17981 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.982573 17981 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.978659 17972 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:12.983771 17972 ts_tablet_manager.cc:1403] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.002s
I20260709 12:30:12.985091 17981 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:12.984678 17597 catalog_manager.cc:5697] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.975327 17973 tablet_bootstrap.cc:654] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.982898 17982 raft_consensus.cc:2749] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.987083 17972 raft_consensus.cc:359] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.987973 17972 raft_consensus.cc:385] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.987813 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:12.988360 17972 raft_consensus.cc:740] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.988855 17702 raft_consensus.cc:2393] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:12.989331 17972 consensus_queue.cc:260] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:12.989744 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:12.976816 17971 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:12.991557 17890 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:12.992059 17971 ts_tablet_manager.cc:1403] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.147s	user 0.018s	sys 0.000s
I20260709 12:30:12.993112 17981 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.996122 17971 raft_consensus.cc:359] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:12.997097 17971 raft_consensus.cc:385] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.997463 17971 raft_consensus.cc:740] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.998394 17971 consensus_queue.cc:260] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.006614 18013 raft_consensus.cc:493] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.006078 17981 raft_consensus.cc:2749] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.000391 17972 ts_tablet_manager.cc:1434] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:30:12.980938 17974 raft_consensus.cc:740] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.000097 17973 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:13.011705 18013 raft_consensus.cc:515] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.013610 17971 ts_tablet_manager.cc:1434] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:30:13.017426 17971 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:13.014308 17973 ts_tablet_manager.cc:1403] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.009s
I20260709 12:30:13.019709 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.020588 17852 raft_consensus.cc:2393] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:13.013126 17972 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:13.022456 17973 raft_consensus.cc:359] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.021709 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20637ddae560402b81988ca96fc61b1e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.023413 17973 raft_consensus.cc:385] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.023767 17973 raft_consensus.cc:740] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.024688 17973 consensus_queue.cc:260] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.025172 17666 leader_election.cc:304] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.013902 17974 consensus_queue.cc:260] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.026355 17978 raft_consensus.cc:3060] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.030184 17972 tablet_bootstrap.cc:654] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.030792 17981 raft_consensus.cc:493] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.032037 17981 raft_consensus.cc:515] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.033253 17974 ts_tablet_manager.cc:1434] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.065s	user 0.000s	sys 0.014s
I20260709 12:30:13.034850 17974 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:13.018122 18013 leader_election.cc:290] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.036010 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:13.038043 17889 leader_election.cc:343] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.041083 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.042039 17852 raft_consensus.cc:2468] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:13.042480 17978 raft_consensus.cc:2749] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.043699 17890 leader_election.cc:304] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.046206 17973 ts_tablet_manager.cc:1434] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 12:30:13.048152 17973 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:13.052309 18042 raft_consensus.cc:2804] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.054838 18042 raft_consensus.cc:493] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.055063 18012 raft_consensus.cc:493] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.055449 18042 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.055974 18012 raft_consensus.cc:515] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.056759 17981 leader_election.cc:290] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.059671 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.060595 17702 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:13.060739 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.061651 17777 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:13.063290 17816 leader_election.cc:304] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.048722 17974 tablet_bootstrap.cc:654] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.064599 18047 raft_consensus.cc:2804] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.065181 18047 raft_consensus.cc:493] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.065610 18047 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.069974 18042 raft_consensus.cc:515] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.072826 18012 leader_election.cc:290] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.073849 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
W20260709 12:30:13.075496 17889 leader_election.cc:343] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.075799 17982 consensus_queue.cc:1048] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.079440 18042 leader_election.cc:290] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.085021 17974 tablet_bootstrap.cc:492] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:13.078977 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:13.085863 17974 ts_tablet_manager.cc:1403] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.001s
I20260709 12:30:13.086046 17852 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.086261 17973 tablet_bootstrap.cc:654] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.078958 18047 raft_consensus.cc:515] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.054179 17971 tablet_bootstrap.cc:654] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.089893 17974 raft_consensus.cc:359] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.090938 17974 raft_consensus.cc:385] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.091383 17974 raft_consensus.cc:740] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.092430 17974 consensus_queue.cc:260] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.094901 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:13.095301 18047 leader_election.cc:290] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.095649 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:13.095938 17702 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.096429 17777 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.103631 17972 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:13.104357 17972 ts_tablet_manager.cc:1403] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.091s	user 0.024s	sys 0.000s
I20260709 12:30:13.110026 17852 raft_consensus.cc:2468] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:13.112475 17890 leader_election.cc:304] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.114667 18042 raft_consensus.cc:2804] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.115530 18042 raft_consensus.cc:697] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:13.116634 18042 consensus_queue.cc:237] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.117946 17702 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.119539 17816 leader_election.cc:304] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.120743 18047 raft_consensus.cc:2804] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.121434 18047 raft_consensus.cc:697] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:13.122258 17974 ts_tablet_manager.cc:1434] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:30:13.122430 18047 consensus_queue.cc:237] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.130059 17972 raft_consensus.cc:359] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.132926 17972 raft_consensus.cc:385] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.136113 17972 raft_consensus.cc:740] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.137455 17972 consensus_queue.cc:260] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.141122 17971 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:13.142921 17971 ts_tablet_manager.cc:1403] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.126s	user 0.020s	sys 0.009s
I20260709 12:30:13.144896 17972 ts_tablet_manager.cc:1434] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:30:13.145891 17978 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.146283 17972 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:13.146533 17978 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.147586 18047 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.148327 18047 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.151139 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.152112 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.152889 17974 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:13.153403 17666 leader_election.cc:343] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:13.161002 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:30:13.161240 17667 leader_election.cc:343] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.164328 17667 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.149204 17978 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.155974 18012 raft_consensus.cc:493] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.165423 18013 raft_consensus.cc:2749] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.165729 18012 raft_consensus.cc:515] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.158694 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.166852 17601 catalog_manager.cc:5697] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.168031 17925 raft_consensus.cc:2393] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate b96ea338e2a940f9b606d5d0fdd10e85 in this term.
I20260709 12:30:13.170594 17971 raft_consensus.cc:359] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:13.174566 17742 consensus_peers.cc:597] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.157011 17598 catalog_manager.cc:5697] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.186558 17971 raft_consensus.cc:385] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.191207 17971 raft_consensus.cc:740] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.187132 17972 tablet_bootstrap.cc:654] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.192561 17971 consensus_queue.cc:260] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.193590 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.195686 17816 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:13.197041 18046 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.196626 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.197743 18012 leader_election.cc:290] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.198206 17702 raft_consensus.cc:2468] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:13.158748 17777 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.205413 17974 tablet_bootstrap.cc:654] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.210498 17816 leader_election.cc:304] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.213825 17979 raft_consensus.cc:2804] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.214373 17979 raft_consensus.cc:493] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.214741 17979 raft_consensus.cc:3060] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.216753 18023 raft_consensus.cc:493] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.217806 18023 raft_consensus.cc:515] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.222061 18023 leader_election.cc:290] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:13.222123 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.228899 17852 raft_consensus.cc:2393] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 1: Already voted for candidate 30d17e55b097491f895c84a3443cb250 in this term.
I20260709 12:30:13.197301 18047 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:13.223657 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.223711 17979 raft_consensus.cc:515] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.195683 17971 ts_tablet_manager.cc:1434] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 12:30:13.230852 18046 raft_consensus.cc:2749] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.232530 17744 leader_election.cc:304] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:13.233654 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:13.234098 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:13.234604 17702 raft_consensus.cc:3060] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.235184 17778 raft_consensus.cc:3060] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.236899 17973 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:13.237515 17973 ts_tablet_manager.cc:1403] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.190s	user 0.019s	sys 0.014s
I20260709 12:30:13.226234 18033 consensus_queue.cc:1048] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.205466 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7db027e2c4f5186a0d0c50ed0d6d6" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.227730 18012 consensus_queue.cc:1048] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:13.234475 18023 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.245579 17979 leader_election.cc:290] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.246678 17971 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:13.225688 17972 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:13.247704 17972 ts_tablet_manager.cc:1403] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.102s	user 0.018s	sys 0.009s
I20260709 12:30:13.255620 17702 raft_consensus.cc:2468] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.257413 17972 raft_consensus.cc:359] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.257887 17816 leader_election.cc:304] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.258410 17972 raft_consensus.cc:385] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.258880 17972 raft_consensus.cc:740] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.259387 17979 raft_consensus.cc:2804] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.260546 17974 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:13.261056 17979 raft_consensus.cc:697] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:13.261785 17974 ts_tablet_manager.cc:1403] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.003s
I20260709 12:30:13.241451 17973 raft_consensus.cc:359] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.262814 17979 consensus_queue.cc:237] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.262900 17973 raft_consensus.cc:385] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.260712 17971 tablet_bootstrap.cc:654] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.271895 18023 raft_consensus.cc:2749] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.265872 17974 raft_consensus.cc:359] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.281847 17974 raft_consensus.cc:385] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.282368 17974 raft_consensus.cc:740] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.266280 17973 raft_consensus.cc:740] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.283440 17974 consensus_queue.cc:260] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.283856 17973 consensus_queue.cc:260] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.275354 17972 consensus_queue.cc:260] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.286340 17979 consensus_queue.cc:1048] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.287302 17972 ts_tablet_manager.cc:1434] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.003s
I20260709 12:30:13.286687 17973 ts_tablet_manager.cc:1434] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 12:30:13.288836 17973 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:13.288981 17601 catalog_manager.cc:5697] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.268169 17778 raft_consensus.cc:2468] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.300673 17972 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:13.310334 17890 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.304333 17974 ts_tablet_manager.cc:1434] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 12:30:13.312305 17974 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:13.314915 17744 consensus_peers.cc:597] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.317214 17973 tablet_bootstrap.cc:654] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.326431 17978 consensus_queue.cc:1048] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.330209 17972 tablet_bootstrap.cc:654] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.340483 17978 consensus_queue.cc:1048] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.347502 17971 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:13.348362 17971 ts_tablet_manager.cc:1403] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.102s	user 0.031s	sys 0.008s
I20260709 12:30:13.352300 17971 raft_consensus.cc:359] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.353291 17971 raft_consensus.cc:385] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.353669 17971 raft_consensus.cc:740] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.358800 17974 tablet_bootstrap.cc:654] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.364202 18013 consensus_queue.cc:1048] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.359539 17971 consensus_queue.cc:260] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.373656 17973 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:13.374336 17973 ts_tablet_manager.cc:1403] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.005s
I20260709 12:30:13.376410 17978 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.380170 18042 consensus_queue.cc:1048] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.386777 17971 ts_tablet_manager.cc:1434] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 12:30:13.377400 17978 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.390812 17973 raft_consensus.cc:359] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.391781 17973 raft_consensus.cc:385] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.392197 17973 raft_consensus.cc:740] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.391841 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.393116 17978 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.395323 17971 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
W20260709 12:30:13.400290 17666 leader_election.cc:343] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.402352 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.405452 17924 raft_consensus.cc:2468] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:13.393142 17973 consensus_queue.cc:260] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.410923 18023 consensus_queue.cc:1048] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.411077 17669 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.412057 17973 ts_tablet_manager.cc:1434] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 12:30:13.413429 17973 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:13.416986 17978 raft_consensus.cc:2804] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.417745 17978 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.418298 17978 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.434409 17972 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:13.435395 17972 ts_tablet_manager.cc:1403] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.135s	user 0.016s	sys 0.012s
I20260709 12:30:13.440363 17972 raft_consensus.cc:359] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.441514 17972 raft_consensus.cc:385] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.441956 17972 raft_consensus.cc:740] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.443851 17972 consensus_queue.cc:260] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.471517 17971 tablet_bootstrap.cc:654] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.471843 18046 raft_consensus.cc:493] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.472499 18046 raft_consensus.cc:515] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.474105 17974 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:13.474722 17974 ts_tablet_manager.cc:1403] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.163s	user 0.018s	sys 0.007s
I20260709 12:30:13.478008 17974 raft_consensus.cc:359] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.478797 17974 raft_consensus.cc:385] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.479547 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.480818 17777 raft_consensus.cc:2468] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:13.482502 17815 leader_election.cc:304] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:13.485023 18047 raft_consensus.cc:2804] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.485282 17973 tablet_bootstrap.cc:654] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.485787 18047 raft_consensus.cc:493] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.486315 18047 raft_consensus.cc:3060] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.492883 18046 leader_election.cc:290] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:13.493510 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:13.495260 17818 leader_election.cc:343] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.479097 17974 raft_consensus.cc:740] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.498863 18047 raft_consensus.cc:515] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.500339 17974 consensus_queue.cc:260] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.503713 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:13.504971 17778 raft_consensus.cc:3060] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.507992 17974 ts_tablet_manager.cc:1434] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:30:13.509346 17974 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:13.509506 18013 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.510178 18013 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:13.524490 17669 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 50704.3us
I20260709 12:30:13.541013 18042 consensus_queue.cc:1048] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.542301 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:13.542680 18023 consensus_queue.cc:1048] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:13.542248 18047 leader_election.cc:290] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:13.544328 17818 leader_election.cc:343] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.548275 17778 raft_consensus.cc:2468] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.557307 17973 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:13.558172 17973 ts_tablet_manager.cc:1403] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.145s	user 0.010s	sys 0.002s
I20260709 12:30:13.559826 17815 leader_election.cc:304] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:13.561388 18047 raft_consensus.cc:2804] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.562054 18047 raft_consensus.cc:697] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:13.563105 18047 consensus_queue.cc:237] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.582139 18042 consensus_queue.cc:1048] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.602581 17973 raft_consensus.cc:359] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.613795 17972 ts_tablet_manager.cc:1434] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.178s	user 0.009s	sys 0.034s
I20260709 12:30:13.613864 17973 raft_consensus.cc:385] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.614372 17973 raft_consensus.cc:740] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.615191 17973 consensus_queue.cc:260] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.622084 17974 tablet_bootstrap.cc:654] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.623021 17972 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:13.633126 17666 consensus_peers.cc:597] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:13.633818 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 48239.9us
I20260709 12:30:13.599817 18046 raft_consensus.cc:493] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.637269 18012 consensus_queue.cc:1048] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.638068 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.637539 18046 raft_consensus.cc:515] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.640867 17973 ts_tablet_manager.cc:1434] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.007s
I20260709 12:30:13.649612 17973 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:13.650631 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.661801 18046 leader_election.cc:290] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:13.662971 17852 raft_consensus.cc:2468] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:13.663615 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.665903 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.646898 18013 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.612987 17978 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.667058 17702 raft_consensus.cc:2468] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:13.669957 18033 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:13.655478 17666 leader_election.cc:343] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:13.671330 17818 leader_election.cc:343] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.672590 17667 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.673033 17816 leader_election.cc:304] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:13.673858 18089 raft_consensus.cc:2804] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.674279 18089 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.674652 18089 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.678730 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:13.679592 17926 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.670626 18033 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.686795 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.687885 17852 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:13.686653 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.682327 18033 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.701439 17973 tablet_bootstrap.cc:654] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.690403 18047 raft_consensus.cc:2804] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.707362 17742 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:13.714175 18033 raft_consensus.cc:2804] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.714684 18033 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.692165 17702 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:13.715075 18033 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.727451 17971 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:13.728363 17971 ts_tablet_manager.cc:1403] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.333s	user 0.023s	sys 0.015s
I20260709 12:30:13.734546 17972 tablet_bootstrap.cc:654] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.735325 17978 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.737114 17597 catalog_manager.cc:5697] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.739499 17971 raft_consensus.cc:359] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.740445 17971 raft_consensus.cc:385] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.740150 18046 consensus_queue.cc:1048] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:30:13.711769 18047 raft_consensus.cc:493] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.741577 17971 raft_consensus.cc:740] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.748601 17971 consensus_queue.cc:260] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.752050 17971 ts_tablet_manager.cc:1434] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 12:30:13.746057 18047 raft_consensus.cc:3060] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.755421 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:13.762182 17776 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.759989 17981 raft_consensus.cc:493] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.765256 17981 raft_consensus.cc:515] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.769258 17981 leader_election.cc:290] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:13.770315 18089 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.777473 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:13.784402 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.784754 18012 consensus_queue.cc:1048] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.785480 17703 raft_consensus.cc:2468] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:13.786468 18089 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.759985 18033 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.789641 17890 leader_election.cc:304] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.791281 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:13.792119 17703 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.784530 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
W20260709 12:30:13.795053 17666 leader_election.cc:343] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.796613 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:13.797370 17850 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.791924 18042 raft_consensus.cc:2804] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.798714 18042 raft_consensus.cc:493] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.800084 17974 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:13.800621 17974 ts_tablet_manager.cc:1403] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.292s	user 0.033s	sys 0.037s
I20260709 12:30:13.804522 17971 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:13.806613 17973 tablet_bootstrap.cc:492] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:13.807554 17973 ts_tablet_manager.cc:1403] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.158s	user 0.015s	sys 0.007s
I20260709 12:30:13.808991 17972 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:13.799376 18042 raft_consensus.cc:3060] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.809886 17972 ts_tablet_manager.cc:1403] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.195s	user 0.013s	sys 0.007s
I20260709 12:30:13.807071 18047 raft_consensus.cc:515] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.812016 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:13.812772 17851 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.802675 18033 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:13.813862 17972 raft_consensus.cc:359] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.815687 17972 raft_consensus.cc:385] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.816149 17972 raft_consensus.cc:740] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.816915 17972 consensus_queue.cc:260] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.817313 17981 raft_consensus.cc:493] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.817544 17852 raft_consensus.cc:2393] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:13.819392 17981 raft_consensus.cc:515] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.809403 17776 raft_consensus.cc:2468] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:13.824445 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:13.827116 17981 leader_election.cc:290] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:13.829891 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.831383 17974 raft_consensus.cc:359] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.832232 17974 raft_consensus.cc:385] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.832542 17974 raft_consensus.cc:740] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.833592 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a94780ceb44c84bbaa8ba8ce21b7c3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:13.836148 17890 leader_election.cc:304] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:13.837756 17981 raft_consensus.cc:2749] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.840986 17926 raft_consensus.cc:2468] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:13.843688 17669 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.844823 17978 raft_consensus.cc:2804] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.847198 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:13.848348 17700 raft_consensus.cc:3060] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.847206 17978 raft_consensus.cc:697] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:13.850692 17978 consensus_queue.cc:237] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.854055 18042 raft_consensus.cc:515] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.856323 17925 raft_consensus.cc:2393] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 30d17e55b097491f895c84a3443cb250 in this term.
I20260709 12:30:13.857982 17972 ts_tablet_manager.cc:1434] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.001s
I20260709 12:30:13.859180 17972 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:13.862533 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:13.846469 18047 leader_election.cc:290] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:13.863793 17852 raft_consensus.cc:2393] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:13.864300 17971 tablet_bootstrap.cc:654] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.865969 18089 raft_consensus.cc:493] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.866549 17974 consensus_queue.cc:260] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:13.866739 18089 raft_consensus.cc:515] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.867276 17703 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:13.868458 17851 raft_consensus.cc:2468] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:13.868801 17742 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:13.869755 18089 leader_election.cc:290] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:13.870613 18023 raft_consensus.cc:2804] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.870682 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:13.871465 18023 raft_consensus.cc:697] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:13.872529 18023 consensus_queue.cc:237] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.876950 17974 ts_tablet_manager.cc:1434] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 12:30:13.878158 17974 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:13.881745 17667 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.873286 17973 raft_consensus.cc:359] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.884270 17925 raft_consensus.cc:3060] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.884931 17700 raft_consensus.cc:2468] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:13.884481 17973 raft_consensus.cc:385] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.885897 18089 raft_consensus.cc:2804] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.886543 18089 raft_consensus.cc:697] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:13.886224 17973 raft_consensus.cc:740] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.887430 18089 consensus_queue.cc:237] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:13.889573 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8bdb4d10d0433c909031a9340f6c1b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.876938 17850 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:13.892206 17816 leader_election.cc:304] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:13.893860 18047 raft_consensus.cc:2804] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.894445 18047 raft_consensus.cc:697] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:13.895471 18047 consensus_queue.cc:237] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:13.898397 17972 tablet_bootstrap.cc:654] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.910948 18101 consensus_queue.cc:1048] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.911432 17600 catalog_manager.cc:5697] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.913321 17974 tablet_bootstrap.cc:654] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.915472 17669 leader_election.cc:304] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:13.918061 18013 raft_consensus.cc:2749] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.921497 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b1348052e745648f1cba4c11a7e863" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:13.922788 17703 raft_consensus.cc:2393] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:13.924386 17890 leader_election.cc:304] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:13.924852 17776 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.927276 18042 leader_election.cc:290] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:13.929179 18042 raft_consensus.cc:2749] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.955657 17973 consensus_queue.cc:260] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.959939 17974 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:13.960788 17974 ts_tablet_manager.cc:1403] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.006s
I20260709 12:30:13.968518 17971 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:13.969146 17971 ts_tablet_manager.cc:1403] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.021s
I20260709 12:30:13.969286 17974 raft_consensus.cc:359] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.959008 17973 ts_tablet_manager.cc:1434] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.151s	user 0.002s	sys 0.004s
I20260709 12:30:13.970074 17974 raft_consensus.cc:385] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.970460 17974 raft_consensus.cc:740] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.976639 17973 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:13.978219 17974 consensus_queue.cc:260] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.979499 18033 raft_consensus.cc:493] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.980401 18033 raft_consensus.cc:515] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:13.985127 17974 ts_tablet_manager.cc:1434] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
W20260709 12:30:13.980424 17818 consensus_peers.cc:597] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.993840 18033 leader_election.cc:290] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:13.994831 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:13.996901 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:13.999817 17744 leader_election.cc:343] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.000398 17974 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:14.001320 17742 leader_election.cc:343] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.002195 17742 leader_election.cc:304] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:14.003795 18033 raft_consensus.cc:2749] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.007062 17972 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.007869 17972 ts_tablet_manager.cc:1403] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.149s	user 0.013s	sys 0.020s
I20260709 12:30:14.009750 17973 tablet_bootstrap.cc:654] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.990532 17600 catalog_manager.cc:5697] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.993230 17971 raft_consensus.cc:359] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.014501 17972 raft_consensus.cc:359] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.007145 18047 consensus_queue.cc:1048] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:30:14.019893 17972 raft_consensus.cc:385] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.020336 17972 raft_consensus.cc:740] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.020845 17974 tablet_bootstrap.cc:654] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.015573 17971 raft_consensus.cc:385] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.022373 17971 raft_consensus.cc:740] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.023264 17971 consensus_queue.cc:260] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.021220 17972 consensus_queue.cc:260] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.026029 18023 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:14.026683 18023 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.028878 17971 ts_tablet_manager.cc:1434] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 12:30:14.026247 17972 ts_tablet_manager.cc:1434] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:30:14.030594 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.031791 17972 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.033311 18023 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.035290 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:14.037587 17742 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:14.046242 18023 raft_consensus.cc:2749] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:14.046756 17974 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:14.047739 17974 ts_tablet_manager.cc:1403] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.003s
I20260709 12:30:14.040190 17972 tablet_bootstrap.cc:654] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.040560 17971 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:14.051537 17974 raft_consensus.cc:359] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.052313 17974 raft_consensus.cc:385] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.052599 17974 raft_consensus.cc:740] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.053620 17974 consensus_queue.cc:260] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.056401 17974 ts_tablet_manager.cc:1434] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:30:14.058264 17974 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:14.059403 17971 tablet_bootstrap.cc:654] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.064239 17597 catalog_manager.cc:5697] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.085261 17973 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:14.086151 17973 ts_tablet_manager.cc:1403] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.115s	user 0.029s	sys 0.016s
I20260709 12:30:14.090735 17973 raft_consensus.cc:359] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.091951 17973 raft_consensus.cc:385] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.092530 17973 raft_consensus.cc:740] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.092044 17974 tablet_bootstrap.cc:654] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.093709 17973 consensus_queue.cc:260] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.097862 17973 ts_tablet_manager.cc:1434] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.004s
I20260709 12:30:14.099020 17973 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:14.101578 17972 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.102286 17972 ts_tablet_manager.cc:1403] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.018s
I20260709 12:30:14.102308 17971 tablet_bootstrap.cc:492] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:14.103055 17971 ts_tablet_manager.cc:1403] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.073s	user 0.019s	sys 0.011s
I20260709 12:30:14.106014 17972 raft_consensus.cc:359] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.106971 17972 raft_consensus.cc:385] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.107362 17972 raft_consensus.cc:740] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.106107 17971 raft_consensus.cc:359] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.108274 17971 raft_consensus.cc:385] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.108785 17971 raft_consensus.cc:740] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.108309 17972 consensus_queue.cc:260] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.111536 17972 ts_tablet_manager.cc:1434] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:30:14.113042 17972 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.110503 17597 catalog_manager.cc:5697] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.115386 17973 tablet_bootstrap.cc:654] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.109817 17971 consensus_queue.cc:260] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.120442 17972 tablet_bootstrap.cc:654] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.121605 17971 ts_tablet_manager.cc:1434] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:30:14.122994 17971 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:14.125135 17974 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:14.125705 17974 ts_tablet_manager.cc:1403] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.005s
I20260709 12:30:14.129247 17972 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.129705 17971 tablet_bootstrap.cc:654] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.130263 17972 ts_tablet_manager.cc:1403] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:30:14.133978 17972 raft_consensus.cc:359] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.134927 17972 raft_consensus.cc:385] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.135396 17972 raft_consensus.cc:740] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.134481 17973 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:14.136541 17972 consensus_queue.cc:260] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.137641 17973 ts_tablet_manager.cc:1403] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.001s
I20260709 12:30:14.138988 17972 ts_tablet_manager.cc:1434] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:30:14.141469 17973 raft_consensus.cc:359] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.142354 17973 raft_consensus.cc:385] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.129094 17974 raft_consensus.cc:359] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.142680 17973 raft_consensus.cc:740] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.143595 17973 consensus_queue.cc:260] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.143044 17974 raft_consensus.cc:385] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.144639 17974 raft_consensus.cc:740] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.146121 17973 ts_tablet_manager.cc:1434] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:30:14.145128 17972 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.145965 17974 consensus_queue.cc:260] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.148772 17974 ts_tablet_manager.cc:1434] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:30:14.150317 17974 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:14.155910 18042 raft_consensus.cc:493] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.156618 18042 raft_consensus.cc:515] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.157550 17974 tablet_bootstrap.cc:654] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.159179 18042 leader_election.cc:290] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:14.161211 17972 tablet_bootstrap.cc:654] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.161383 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:14.162240 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da5cd04765b43bdaec5b9bae211aae0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:14.163295 18033 raft_consensus.cc:493] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.174700 17889 leader_election.cc:304] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:14.175705 17700 raft_consensus.cc:3060] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.176484 17981 raft_consensus.cc:2749] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.174837 18033 raft_consensus.cc:515] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.179600 17971 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:14.180600 17971 ts_tablet_manager.cc:1403] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.000s
I20260709 12:30:14.181093 18033 leader_election.cc:290] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.182047 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:14.183326 17703 raft_consensus.cc:2468] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:14.183689 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.187296 17742 leader_election.cc:304] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
W20260709 12:30:14.189527 17744 leader_election.cc:343] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.194525 18033 raft_consensus.cc:2804] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.195473 18033 raft_consensus.cc:493] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.196132 18033 raft_consensus.cc:3060] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.203934 17972 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.204625 17972 ts_tablet_manager.cc:1403] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.020s
I20260709 12:30:14.212592 17972 raft_consensus.cc:359] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.213382 17972 raft_consensus.cc:385] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.213690 17972 raft_consensus.cc:740] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.172314 17973 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:14.213848 17971 raft_consensus.cc:359] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.219551 17971 raft_consensus.cc:385] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.219980 17971 raft_consensus.cc:740] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.214581 17972 consensus_queue.cc:260] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.222656 17971 consensus_queue.cc:260] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.223327 17974 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:14.224175 17974 ts_tablet_manager.cc:1403] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.019s
I20260709 12:30:14.224669 17972 ts_tablet_manager.cc:1434] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
W20260709 12:30:14.225404 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:14.225924 17972 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.227140 17974 raft_consensus.cc:359] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.228085 17974 raft_consensus.cc:385] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.228367 17974 raft_consensus.cc:740] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.229193 17974 consensus_queue.cc:260] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.229813 17971 ts_tablet_manager.cc:1434] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 12:30:14.231186 17971 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:14.239347 17972 tablet_bootstrap.cc:654] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.233153 18033 raft_consensus.cc:515] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.243760 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:14.248139 17850 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.246547 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:14.249599 17703 raft_consensus.cc:3060] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.247118 18033 leader_election.cc:290] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:14.252928 17744 leader_election.cc:343] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.254379 17971 tablet_bootstrap.cc:654] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.232625 17974 ts_tablet_manager.cc:1434] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:14.256042 17974 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:14.233867 17973 tablet_bootstrap.cc:654] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.263082 17974 tablet_bootstrap.cc:654] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.271481 17703 raft_consensus.cc:2468] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:14.279932 17742 leader_election.cc:304] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:14.283176 18089 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.284516 18089 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.288218 17978 raft_consensus.cc:493] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.289233 17978 raft_consensus.cc:515] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.291724 18033 raft_consensus.cc:2804] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.295367 18033 raft_consensus.cc:697] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:14.296985 18033 consensus_queue.cc:237] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:14.305655 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.307548 17973 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:14.308365 17973 ts_tablet_manager.cc:1403] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.153s	user 0.021s	sys 0.007s
W20260709 12:30:14.309072 17669 leader_election.cc:343] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.311055 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:14.312155 17973 raft_consensus.cc:359] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.312956 17973 raft_consensus.cc:385] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.313289 17973 raft_consensus.cc:740] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.313199 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:14.313997 17971 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:14.313680 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:14.311195 17925 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.314747 17971 ts_tablet_manager.cc:1403] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.084s	user 0.030s	sys 0.034s
I20260709 12:30:14.315368 17978 leader_election.cc:290] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:14.314806 18089 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:14.314415 17973 consensus_queue.cc:260] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.320878 17851 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:14.323845 17667 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:14.324050 17973 ts_tablet_manager.cc:1434] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 12:30:14.324633 17971 raft_consensus.cc:359] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:14.325035 17666 leader_election.cc:343] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.325420 17971 raft_consensus.cc:385] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.325848 17971 raft_consensus.cc:740] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
W20260709 12:30:14.326292 17666 leader_election.cc:343] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.325057 18013 raft_consensus.cc:2804] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.327733 18013 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.328094 18013 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.325546 17973 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:14.329319 17971 consensus_queue.cc:260] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.326876 17666 leader_election.cc:304] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:14.333782 17971 ts_tablet_manager.cc:1434] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:30:14.334156 17978 raft_consensus.cc:2749] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.331079 17972 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.336071 17972 ts_tablet_manager.cc:1403] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.110s	user 0.004s	sys 0.015s
I20260709 12:30:14.340330 17972 raft_consensus.cc:359] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.341266 17972 raft_consensus.cc:385] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.341648 17972 raft_consensus.cc:740] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.342753 17972 consensus_queue.cc:260] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.346165 17972 ts_tablet_manager.cc:1434] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:30:14.364367 17973 tablet_bootstrap.cc:654] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.365046 17974 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:14.365331 17972 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.365799 17974 ts_tablet_manager.cc:1403] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.025s
I20260709 12:30:14.346969 18012 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.349902 18047 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:14.366869 18012 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.366931 18047 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.371413 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.372241 17926 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:14.373695 17818 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:14.374821 17979 raft_consensus.cc:2804] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.375753 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:14.378288 17599 catalog_manager.cc:5697] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:14.378329 17979 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.379801 18012 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.345695 18013 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.381174 17979 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.352448 17971 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:14.385282 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.387589 17818 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:14.390352 17974 raft_consensus.cc:359] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.391271 17974 raft_consensus.cc:385] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.391582 17974 raft_consensus.cc:740] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.393185 18101 raft_consensus.cc:2749] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:14.377408 18047 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.396133 17971 tablet_bootstrap.cc:654] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.408376 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:14.409349 17850 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.398963 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:14.399111 18013 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:14.407352 17974 consensus_queue.cc:260] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.405349 18033 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.413887 18033 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.415124 17974 ts_tablet_manager.cc:1434] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 12:30:14.416137 17776 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.416540 18033 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:14.416777 17778 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.405526 17978 consensus_queue.cc:1048] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:30:14.419473 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:14.420466 17851 raft_consensus.cc:2393] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 1: Already voted for candidate b96ea338e2a940f9b606d5d0fdd10e85 in this term.
I20260709 12:30:14.416450 17974 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:14.425269 17978 consensus_queue.cc:1048] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.423718 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:14.429860 17742 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:14.433815 17778 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:14.443396 18047 raft_consensus.cc:493] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.444023 18047 raft_consensus.cc:515] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.449776 18023 raft_consensus.cc:2749] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.451071 17973 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:14.451805 17973 ts_tablet_manager.cc:1403] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.127s	user 0.017s	sys 0.009s
I20260709 12:30:14.455689 17973 raft_consensus.cc:359] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.457851 17973 raft_consensus.cc:385] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.458276 17973 raft_consensus.cc:740] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.460984 17974 tablet_bootstrap.cc:654] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.465250 17973 consensus_queue.cc:260] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.468859 18013 consensus_queue.cc:1048] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.470935 17973 ts_tablet_manager.cc:1434] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:30:14.475739 18023 consensus_queue.cc:1048] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.427774 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:14.480158 17703 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:14.430267 17972 tablet_bootstrap.cc:654] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.498541 17926 raft_consensus.cc:3060] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.485999 18047 leader_election.cc:290] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.505232 17973 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:14.525765 17971 tablet_bootstrap.cc:492] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:14.527675 17971 ts_tablet_manager.cc:1403] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.176s	user 0.030s	sys 0.013s
I20260709 12:30:14.533322 17850 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:14.515945 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:14.526304 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.527993 18012 consensus_queue.cc:1048] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:30:14.518862 17666 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:14.518071 17979 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.557948 17778 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:14.568845 18101 raft_consensus.cc:493] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.564571 18047 consensus_queue.cc:1048] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:30:14.572131 17979 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.567937 18013 raft_consensus.cc:2804] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.575626 18136 raft_consensus.cc:493] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.585103 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:14.585963 17923 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.585582 18136 raft_consensus.cc:515] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.586822 17974 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:14.587340 18101 raft_consensus.cc:515] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.576392 18137 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.589677 18137 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.574555 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:14.590224 18101 leader_election.cc:290] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.597767 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:14.598713 17702 raft_consensus.cc:2468] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:14.599282 17923 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:14.592404 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.600289 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.601500 17921 raft_consensus.cc:2468] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:14.601804 17816 leader_election.cc:304] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:14.594691 17703 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.602980 18101 raft_consensus.cc:2804] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.603544 18101 raft_consensus.cc:493] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.603971 17973 tablet_bootstrap.cc:654] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.601969 17974 ts_tablet_manager.cc:1403] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.186s	user 0.023s	sys 0.011s
I20260709 12:30:14.603971 18101 raft_consensus.cc:3060] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:14.566628 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 47880.4us
I20260709 12:30:14.611543 17972 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:14.612092 17972 ts_tablet_manager.cc:1403] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.247s	user 0.020s	sys 0.016s
I20260709 12:30:14.606693 18137 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.609542 18013 raft_consensus.cc:697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:14.620290 17744 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:14.625026 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:14.614929 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:14.614426 17818 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:14.622895 18136 leader_election.cc:290] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.628060 18137 raft_consensus.cc:2804] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.638474 17979 raft_consensus.cc:2804] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:30:14.639824 17669 leader_election.cc:343] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.640306 17979 raft_consensus.cc:697] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:14.637787 18137 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.641404 17979 consensus_queue.cc:237] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.635516 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:14.642892 18139 log.cc:927] Time spent T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.110s	user 0.000s	sys 0.002s
I20260709 12:30:14.632902 17815 leader_election.cc:304] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:14.644739 18046 raft_consensus.cc:2804] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:14.644888 17818 leader_election.cc:343] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.645406 18046 raft_consensus.cc:493] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.645892 18046 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.647244 17971 raft_consensus.cc:359] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.648165 17971 raft_consensus.cc:385] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.648681 17971 raft_consensus.cc:740] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.642614 18137 raft_consensus.cc:3060] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.626967 17978 consensus_queue.cc:1048] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:30:14.653542 17971 consensus_queue.cc:260] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.659469 17972 raft_consensus.cc:359] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.660316 17972 raft_consensus.cc:385] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.660625 17972 raft_consensus.cc:740] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.661358 17972 consensus_queue.cc:260] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.661530 17974 raft_consensus.cc:359] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.663081 17971 ts_tablet_manager.cc:1434] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.135s	user 0.007s	sys 0.001s
I20260709 12:30:14.662333 17974 raft_consensus.cc:385] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.663934 17974 raft_consensus.cc:740] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.669688 17972 ts_tablet_manager.cc:1434] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.003s
I20260709 12:30:14.664949 17974 consensus_queue.cc:260] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.675031 17974 ts_tablet_manager.cc:1434] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 12:30:14.676478 17974 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:14.689507 17971 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
W20260709 12:30:14.644708 17742 leader_election.cc:343] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.701980 18046 raft_consensus.cc:515] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.664255 18013 consensus_queue.cc:237] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.768965 17703 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:14.771178 18101 raft_consensus.cc:515] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:14.774178 18101 leader_election.cc:290] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:14.776460 17850 raft_consensus.cc:2393] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:14.779501 17667 leader_election.cc:304] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:14.781199 18136 raft_consensus.cc:3060] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:14.750602 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 105936us
I20260709 12:30:14.788563 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:14.800524 17818 leader_election.cc:343] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.771826 17974 tablet_bootstrap.cc:654] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.813745 17972 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:14.816954 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09de1ae34c854240a05f695427ac7c94" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:14.846594 18046 leader_election.cc:290] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.862190 17973 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:14.863072 17973 ts_tablet_manager.cc:1403] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.358s	user 0.038s	sys 0.047s
W20260709 12:30:14.850008 17744 consensus_peers.cc:597] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.878415 17971 tablet_bootstrap.cc:654] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.852726 18033 consensus_queue.cc:1048] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:30:14.856648 18137 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.920866 17972 tablet_bootstrap.cc:654] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.903707 17924 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:14.924355 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:14.929698 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:14.961995 17925 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.965611 18023 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.964412 17775 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.986073 18137 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:14.988636 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:14.989446 17924 raft_consensus.cc:3060] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.991660 18145 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.993469 18145 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:14.998219 17973 raft_consensus.cc:359] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:14.999413 17973 raft_consensus.cc:385] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.999730 17973 raft_consensus.cc:740] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.000597 18023 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.000538 17973 consensus_queue.cc:260] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.022531 17973 ts_tablet_manager.cc:1434] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.159s	user 0.006s	sys 0.003s
I20260709 12:30:15.026125 18136 raft_consensus.cc:2749] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.030716 17974 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:15.031457 17974 ts_tablet_manager.cc:1403] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.355s	user 0.026s	sys 0.028s
I20260709 12:30:15.034643 17974 raft_consensus.cc:359] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.040012 17974 raft_consensus.cc:385] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.040386 17974 raft_consensus.cc:740] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.041106 17974 consensus_queue.cc:260] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.051084 17974 ts_tablet_manager.cc:1434] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:30:15.052372 17974 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:15.053735 18023 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:15.070349 17971 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.070659 17972 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:15.071019 17971 ts_tablet_manager.cc:1403] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.382s	user 0.029s	sys 0.003s
I20260709 12:30:15.071342 17972 ts_tablet_manager.cc:1403] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.258s	user 0.024s	sys 0.004s
I20260709 12:30:15.074393 17971 raft_consensus.cc:359] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.075344 17971 raft_consensus.cc:385] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.075734 17971 raft_consensus.cc:740] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.076530 17971 consensus_queue.cc:260] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.079375 17972 raft_consensus.cc:359] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.085397 17972 raft_consensus.cc:385] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.085711 17972 raft_consensus.cc:740] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.088891 17703 raft_consensus.cc:2468] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:15.093022 17971 ts_tablet_manager.cc:1434] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:30:15.098198 18145 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:15.099709 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:15.101671 17667 leader_election.cc:343] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.097400 17972 consensus_queue.cc:260] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.105695 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:15.106549 17700 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:15.108726 17971 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.108654 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:15.110498 17974 tablet_bootstrap.cc:654] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.110410 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:15.113199 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.114529 17973 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:15.116462 17972 ts_tablet_manager.cc:1434] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 12:30:15.114296 17777 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:15.119889 17972 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:15.122483 17816 leader_election.cc:304] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:15.123059 17666 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:15.124037 18145 raft_consensus.cc:2804] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.124287 18046 raft_consensus.cc:2804] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.124490 18145 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.124989 18145 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.125054 18101 consensus_queue.cc:1048] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.124913 18046 raft_consensus.cc:697] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:15.127622 18046 consensus_queue.cc:237] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:15.133926 17742 leader_election.cc:343] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.134477 17924 raft_consensus.cc:2468] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:15.134989 17597 catalog_manager.cc:5697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.141852 17775 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:15.148850 17971 tablet_bootstrap.cc:654] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.139504 17925 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
W20260709 12:30:15.134941 17742 leader_election.cc:343] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.150525 17818 leader_election.cc:304] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:15.151677 18047 raft_consensus.cc:2804] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.152275 18047 raft_consensus.cc:697] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:15.153313 18047 consensus_queue.cc:237] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.154560 17742 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:15.157347 18033 raft_consensus.cc:2804] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.157857 18033 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.158245 18033 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.163020 18046 consensus_queue.cc:1048] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.170113 17973 tablet_bootstrap.cc:654] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.172468 17982 consensus_queue.cc:1048] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.177621 18033 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.180163 17744 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:15.184262 17982 raft_consensus.cc:2804] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.185173 17982 raft_consensus.cc:697] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:15.186154 17982 consensus_queue.cc:237] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.193317 17981 raft_consensus.cc:493] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.194319 17981 raft_consensus.cc:515] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.197563 17981 leader_election.cc:290] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.199194 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:15.199656 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3120efae4a24a319065841f0dff065a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.207463 17972 tablet_bootstrap.cc:654] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.208737 18033 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:15.212806 17889 leader_election.cc:304] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:15.214644 17981 raft_consensus.cc:2749] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.218318 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:15.221489 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:15.220417 18145 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.224835 18145 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:15.226195 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:15.227394 17701 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 1: Already voted for candidate b96ea338e2a940f9b606d5d0fdd10e85 in this term.
I20260709 12:30:15.228412 17971 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
W20260709 12:30:15.228494 17667 leader_election.cc:343] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.229148 17971 ts_tablet_manager.cc:1403] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.016s
I20260709 12:30:15.230024 17974 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:15.230652 17974 ts_tablet_manager.cc:1403] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.179s	user 0.013s	sys 0.003s
I20260709 12:30:15.233387 17971 raft_consensus.cc:359] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.233968 17974 raft_consensus.cc:359] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.234675 17971 raft_consensus.cc:385] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.234890 17974 raft_consensus.cc:385] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.273663 17974 raft_consensus.cc:740] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.273488 17601 catalog_manager.cc:5697] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.235286 17971 raft_consensus.cc:740] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.252817 18145 raft_consensus.cc:493] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.275027 17974 consensus_queue.cc:260] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.275774 18145 raft_consensus.cc:515] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:15.252007 17742 leader_election.cc:343] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.279052 17974 ts_tablet_manager.cc:1434] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 12:30:15.279479 17742 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:15.279594 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:15.280370 17974 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:15.282117 17669 leader_election.cc:343] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.282833 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:15.283565 17982 raft_consensus.cc:2749] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.285243 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.287443 18145 leader_election.cc:290] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:15.288419 17666 leader_election.cc:343] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.288594 17778 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:15.288772 17974 tablet_bootstrap.cc:654] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.289126 17666 leader_election.cc:304] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.290745 18145 raft_consensus.cc:2749] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.290963 17666 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.297374 17971 consensus_queue.cc:260] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.297740 18013 raft_consensus.cc:2749] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.300588 17971 ts_tablet_manager.cc:1434] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.002s
I20260709 12:30:15.302050 17971 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.268853 17973 tablet_bootstrap.cc:492] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:15.305824 17973 ts_tablet_manager.cc:1403] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.027s
I20260709 12:30:15.294631 17972 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:15.304641 18013 consensus_queue.cc:1048] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.310828 17972 ts_tablet_manager.cc:1403] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.191s	user 0.030s	sys 0.021s
I20260709 12:30:15.315851 17972 raft_consensus.cc:359] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.317571 17972 raft_consensus.cc:385] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.318706 17973 raft_consensus.cc:359] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.319927 17973 raft_consensus.cc:385] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.320303 17973 raft_consensus.cc:740] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.321972 17973 consensus_queue.cc:260] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.318085 17972 raft_consensus.cc:740] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.324337 17972 consensus_queue.cc:260] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.328497 17973 ts_tablet_manager.cc:1434] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:30:15.333238 17971 tablet_bootstrap.cc:654] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.353284 18047 raft_consensus.cc:493] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.354339 18047 raft_consensus.cc:515] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.360293 17974 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:15.362895 17974 ts_tablet_manager.cc:1403] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.083s	user 0.035s	sys 0.033s
I20260709 12:30:15.336426 18145 consensus_queue.cc:1048] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.367383 17974 raft_consensus.cc:359] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.368768 17974 raft_consensus.cc:385] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.330350 17973 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:15.360148 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:15.362030 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:15.363363 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:15.443318 17778 raft_consensus.cc:2468] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:15.351799 17599 catalog_manager.cc:5697] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:15.444552 17818 leader_election.cc:343] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.446226 17815 leader_election.cc:304] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:15.334560 17972 ts_tablet_manager.cc:1434] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:30:15.447221 18012 raft_consensus.cc:2804] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.447680 18012 raft_consensus.cc:493] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.447850 17972 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:15.448073 18012 raft_consensus.cc:3060] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.456179 18046 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.457086 18046 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.457965 18012 raft_consensus.cc:515] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.458004 17601 catalog_manager.cc:5697] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.459756 18046 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:15.460661 18012 leader_election.cc:290] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:15.461498 17601 catalog_manager.cc:5697] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.463797 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:15.464632 17972 tablet_bootstrap.cc:654] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:15.465860 17818 leader_election.cc:343] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.467552 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.469444 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:15.470144 17778 raft_consensus.cc:3060] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.477494 17973 tablet_bootstrap.cc:654] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.379480 18047 leader_election.cc:290] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:15.369995 17971 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.369318 17974 raft_consensus.cc:740] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.425941 17925 raft_consensus.cc:3060] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.466019 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:15.481750 17971 ts_tablet_manager.cc:1403] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.180s	user 0.040s	sys 0.034s
I20260709 12:30:15.482164 17924 raft_consensus.cc:2393] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:15.397523 18013 raft_consensus.cc:493] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.483654 18013 raft_consensus.cc:515] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.483994 17818 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.490562 18012 raft_consensus.cc:3060] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.492401 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:15.494227 17667 leader_election.cc:343] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.495573 17971 raft_consensus.cc:359] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.496419 17971 raft_consensus.cc:385] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.496699 17971 raft_consensus.cc:740] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.497439 17971 consensus_queue.cc:260] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.500851 18013 leader_election.cc:290] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:15.480487 17778 raft_consensus.cc:2468] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:15.502869 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:15.504817 17669 leader_election.cc:343] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.504501 17974 consensus_queue.cc:260] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.505394 17669 leader_election.cc:304] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:15.511271 17815 leader_election.cc:304] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:15.504910 17971 ts_tablet_manager.cc:1434] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:30:15.512444 18047 raft_consensus.cc:2804] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.512969 18013 raft_consensus.cc:2749] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.513325 17971 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.513159 18047 raft_consensus.cc:697] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:15.514569 18047 consensus_queue.cc:237] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.518354 17974 ts_tablet_manager.cc:1434] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.154s	user 0.009s	sys 0.045s
I20260709 12:30:15.521365 17974 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:15.530803 17974 tablet_bootstrap.cc:654] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.531260 18012 raft_consensus.cc:2749] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.533989 17971 tablet_bootstrap.cc:654] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.549898 17971 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.550662 17971 ts_tablet_manager.cc:1403] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.002s
I20260709 12:30:15.554903 17971 raft_consensus.cc:359] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.556353 17971 raft_consensus.cc:385] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.558553 17971 raft_consensus.cc:740] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.559505 17971 consensus_queue.cc:260] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.561020 17601 catalog_manager.cc:5697] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.563395 17971 ts_tablet_manager.cc:1434] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
I20260709 12:30:15.565373 17971 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.567562 17924 raft_consensus.cc:3060] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.572095 17971 tablet_bootstrap.cc:654] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.579486 18012 consensus_queue.cc:1048] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.581138 17974 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:15.567632 17972 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:15.585424 17972 ts_tablet_manager.cc:1403] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.138s	user 0.022s	sys 0.011s
I20260709 12:30:15.582126 17974 ts_tablet_manager.cc:1403] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.005s
I20260709 12:30:15.588758 17971 tablet_bootstrap.cc:492] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.589777 17971 ts_tablet_manager.cc:1403] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 12:30:15.593372 17971 raft_consensus.cc:359] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.594529 17971 raft_consensus.cc:385] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.594857 17971 raft_consensus.cc:740] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.595745 17971 consensus_queue.cc:260] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.597724 17972 raft_consensus.cc:359] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.598487 17973 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:15.598567 17972 raft_consensus.cc:385] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.599289 17973 ts_tablet_manager.cc:1403] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.269s	user 0.029s	sys 0.029s
I20260709 12:30:15.600409 17971 ts_tablet_manager.cc:1434] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:30:15.602066 17971 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.602587 17973 raft_consensus.cc:359] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.603500 17973 raft_consensus.cc:385] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.603829 17973 raft_consensus.cc:740] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.599045 17972 raft_consensus.cc:740] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.604630 17973 consensus_queue.cc:260] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.606098 17981 raft_consensus.cc:493] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.605835 17972 consensus_queue.cc:260] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.606815 17981 raft_consensus.cc:515] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.607645 17973 ts_tablet_manager.cc:1434] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:30:15.608883 17973 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:15.611835 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:15.612879 17701 raft_consensus.cc:2468] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:15.614252 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.614570 17890 leader_election.cc:304] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:15.615145 17775 raft_consensus.cc:2468] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:15.615854 17973 tablet_bootstrap.cc:654] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.618780 17981 leader_election.cc:290] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.619645 17981 raft_consensus.cc:2804] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.620079 17981 raft_consensus.cc:493] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.620452 17981 raft_consensus.cc:3060] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.631093 17973 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:15.631522 17971 tablet_bootstrap.cc:654] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.631955 17973 ts_tablet_manager.cc:1403] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 12:30:15.636366 17972 ts_tablet_manager.cc:1434] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:30:15.637598 18013 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.640161 18013 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.641816 17971 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.641527 17973 raft_consensus.cc:359] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.642418 17973 raft_consensus.cc:385] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.642522 17971 ts_tablet_manager.cc:1403] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.004s
I20260709 12:30:15.644151 18013 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.637642 17981 raft_consensus.cc:515] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.644536 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.644953 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:15.638501 18012 consensus_queue.cc:1048] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:15.646541 17666 leader_election.cc:343] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.647487 17981 leader_election.cc:290] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.648303 17981 raft_consensus.cc:493] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.648847 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:15.649667 17775 raft_consensus.cc:3060] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.648957 17981 raft_consensus.cc:515] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:15.649178 17669 leader_election.cc:343] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.651973 17669 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.654903 18013 raft_consensus.cc:2749] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.615919 17974 raft_consensus.cc:359] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.672148 17974 raft_consensus.cc:385] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.674932 17974 raft_consensus.cc:740] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.646453 17971 raft_consensus.cc:359] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.676919 17971 raft_consensus.cc:385] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.677418 17971 raft_consensus.cc:740] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.642781 17973 raft_consensus.cc:740] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.706040 18195 raft_consensus.cc:493] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.703667 17972 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:15.679949 17974 consensus_queue.cc:260] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.710368 17971 consensus_queue.cc:260] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.712990 17981 leader_election.cc:290] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.710086 17973 consensus_queue.cc:260] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.717934 17971 ts_tablet_manager.cc:1434] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.075s	user 0.015s	sys 0.001s
I20260709 12:30:15.717551 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.693343 17775 raft_consensus.cc:2468] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:15.718734 17778 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:15.720532 17971 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.721841 17889 leader_election.cc:304] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:15.725529 17973 ts_tablet_manager.cc:1434] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.004s
I20260709 12:30:15.727931 17974 ts_tablet_manager.cc:1434] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.138s	user 0.013s	sys 0.001s
I20260709 12:30:15.729091 17974 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:15.732136 17973 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:15.729583 17981 raft_consensus.cc:2804] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.733287 17981 raft_consensus.cc:697] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:15.731897 18012 raft_consensus.cc:493] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.734163 18012 raft_consensus.cc:515] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.734289 17981 consensus_queue.cc:237] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:15.709673 17669 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 60544.6us
I20260709 12:30:15.743098 17973 tablet_bootstrap.cc:654] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.714376 18195 raft_consensus.cc:515] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.741649 17889 leader_election.cc:304] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:15.752753 17974 tablet_bootstrap.cc:654] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.757567 17971 tablet_bootstrap.cc:654] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.758103 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.724781 17982 consensus_queue.cc:1048] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:30:15.765168 18012 leader_election.cc:290] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:15.766104 17972 tablet_bootstrap.cc:654] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.776100 18191 raft_consensus.cc:2804] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:15.744760 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 130317us
I20260709 12:30:15.779832 18191 raft_consensus.cc:493] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.780416 18191 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.809113 18195 leader_election.cc:290] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:15.803499 17815 leader_election.cc:343] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.816818 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:15.817317 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.818462 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57e8cc31758742738306cb81f8d38d38" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:15.818468 17775 raft_consensus.cc:2393] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:15.824044 18012 consensus_queue.cc:1048] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.825299 17890 leader_election.cc:304] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.816952 17974 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:15.827275 17974 ts_tablet_manager.cc:1403] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.009s
W20260709 12:30:15.827345 18194 log.cc:927] Time spent T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:30:15.827431 18196 raft_consensus.cc:3060] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.829931 17979 consensus_queue.cc:1048] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.838272 18191 raft_consensus.cc:515] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:15.840637 17818 leader_election.cc:343] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.841277 17818 leader_election.cc:304] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:15.841707 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeea622a1d642ffb92c2fe4b4dcaa57" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:15.842381 17701 raft_consensus.cc:3060] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.842934 18012 raft_consensus.cc:2749] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.838804 17974 raft_consensus.cc:359] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.844033 17974 raft_consensus.cc:385] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.844376 17974 raft_consensus.cc:740] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.847260 17973 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:15.847958 17973 ts_tablet_manager.cc:1403] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.119s	user 0.020s	sys 0.023s
I20260709 12:30:15.848685 18160 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.849330 18160 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.850121 18191 leader_election.cc:290] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.851817 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:15.852648 17775 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.853799 17973 raft_consensus.cc:359] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.854212 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:15.854702 17973 raft_consensus.cc:385] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.855054 17973 raft_consensus.cc:740] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
W20260709 12:30:15.855893 17742 leader_election.cc:343] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.857111 18196 raft_consensus.cc:2749] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.857184 18160 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:15.859933 17971 tablet_bootstrap.cc:492] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.860545 17971 ts_tablet_manager.cc:1403] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.140s	user 0.008s	sys 0.005s
I20260709 12:30:15.864459 17971 raft_consensus.cc:359] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.855896 17973 consensus_queue.cc:260] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.865294 17971 raft_consensus.cc:385] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.865371 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:15.865926 18160 consensus_queue.cc:1048] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.866264 17703 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:15.867633 17973 ts_tablet_manager.cc:1434] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:30:15.865638 17971 raft_consensus.cc:740] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.868877 17973 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:15.868906 17971 consensus_queue.cc:260] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.872792 17971 ts_tablet_manager.cc:1434] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:30:15.845170 17974 consensus_queue.cc:260] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:15.878408 17973 tablet_bootstrap.cc:654] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.881739 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:15.882511 17703 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.878530 17974 ts_tablet_manager.cc:1434] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 12:30:15.882994 17775 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:15.884177 17974 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:15.884771 17889 leader_election.cc:304] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:15.874408 17971 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:15.900823 17601 catalog_manager.cc:5697] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.921298 17971 tablet_bootstrap.cc:654] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.923156 17972 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:15.929787 17972 ts_tablet_manager.cc:1403] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.226s	user 0.010s	sys 0.013s
I20260709 12:30:15.902609 18195 raft_consensus.cc:2804] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.926738 18012 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.932252 18195 raft_consensus.cc:697] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:15.932179 18012 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.937556 17974 tablet_bootstrap.cc:654] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.897372 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:15.942934 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:15.949258 17775 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:15.949111 18195 consensus_queue.cc:237] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.951822 17701 raft_consensus.cc:2468] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:15.956916 18191 raft_consensus.cc:493] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:15.957401 17742 leader_election.cc:343] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.945833 18012 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:15.892154 17703 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:15.950189 17971 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:15.968693 18191 raft_consensus.cc:515] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:15.974895 17971 ts_tablet_manager.cc:1403] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.101s	user 0.021s	sys 0.004s
I20260709 12:30:15.960811 17972 raft_consensus.cc:359] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.976218 17972 raft_consensus.cc:385] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.976642 17972 raft_consensus.cc:740] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.977463 17972 consensus_queue.cc:260] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.978910 18191 leader_election.cc:290] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:15.980037 17971 raft_consensus.cc:359] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.982175 17971 raft_consensus.cc:385] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.982519 17971 raft_consensus.cc:740] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.979945 17972 ts_tablet_manager.cc:1434] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.003s
I20260709 12:30:15.984603 17972 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:15.980875 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:15.988221 17850 raft_consensus.cc:2468] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:15.990154 17971 consensus_queue.cc:260] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:15.994290 17890 leader_election.cc:304] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:15.998546 17972 tablet_bootstrap.cc:654] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.002431 17974 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:16.003297 17974 ts_tablet_manager.cc:1403] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.119s	user 0.018s	sys 0.005s
I20260709 12:30:16.006371 17974 raft_consensus.cc:359] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.006889 17972 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:16.007157 17974 raft_consensus.cc:385] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.007469 17972 ts_tablet_manager.cc:1403] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.002s
I20260709 12:30:16.007522 17974 raft_consensus.cc:740] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.065733 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.068465 17703 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate b96ea338e2a940f9b606d5d0fdd10e85 in this term.
I20260709 12:30:16.088176 17971 ts_tablet_manager.cc:1434] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.112s	user 0.011s	sys 0.002s
W20260709 12:30:16.088284 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 93955.5us
I20260709 12:30:16.096576 18196 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.098193 18196 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.102793 18195 raft_consensus.cc:2804] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.103646 18195 raft_consensus.cc:493] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.104117 18195 raft_consensus.cc:3060] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.108174 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:16.118366 17889 leader_election.cc:343] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.122080 18196 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:16.156651 17971 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:16.153412 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.167254 17703 raft_consensus.cc:2468] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:16.168228 17973 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:16.169726 17973 ts_tablet_manager.cc:1403] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.301s	user 0.041s	sys 0.043s
I20260709 12:30:16.008564 17974 consensus_queue.cc:260] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.152644 17972 raft_consensus.cc:359] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:15.974254 17742 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:16.193759 17974 ts_tablet_manager.cc:1434] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.190s	user 0.005s	sys 0.005s
I20260709 12:30:16.195946 17974 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:16.168259 18211 raft_consensus.cc:493] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.183946 17972 raft_consensus.cc:385] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.200209 18211 raft_consensus.cc:515] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:16.201882 17982 raft_consensus.cc:2749] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.201170 17972 raft_consensus.cc:740] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.208045 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:16.214078 17972 consensus_queue.cc:260] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:16.217296 17667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 46814.5us
I20260709 12:30:16.220278 17816 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:16.223143 17972 ts_tablet_manager.cc:1434] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.215s	user 0.013s	sys 0.001s
I20260709 12:30:16.225673 17972 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:16.228418 18012 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.229753 17699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.217919 17971 tablet_bootstrap.cc:654] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:16.219388 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 261868us
I20260709 12:30:16.235546 17924 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:16.240182 17973 raft_consensus.cc:359] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.243710 17973 raft_consensus.cc:385] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.244129 17973 raft_consensus.cc:740] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.250054 17601 catalog_manager.cc:5697] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.251843 17744 leader_election.cc:304] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:16.253194 17982 raft_consensus.cc:2804] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.253715 17982 raft_consensus.cc:493] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.254108 17982 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.209407 18211 leader_election.cc:290] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:16.265712 17890 leader_election.cc:343] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.266424 17890 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:16.274937 18191 raft_consensus.cc:2749] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.277699 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:16.278966 17850 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:16.273566 17973 consensus_queue.cc:260] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.305445 18152 consensus_queue.cc:1048] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:30:16.312409 18195 raft_consensus.cc:515] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.332212 18046 consensus_queue.cc:1048] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.332726 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:16.336022 17850 raft_consensus.cc:3060] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.340757 18214 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.351615 18160 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.352785 18160 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.359824 18195 leader_election.cc:290] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.381882 17971 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:16.382673 17971 ts_tablet_manager.cc:1403] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.227s	user 0.018s	sys 0.012s
I20260709 12:30:16.366716 18012 raft_consensus.cc:2749] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.375734 18214 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.402697 17972 tablet_bootstrap.cc:654] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.388803 18195 consensus_queue.cc:1048] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:30:16.392921 18216 raft_consensus.cc:493] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.411473 17982 raft_consensus.cc:515] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:16.410543 18214 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:16.413583 18160 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.415688 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:16.422812 17667 leader_election.cc:343] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.416884 17971 raft_consensus.cc:359] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.424074 17971 raft_consensus.cc:385] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.424372 17971 raft_consensus.cc:740] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.423461 17982 leader_election.cc:290] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:16.412168 18216 raft_consensus.cc:515] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.434747 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:16.435693 17852 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.437059 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:16.439527 17667 leader_election.cc:343] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.440377 18216 leader_election.cc:290] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:16.444648 17699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:16.445304 17699 raft_consensus.cc:3060] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.448700 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:16.453047 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:16.453842 17921 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.454039 17852 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:16.456243 17973 ts_tablet_manager.cc:1434] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.286s	user 0.011s	sys 0.004s
I20260709 12:30:16.456435 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:16.457537 17774 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:16.458215 17973 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
W20260709 12:30:16.460497 18218 log.cc:927] Time spent T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:30:16.463742 17974 tablet_bootstrap.cc:654] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.459918 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:16.473634 17744 leader_election.cc:343] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.474647 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.475623 17701 raft_consensus.cc:2468] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:16.476760 17971 consensus_queue.cc:260] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.479710 17971 ts_tablet_manager.cc:1434] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.097s	user 0.012s	sys 0.000s
I20260709 12:30:16.483798 17972 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:16.484529 17972 ts_tablet_manager.cc:1403] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.260s	user 0.021s	sys 0.000s
I20260709 12:30:16.488153 18191 consensus_queue.cc:1048] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.491371 18191 consensus_queue.cc:1048] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.505002 17699 raft_consensus.cc:2468] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:16.514241 18195 consensus_queue.cc:1048] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.515961 17972 raft_consensus.cc:359] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.518796 17742 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:16.529795 17921 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:16.530262 17971 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:16.532119 17972 raft_consensus.cc:385] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.534279 17972 raft_consensus.cc:740] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.536293 17972 consensus_queue.cc:260] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.543174 17973 tablet_bootstrap.cc:654] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.535349 18160 raft_consensus.cc:2804] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:16.551849 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:16.555653 17890 leader_election.cc:304] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:16.562783 17850 raft_consensus.cc:2468] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:16.548206 17744 leader_election.cc:304] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:16.568243 17982 raft_consensus.cc:2804] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.568939 17982 raft_consensus.cc:697] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:16.570228 17982 consensus_queue.cc:237] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:16.573634 17971 tablet_bootstrap.cc:654] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.577478 18195 raft_consensus.cc:2804] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.578477 18195 raft_consensus.cc:697] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:16.579594 17666 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:16.579789 18195 consensus_queue.cc:237] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.557585 17972 ts_tablet_manager.cc:1434] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.000s
I20260709 12:30:16.549762 18160 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:30:16.585292 17666 leader_election.cc:343] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.600391 18147 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.601205 18147 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.601704 18147 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.606490 17974 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:16.609617 17974 ts_tablet_manager.cc:1403] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.414s	user 0.013s	sys 0.007s
I20260709 12:30:16.607839 17666 leader_election.cc:304] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:16.618052 18013 raft_consensus.cc:2749] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.597481 18160 raft_consensus.cc:3060] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.641206 17974 raft_consensus.cc:359] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.647133 17974 raft_consensus.cc:385] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.648010 17974 raft_consensus.cc:740] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.658281 17972 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:16.669873 17974 consensus_queue.cc:260] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.689103 17972 tablet_bootstrap.cc:654] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.688578 17979 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.697340 17979 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.698690 18147 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.701077 17979 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.703707 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:16.704468 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.705381 17973 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:16.705996 17973 ts_tablet_manager.cc:1403] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.248s	user 0.033s	sys 0.017s
W20260709 12:30:16.707141 17818 leader_election.cc:343] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:16.708029 17816 leader_election.cc:343] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.708765 17974 ts_tablet_manager.cc:1434] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.000s
I20260709 12:30:16.708582 17816 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:16.723114 18047 raft_consensus.cc:2749] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.725826 18147 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:16.725998 17973 raft_consensus.cc:359] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.727000 17973 raft_consensus.cc:385] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.727372 17973 raft_consensus.cc:740] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.729408 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
W20260709 12:30:16.731417 17667 leader_election.cc:343] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.736129 17973 consensus_queue.cc:260] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.738209 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:16.739028 17778 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.739254 17973 ts_tablet_manager.cc:1434] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 12:30:16.745960 17601 catalog_manager.cc:5697] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.765045 17599 catalog_manager.cc:5697] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.774078 17778 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:16.777606 17974 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:16.778626 18160 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.777333 17666 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:16.784474 18214 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.786656 18214 raft_consensus.cc:697] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:16.788331 18214 consensus_queue.cc:237] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.789134 17971 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:16.803828 17971 ts_tablet_manager.cc:1403] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.320s	user 0.017s	sys 0.009s
I20260709 12:30:16.805831 17973 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:16.807541 17971 raft_consensus.cc:359] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.808405 17971 raft_consensus.cc:385] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.808715 17971 raft_consensus.cc:740] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.810189 18160 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.813124 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:16.815039 17744 leader_election.cc:343] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.818516 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:16.824415 17974 tablet_bootstrap.cc:654] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.816066 17971 consensus_queue.cc:260] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.827535 17700 raft_consensus.cc:3060] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.829162 17971 ts_tablet_manager.cc:1434] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 12:30:16.834609 17971 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:16.845134 17971 tablet_bootstrap.cc:654] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.855324 17972 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:16.856261 17972 ts_tablet_manager.cc:1403] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.204s	user 0.027s	sys 0.065s
I20260709 12:30:16.860692 17972 raft_consensus.cc:359] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.861853 17972 raft_consensus.cc:385] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.862242 17972 raft_consensus.cc:740] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.863336 17972 consensus_queue.cc:260] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.872597 17973 tablet_bootstrap.cc:654] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.875567 17979 raft_consensus.cc:493] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.876268 17979 raft_consensus.cc:515] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.881153 18195 raft_consensus.cc:493] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.881879 18195 raft_consensus.cc:515] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.882310 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.888701 18195 leader_election.cc:290] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.883987 17979 leader_election.cc:290] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:16.889477 17974 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:16.889568 17702 raft_consensus.cc:2468] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:16.891111 17974 ts_tablet_manager.cc:1403] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.012s
I20260709 12:30:16.891822 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:16.892858 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:16.893786 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:16.896379 17816 leader_election.cc:304] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:16.898190 18012 raft_consensus.cc:2804] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.898708 18012 raft_consensus.cc:493] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.899056 18012 raft_consensus.cc:3060] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:16.896523 17890 leader_election.cc:343] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.901854 17599 catalog_manager.cc:5697] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:16.903723 17815 leader_election.cc:343] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.904127 17700 raft_consensus.cc:2468] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:16.895279 17702 raft_consensus.cc:2468] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:16.905745 17742 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:16.906792 18160 raft_consensus.cc:2804] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.907461 18160 raft_consensus.cc:697] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:16.907639 17890 leader_election.cc:304] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:16.908722 18195 raft_consensus.cc:2804] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.908445 18160 consensus_queue.cc:237] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.909196 18195 raft_consensus.cc:493] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.909637 18195 raft_consensus.cc:3060] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.904127 17972 ts_tablet_manager.cc:1434] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.003s
I20260709 12:30:16.906442 17974 raft_consensus.cc:359] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.914420 17974 raft_consensus.cc:385] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.914698 17974 raft_consensus.cc:740] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.916298 17974 consensus_queue.cc:260] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.919051 17974 ts_tablet_manager.cc:1434] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 12:30:16.920567 17974 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:16.926856 17972 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:16.927901 18012 raft_consensus.cc:515] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.937749 17971 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:16.938449 17971 ts_tablet_manager.cc:1403] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.001s
I20260709 12:30:16.940711 17973 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:16.941460 17973 ts_tablet_manager.cc:1403] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.191s	user 0.011s	sys 0.018s
I20260709 12:30:16.931768 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:16.949314 17597 catalog_manager.cc:5697] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:16.950227 17971 raft_consensus.cc:359] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.951047 17971 raft_consensus.cc:385] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.951370 17971 raft_consensus.cc:740] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.952241 18195 raft_consensus.cc:515] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.953294 17972 tablet_bootstrap.cc:654] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.954419 18195 leader_election.cc:290] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.955891 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:16.956717 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
W20260709 12:30:16.958411 17815 leader_election.cc:343] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.958894 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:16.948712 17700 raft_consensus.cc:3060] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.959734 17702 raft_consensus.cc:3060] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.960055 17971 consensus_queue.cc:260] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.959124 18012 leader_election.cc:290] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:16.958410 17890 leader_election.cc:343] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.966519 17971 ts_tablet_manager.cc:1434] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.008s
I20260709 12:30:16.971808 17971 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:16.956313 17973 raft_consensus.cc:359] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.974110 17973 raft_consensus.cc:385] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.974467 17973 raft_consensus.cc:740] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.977769 17974 tablet_bootstrap.cc:654] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.979027 17971 tablet_bootstrap.cc:654] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.985792 17702 raft_consensus.cc:2468] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:16.988068 17890 leader_election.cc:304] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:16.988600 17982 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.989190 17982 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:16.989665 18195 raft_consensus.cc:2804] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.986851 17700 raft_consensus.cc:2468] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:16.990286 18195 raft_consensus.cc:697] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:16.992491 17816 leader_election.cc:304] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:16.993472 17982 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:16.993950 18012 raft_consensus.cc:2804] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.994613 18012 raft_consensus.cc:697] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:16.995594 18012 consensus_queue.cc:237] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:16.998407 17973 consensus_queue.cc:260] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.000690 18147 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.001449 18147 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:16.993762 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:17.004871 17744 leader_election.cc:343] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.004638 18195 consensus_queue.cc:237] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.008181 17971 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:17.008723 18147 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:17.008960 17971 ts_tablet_manager.cc:1403] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.008s
I20260709 12:30:17.009630 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:17.010450 17850 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:17.010556 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:17.011443 17924 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:17.012544 17667 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:17.014364 17973 ts_tablet_manager.cc:1434] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.002s
I20260709 12:30:17.014456 17971 raft_consensus.cc:359] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.015793 17971 raft_consensus.cc:385] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.016007 17973 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:17.016256 17971 raft_consensus.cc:740] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.017046 18147 raft_consensus.cc:2804] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.018692 18147 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.019361 18147 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.017411 17971 consensus_queue.cc:260] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.029269 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:17.037711 18191 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.051738 18191 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.053330 17971 ts_tablet_manager.cc:1434] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:30:17.039088 18160 consensus_queue.cc:1048] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.041141 17974 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:17.056488 17974 ts_tablet_manager.cc:1403] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.006s
I20260709 12:30:17.056510 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:17.058955 17889 leader_election.cc:343] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.044144 17973 tablet_bootstrap.cc:654] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.045128 17972 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:17.061789 17971 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:17.063498 17972 ts_tablet_manager.cc:1403] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.003s
I20260709 12:30:17.059960 17974 raft_consensus.cc:359] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.065100 17974 raft_consensus.cc:385] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.065377 17974 raft_consensus.cc:740] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.066996 17974 consensus_queue.cc:260] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.054983 18191 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:17.074592 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:17.085621 17972 raft_consensus.cc:359] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.088892 17972 raft_consensus.cc:385] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.089286 17972 raft_consensus.cc:740] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.086004 18160 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:17.091428 17742 leader_election.cc:343] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.092381 18160 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.092422 17742 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:17.092628 18147 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.090600 17972 consensus_queue.cc:260] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.098206 18160 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.106215 17972 ts_tablet_manager.cc:1434] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 12:30:17.109666 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:17.110558 17850 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.101277 17597 catalog_manager.cc:5697] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:17.112938 18230 consensus_queue.cc:1048] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.115633 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:17.116588 17851 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:17.118710 17742 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:17.121196 18191 consensus_queue.cc:1048] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.121629 17973 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:17.122596 17973 ts_tablet_manager.cc:1403] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.003s
I20260709 12:30:17.104600 18160 raft_consensus.cc:2749] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.122514 17601 catalog_manager.cc:5697] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:17.124020 17974 ts_tablet_manager.cc:1434] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.001s
I20260709 12:30:17.126008 18033 raft_consensus.cc:2804] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:17.128281 18033 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.128753 18033 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:17.129185 17890 leader_election.cc:343] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.129731 17890 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:17.127656 18214 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.131515 17982 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.132108 17982 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.132146 18214 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.133293 18195 raft_consensus.cc:2749] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.134790 17982 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.126390 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:17.137730 18214 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.145051 17974 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:17.138855 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:17.163291 17851 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:17.165959 17667 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:17.146103 18147 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:17.167337 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:17.173020 17971 tablet_bootstrap.cc:654] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.145254 18033 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.177312 17972 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:17.157599 17700 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:17.175670 17921 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.182425 18214 raft_consensus.cc:2804] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:17.179073 18160 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.191100 17973 raft_consensus.cc:359] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.184175 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:17.194538 17973 raft_consensus.cc:385] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.187140 18230 raft_consensus.cc:493] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.196939 17973 raft_consensus.cc:740] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.193342 18160 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.204850 17850 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:17.209019 18033 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.209899 17973 consensus_queue.cc:260] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.210881 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:17.211771 17778 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 2: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:17.211862 17973 ts_tablet_manager.cc:1434] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.089s	user 0.004s	sys 0.006s
I20260709 12:30:17.213171 17973 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:17.205020 18230 raft_consensus.cc:515] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.214430 17667 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:17.215746 18214 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.216599 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:17.216738 18214 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:17.210427 17974 tablet_bootstrap.cc:654] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.221444 17973 tablet_bootstrap.cc:654] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.223511 18216 raft_consensus.cc:2804] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.221171 17851 raft_consensus.cc:2468] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:17.232785 18216 raft_consensus.cc:697] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:17.237505 18033 consensus_queue.cc:1048] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:30:17.251238 18230 leader_election.cc:290] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.251760 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:17.253587 17889 leader_election.cc:343] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:17.254769 17742 leader_election.cc:343] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.262672 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:17.267117 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:17.267195 17742 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:17.268426 17850 raft_consensus.cc:3060] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:17.269333 18251 raft_consensus.cc:2804] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.269981 18251 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.270391 18251 raft_consensus.cc:3060] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.309664 18160 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.317265 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:17.308920 18216 consensus_queue.cc:237] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.341974 17973 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:17.342571 17973 ts_tablet_manager.cc:1403] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.130s	user 0.043s	sys 0.044s
I20260709 12:30:17.342823 17972 tablet_bootstrap.cc:654] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.346460 17973 raft_consensus.cc:359] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.347472 17973 raft_consensus.cc:385] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.348562 17974 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:17.349393 17974 ts_tablet_manager.cc:1403] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.205s	user 0.019s	sys 0.020s
I20260709 12:30:17.347786 17973 raft_consensus.cc:740] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
W20260709 12:30:17.366353 17667 consensus_peers.cc:597] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:17.371284 17973 consensus_queue.cc:260] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.390282 17699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
W20260709 12:30:17.389698 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 59687.1us
I20260709 12:30:17.425098 17973 ts_tablet_manager.cc:1434] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.082s	user 0.013s	sys 0.000s
I20260709 12:30:17.418023 17921 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
W20260709 12:30:17.413928 17890 leader_election.cc:343] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.432543 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:17.433758 17742 leader_election.cc:343] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.431528 18251 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.443845 18216 consensus_queue.cc:1048] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.447060 17890 leader_election.cc:304] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:17.453960 18191 raft_consensus.cc:2749] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.449916 18251 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.465926 17973 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
W20260709 12:30:17.487124 17742 leader_election.cc:343] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.490851 17742 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:17.491439 18152 raft_consensus.cc:493] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.491838 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:17.492065 18152 raft_consensus.cc:515] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:17.493371 18251 raft_consensus.cc:2749] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.485128 17597 catalog_manager.cc:5697] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:17.496110 18152 leader_election.cc:290] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:17.498515 17973 tablet_bootstrap.cc:654] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:17.498764 17742 leader_election.cc:343] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.505853 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:17.506953 17972 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:17.508379 17972 ts_tablet_manager.cc:1403] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.338s	user 0.027s	sys 0.021s
I20260709 12:30:17.507030 17703 raft_consensus.cc:2393] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 30d17e55b097491f895c84a3443cb250 in this term.
I20260709 12:30:17.512758 18263 consensus_queue.cc:1048] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.517910 17974 raft_consensus.cc:359] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.523962 17974 raft_consensus.cc:385] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.528831 18216 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.529583 18216 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:17.487437 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 77797.2us
I20260709 12:30:17.532903 18216 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:17.534538 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:17.544265 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:17.515285 18152 consensus_queue.cc:1048] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:30:17.564252 17850 raft_consensus.cc:2468] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:17.525130 17974 raft_consensus.cc:740] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.550688 17972 raft_consensus.cc:359] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.568540 17972 raft_consensus.cc:385] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.569103 17972 raft_consensus.cc:740] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.570518 17974 consensus_queue.cc:260] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.583353 17972 consensus_queue.cc:260] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.575603 17776 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:17.598922 17974 ts_tablet_manager.cc:1434] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.249s	user 0.002s	sys 0.012s
I20260709 12:30:17.581315 17818 leader_election.cc:304] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:17.617287 18271 raft_consensus.cc:3060] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.617967 17971 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:17.618649 17971 ts_tablet_manager.cc:1403] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.557s	user 0.017s	sys 0.016s
I20260709 12:30:17.641233 17972 ts_tablet_manager.cc:1434] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.132s	user 0.013s	sys 0.000s
I20260709 12:30:17.641883 17974 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:17.644366 17972 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:17.600975 18191 consensus_queue.cc:1048] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260709 12:30:17.647838 17666 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:17.677804 18216 raft_consensus.cc:2804] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.678289 18216 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.678658 18216 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.650527 17742 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:17.683883 18257 raft_consensus.cc:2804] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:30:17.597872 17744 consensus_peers.cc:597] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:17.685287 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 94126.4us
W20260709 12:30:17.623719 17815 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:17.651758 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:17.655900 17974 tablet_bootstrap.cc:654] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.686429 17851 raft_consensus.cc:3060] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:17.686157 17815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 88384us
I20260709 12:30:17.615191 18214 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.686074 18257 raft_consensus.cc:697] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:17.690088 17702 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 2: Already voted for candidate b96ea338e2a940f9b606d5d0fdd10e85 in this term.
I20260709 12:30:17.690442 18257 consensus_queue.cc:237] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.641321 17971 raft_consensus.cc:359] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.694437 17971 raft_consensus.cc:385] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.694685 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:17.696768 18214 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:17.697124 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874bb3d07a84cc1b883375c3e83bcfe" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:17.698429 17850 raft_consensus.cc:2393] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 2: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:17.694832 17971 raft_consensus.cc:740] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.700016 17971 consensus_queue.cc:260] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.698305 17851 raft_consensus.cc:2468] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:17.701838 17972 tablet_bootstrap.cc:654] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.716722 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:17.718080 17921 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:17.723492 17971 ts_tablet_manager.cc:1434] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.104s	user 0.006s	sys 0.004s
I20260709 12:30:17.734786 17742 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:17.714594 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 96971.3us
I20260709 12:30:17.746012 17667 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:17.746567 18256 raft_consensus.cc:2804] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.747174 18256 raft_consensus.cc:697] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:17.747721 18214 raft_consensus.cc:2749] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:30:17.762215 17971 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:17.762617 17973 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:17.770759 17971 tablet_bootstrap.cc:654] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.772425 17973 ts_tablet_manager.cc:1403] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.298s	user 0.009s	sys 0.013s
W20260709 12:30:17.781445 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:17.748178 18256 consensus_queue.cc:237] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.795915 17974 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:17.796684 17974 ts_tablet_manager.cc:1403] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.155s	user 0.019s	sys 0.020s
I20260709 12:30:17.800235 17601 catalog_manager.cc:5697] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 2, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:17.801124 18147 consensus_queue.cc:1048] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.801667 17974 raft_consensus.cc:359] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:17.802421 17974 raft_consensus.cc:385] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.802736 17974 raft_consensus.cc:740] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.803464 17974 consensus_queue.cc:260] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:17.808810 18267 log.cc:927] Time spent T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:30:17.812255 17974 ts_tablet_manager.cc:1434] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:30:17.814620 17974 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:17.856397 18271 raft_consensus.cc:2749] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.877017 18195 consensus_queue.cc:1048] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.884282 17973 raft_consensus.cc:359] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.885003 17973 raft_consensus.cc:385] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.885272 17973 raft_consensus.cc:740] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.888299 17973 consensus_queue.cc:260] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.891537 17972 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:17.892306 17972 ts_tablet_manager.cc:1403] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.248s	user 0.047s	sys 0.013s
I20260709 12:30:17.897258 18216 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.897369 17972 raft_consensus.cc:359] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.900198 17972 raft_consensus.cc:385] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.901033 17972 raft_consensus.cc:740] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.911294 17972 consensus_queue.cc:260] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.917441 17973 ts_tablet_manager.cc:1434] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.144s	user 0.030s	sys 0.021s
I20260709 12:30:17.931845 17972 ts_tablet_manager.cc:1434] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 12:30:17.938827 17971 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:17.926689 18195 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.940702 17971 ts_tablet_manager.cc:1403] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.190s	user 0.004s	sys 0.014s
I20260709 12:30:17.944681 17973 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:17.962368 17972 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:17.955224 18195 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:17.966363 18195 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:17.967670 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:17.973153 17971 raft_consensus.cc:359] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:17.974236 17971 raft_consensus.cc:385] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.974579 17971 raft_consensus.cc:740] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.975404 17597 catalog_manager.cc:5697] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:17.980057 17974 tablet_bootstrap.cc:654] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.985721 17921 raft_consensus.cc:3060] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.993018 17972 tablet_bootstrap.cc:654] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.993541 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:17.994395 17778 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.993541 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:18.002825 17971 consensus_queue.cc:260] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.007807 18255 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.030663 18255 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.031929 18257 raft_consensus.cc:493] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.032733 17890 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:18.032682 18257 raft_consensus.cc:515] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.036366 18257 leader_election.cc:290] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:18.037320 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:18.038933 17776 raft_consensus.cc:3060] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.002139 18216 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:18.000046 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 56497.1us
I20260709 12:30:18.046941 17972 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
W20260709 12:30:18.047888 17666 leader_election.cc:343] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.050947 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:18.052307 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f211414d5e584164a01d70059b214a4b" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:18.050465 17972 ts_tablet_manager.cc:1403] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.091s	user 0.024s	sys 0.002s
I20260709 12:30:18.059262 17742 leader_election.cc:304] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:18.073750 17778 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:18.073877 17972 raft_consensus.cc:359] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.074775 17972 raft_consensus.cc:385] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.075124 17972 raft_consensus.cc:740] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.076154 17972 consensus_queue.cc:260] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.077725 17666 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:18.044589 18255 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:18.084190 18214 raft_consensus.cc:2804] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.084784 18214 raft_consensus.cc:697] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:18.085057 17974 tablet_bootstrap.cc:492] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:18.085675 18214 consensus_queue.cc:237] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.085695 17974 ts_tablet_manager.cc:1403] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.271s	user 0.031s	sys 0.019s
I20260709 12:30:18.055052 17973 tablet_bootstrap.cc:654] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.090986 17974 raft_consensus.cc:359] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.094950 17974 raft_consensus.cc:385] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.095423 17974 raft_consensus.cc:740] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.098869 17971 ts_tablet_manager.cc:1434] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.156s	user 0.017s	sys 0.002s
I20260709 12:30:18.100757 17971 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:18.105391 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.106392 17922 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.109200 18147 consensus_queue.cc:1048] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.110939 17974 consensus_queue.cc:260] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.121045 18256 raft_consensus.cc:493] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.121951 18256 raft_consensus.cc:515] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.137238 18281 raft_consensus.cc:493] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.137966 18281 raft_consensus.cc:515] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.139552 18256 leader_election.cc:290] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.132584 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.140611 18281 leader_election.cc:290] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.141075 17926 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:18.142936 17971 tablet_bootstrap.cc:654] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.143896 17974 ts_tablet_manager.cc:1434] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 12:30:18.144765 17972 ts_tablet_manager.cc:1434] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.003s
I20260709 12:30:18.140111 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:18.145943 17852 raft_consensus.cc:2468] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:18.147656 17742 leader_election.cc:304] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:18.148635 18256 raft_consensus.cc:2804] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.149084 18256 raft_consensus.cc:493] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.149482 18256 raft_consensus.cc:3060] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.156203 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.158425 17669 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:18.159420 18214 raft_consensus.cc:2804] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.159976 18214 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.160283 18288 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.160446 18214 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.166791 18230 raft_consensus.cc:2749] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.157092 17924 raft_consensus.cc:2468] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:18.180090 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:18.160912 18288 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.184921 17922 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:18.187736 17972 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:18.189328 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:18.190392 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:18.191141 17890 leader_election.cc:343] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.191574 17974 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:18.189396 17700 raft_consensus.cc:1275] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 12:30:18.192272 17889 leader_election.cc:343] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.192843 17889 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:18.194005 18230 raft_consensus.cc:2749] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.198644 18257 consensus_queue.cc:1048] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.203565 18256 raft_consensus.cc:515] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.198565 17597 catalog_manager.cc:5697] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:18.186990 18288 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:18.210065 17818 leader_election.cc:304] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:18.210485 18216 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.176007 18214 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.211196 18216 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.210052 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.212761 17923 raft_consensus.cc:2468] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:18.213043 18271 raft_consensus.cc:2804] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.213569 18271 raft_consensus.cc:493] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.213930 18271 raft_consensus.cc:3060] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.216542 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:18.217259 18216 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:18.219165 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:18.220023 17667 leader_election.cc:343] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.220098 17775 raft_consensus.cc:2468] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:18.219281 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:18.225317 17666 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:18.226374 18216 raft_consensus.cc:2804] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:18.226804 17666 leader_election.cc:343] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.226842 18216 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.228075 18216 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.231974 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.232820 17921 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.236270 17972 tablet_bootstrap.cc:654] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.237067 17778 raft_consensus.cc:2393] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:18.213125 18214 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:18.246341 17974 tablet_bootstrap.cc:654] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.248261 17973 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:18.248944 17973 ts_tablet_manager.cc:1403] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.314s	user 0.012s	sys 0.012s
I20260709 12:30:18.246448 18271 raft_consensus.cc:515] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.262636 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:18.264320 17778 raft_consensus.cc:2393] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:18.268682 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.269515 17924 raft_consensus.cc:3060] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.270066 17971 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:18.270717 17971 ts_tablet_manager.cc:1403] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.170s	user 0.015s	sys 0.003s
I20260709 12:30:18.274613 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:18.276043 17852 raft_consensus.cc:2393] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:18.276557 17921 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:18.269618 18271 leader_election.cc:290] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.279954 17973 raft_consensus.cc:359] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.279990 17971 raft_consensus.cc:359] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:18.281106 17973 raft_consensus.cc:385] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.281219 17971 raft_consensus.cc:385] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.280623 17924 raft_consensus.cc:2468] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:18.281505 17973 raft_consensus.cc:740] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.281693 17971 raft_consensus.cc:740] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.282974 18251 raft_consensus.cc:2749] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.299108 17974 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:18.286267 17852 raft_consensus.cc:1275] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:18.288722 17818 leader_election.cc:304] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:18.300218 18281 raft_consensus.cc:2804] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.300957 18281 raft_consensus.cc:697] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:18.302706 17974 ts_tablet_manager.cc:1403] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.000s
I20260709 12:30:18.303966 18251 consensus_queue.cc:1048] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.306172 17974 raft_consensus.cc:359] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.294610 17669 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:18.307256 17974 raft_consensus.cc:385] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.307763 17974 raft_consensus.cc:740] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.308676 17974 consensus_queue.cc:260] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.309264 18214 raft_consensus.cc:2804] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.310204 18214 raft_consensus.cc:697] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:18.311937 17974 ts_tablet_manager.cc:1434] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:18.311667 18214 consensus_queue.cc:237] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.316254 17971 consensus_queue.cc:260] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:18.317677 18263 consensus_queue.cc:1048] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:18.322907 17974 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:18.328958 17971 ts_tablet_manager.cc:1434] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.001s
W20260709 12:30:18.341383 17742 consensus_peers.cc:597] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.349622 18147 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.350560 18147 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.356803 17974 tablet_bootstrap.cc:654] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.285771 18256 leader_election.cc:290] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.292500 18216 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.360167 17973 consensus_queue.cc:260] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.361581 18216 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:18.364470 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:18.302269 18281 consensus_queue.cc:237] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:18.367790 17667 leader_election.cc:343] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.355888 18147 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:18.358897 17971 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:18.296766 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3592a718f974e5f9f26e16b1942834f" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.387529 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:18.388676 17774 raft_consensus.cc:2468] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:18.389361 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:18.390087 17778 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.393055 17666 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:18.400234 18271 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.401198 18271 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.400128 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.398101 18255 raft_consensus.cc:2804] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.410621 18255 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.411019 18255 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.412398 18271 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:18.421339 18297 log.cc:927] Time spent T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:30:18.422987 17599 catalog_manager.cc:5697] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:18.428524 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:18.434614 18295 log.cc:927] Time spent T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:30:18.435806 18256 raft_consensus.cc:493] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.436539 18256 raft_consensus.cc:515] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.439194 18256 leader_election.cc:290] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.441141 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:18.443295 17778 raft_consensus.cc:2468] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:18.449426 17973 ts_tablet_manager.cc:1434] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.200s	user 0.009s	sys 0.001s
W20260709 12:30:18.457708 17669 leader_election.cc:343] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.456422 17924 raft_consensus.cc:2393] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:18.465564 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:18.463270 17666 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:18.470186 18214 raft_consensus.cc:2804] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.471006 18214 raft_consensus.cc:697] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:18.471459 17971 tablet_bootstrap.cc:654] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.472285 18214 consensus_queue.cc:237] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.463064 17815 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:18.477458 18012 raft_consensus.cc:2749] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.496513 18255 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.509248 17973 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:18.537895 17744 leader_election.cc:304] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:18.539966 18255 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:18.542243 18256 raft_consensus.cc:2749] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.542591 17972 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:18.545322 17972 ts_tablet_manager.cc:1403] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.358s	user 0.056s	sys 0.101s
I20260709 12:30:18.549628 17972 raft_consensus.cc:359] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.551662 17972 raft_consensus.cc:385] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.543318 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.552083 17972 raft_consensus.cc:740] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.553350 17972 consensus_queue.cc:260] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.557649 17599 catalog_manager.cc:5697] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:18.561879 17974 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:18.562618 17974 ts_tablet_manager.cc:1403] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.249s	user 0.014s	sys 0.009s
I20260709 12:30:18.566313 17974 raft_consensus.cc:359] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.567113 17974 raft_consensus.cc:385] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.567459 17974 raft_consensus.cc:740] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.568248 17974 consensus_queue.cc:260] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.571014 17974 ts_tablet_manager.cc:1434] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:30:18.572443 17974 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:18.573808 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.580312 17974 tablet_bootstrap.cc:654] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.596354 17974 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:18.597158 17974 ts_tablet_manager.cc:1403] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.007s
I20260709 12:30:18.600450 17974 raft_consensus.cc:359] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.601253 17974 raft_consensus.cc:385] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.601580 17974 raft_consensus.cc:740] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.602594 17974 consensus_queue.cc:260] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:18.602731 17669 leader_election.cc:343] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.610082 17972 ts_tablet_manager.cc:1434] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.001s
I20260709 12:30:18.612021 17974 ts_tablet_manager.cc:1434] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.003s
I20260709 12:30:18.613073 17972 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:18.613940 17974 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:18.613693 17971 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:18.614712 17971 ts_tablet_manager.cc:1403] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.266s	user 0.020s	sys 0.058s
I20260709 12:30:18.616807 17973 tablet_bootstrap.cc:654] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:18.530372 17742 leader_election.cc:343] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:18.623553 17744 leader_election.cc:343] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.620594 17971 raft_consensus.cc:359] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.622206 17974 tablet_bootstrap.cc:654] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.625844 17744 leader_election.cc:304] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:18.626653 17971 raft_consensus.cc:385] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.627044 18256 raft_consensus.cc:2749] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.627091 17971 raft_consensus.cc:740] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
W20260709 12:30:18.627985 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 44086.6us
I20260709 12:30:18.628978 17971 consensus_queue.cc:260] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.635800 17971 ts_tablet_manager.cc:1434] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
W20260709 12:30:18.636395 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 106103us
I20260709 12:30:18.639390 18255 consensus_queue.cc:1048] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.632608 17972 tablet_bootstrap.cc:654] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.644094 17971 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:18.655198 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:18.656421 17775 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.699090 18288 raft_consensus.cc:493] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.700039 18288 raft_consensus.cc:515] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:18.703429 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:18.706280 17890 leader_election.cc:343] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.703614 17971 tablet_bootstrap.cc:654] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.707100 18288 leader_election.cc:290] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:18.712672 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
W20260709 12:30:18.720016 17890 leader_election.cc:343] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.720610 17890 leader_election.cc:304] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:18.723837 17601 catalog_manager.cc:5697] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:18.723290 18230 raft_consensus.cc:2749] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.733577 17852 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.759768 17972 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:18.760526 17972 ts_tablet_manager.cc:1403] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.148s	user 0.016s	sys 0.009s
I20260709 12:30:18.759768 17974 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:18.767392 17972 raft_consensus.cc:359] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.768321 17972 raft_consensus.cc:385] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.769994 17974 ts_tablet_manager.cc:1403] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.156s	user 0.016s	sys 0.015s
I20260709 12:30:18.770160 18214 consensus_queue.cc:1048] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.775624 18270 consensus_queue.cc:1048] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.778673 17972 raft_consensus.cc:740] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.779776 17972 consensus_queue.cc:260] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.757980 17973 tablet_bootstrap.cc:492] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:18.782768 17973 ts_tablet_manager.cc:1403] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.320s	user 0.047s	sys 0.011s
I20260709 12:30:18.784948 17972 ts_tablet_manager.cc:1434] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 12:30:18.786276 17974 raft_consensus.cc:359] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.787266 17974 raft_consensus.cc:385] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.787425 17973 raft_consensus.cc:359] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.790426 17973 raft_consensus.cc:385] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.790838 17973 raft_consensus.cc:740] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.790380 17974 raft_consensus.cc:740] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.789647 17972 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:18.792425 17974 consensus_queue.cc:260] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.805111 17974 ts_tablet_manager.cc:1434] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:30:18.806378 17974 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:18.808494 17971 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:18.811460 17971 ts_tablet_manager.cc:1403] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.168s	user 0.025s	sys 0.017s
I20260709 12:30:18.811707 17973 consensus_queue.cc:260] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.815902 17700 raft_consensus.cc:3060] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.818104 17971 raft_consensus.cc:359] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.819146 17971 raft_consensus.cc:385] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.803149 17972 tablet_bootstrap.cc:654] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.823446 18270 raft_consensus.cc:493] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.824167 18270 raft_consensus.cc:515] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.829936 17971 raft_consensus.cc:740] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.815943 18214 raft_consensus.cc:493] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.836301 17973 ts_tablet_manager.cc:1434] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 12:30:18.820359 17775 raft_consensus.cc:2468] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:18.841625 18270 leader_election.cc:290] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.842753 17973 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:18.838184 17971 consensus_queue.cc:260] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:18.852934 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:18.855566 17971 ts_tablet_manager.cc:1434] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:30:18.856925 17666 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:18.857267 17971 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:18.861639 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:18.862058 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:18.862537 17775 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:18.862732 17922 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:18.865672 18255 raft_consensus.cc:2804] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.866380 18255 raft_consensus.cc:697] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:18.867507 18255 consensus_queue.cc:237] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:18.875345 17971 tablet_bootstrap.cc:654] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.876351 17815 leader_election.cc:304] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:18.877444 18012 raft_consensus.cc:2804] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.877851 18012 raft_consensus.cc:493] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.878190 18012 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.839210 18214 raft_consensus.cc:515] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:18.892412 17974 tablet_bootstrap.cc:654] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.893733 18012 raft_consensus.cc:515] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:18.912762 18012 leader_election.cc:290] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:18.915450 17778 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.916092 18214 leader_election.cc:290] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:18.921185 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:18.922118 17922 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.946444 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:18.947721 17775 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.947945 17971 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:18.948843 17971 ts_tablet_manager.cc:1403] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.092s	user 0.028s	sys 0.009s
I20260709 12:30:18.956414 18012 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.958112 18012 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:18.940392 18292 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.988057 18012 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:18.969898 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 65199.3us
I20260709 12:30:18.979496 18257 raft_consensus.cc:493] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.011334 17971 raft_consensus.cc:359] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.012228 17971 raft_consensus.cc:385] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.012494 17971 raft_consensus.cc:740] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.973006 18292 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.013563 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:19.013381 17971 consensus_queue.cc:260] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.019496 17972 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:19.019917 18292 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:19.020195 17972 ts_tablet_manager.cc:1403] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.231s	user 0.012s	sys 0.026s
I20260709 12:30:19.004935 18147 consensus_queue.cc:1048] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:30:19.022495 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:19.024236 17973 tablet_bootstrap.cc:654] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:19.024534 17742 leader_election.cc:343] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.036794 18033 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.050554 18033 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:19.051734 17775 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:19.053215 18304 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
W20260709 12:30:19.053535 17816 leader_election.cc:343] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.053916 18304 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:19.007273 18257 raft_consensus.cc:515] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:19.055612 18314 log.cc:927] Time spent T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.092s	user 0.004s	sys 0.000s
I20260709 12:30:19.056903 18257 leader_election.cc:290] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:19.059741 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.060791 17926 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:19.061676 17972 raft_consensus.cc:359] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:19.062526 17972 raft_consensus.cc:385] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.062827 17972 raft_consensus.cc:740] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.069387 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.073014 17972 consensus_queue.cc:260] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:19.067844 18255 consensus_queue.cc:1048] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:30:19.085004 17972 ts_tablet_manager.cc:1434] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:30:19.087564 17972 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:19.090711 17922 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:19.099004 17971 ts_tablet_manager.cc:1434] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.150s	user 0.011s	sys 0.008s
I20260709 12:30:19.072027 17818 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:19.108492 18033 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:19.099488 18323 raft_consensus.cc:493] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.112920 18152 raft_consensus.cc:2804] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.110780 18304 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:19.113417 18152 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.113823 18152 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.116796 17973 tablet_bootstrap.cc:492] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:19.117551 17973 ts_tablet_manager.cc:1403] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.275s	user 0.017s	sys 0.006s
I20260709 12:30:19.119140 17818 leader_election.cc:304] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:19.112604 18323 raft_consensus.cc:515] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.126835 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.138091 17971 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:19.140076 18271 raft_consensus.cc:2804] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.140756 18271 raft_consensus.cc:697] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:19.141772 18271 consensus_queue.cc:237] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.142800 17973 raft_consensus.cc:359] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.143632 17973 raft_consensus.cc:385] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.143904 17973 raft_consensus.cc:740] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.144610 17973 consensus_queue.cc:260] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.145574 18323 leader_election.cc:290] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:19.151196 17669 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 71777.9us
I20260709 12:30:19.184226 17973 ts_tablet_manager.cc:1434] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:30:19.197588 18012 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.172920 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
W20260709 12:30:19.182072 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 70571us
I20260709 12:30:19.198333 18012 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.162616 18216 consensus_queue.cc:1048] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
W20260709 12:30:19.180708 17666 leader_election.cc:343] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.207540 17702 raft_consensus.cc:2468] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:19.213555 17974 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:19.218372 18012 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:19.188324 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:19.226213 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.293712 17972 tablet_bootstrap.cc:654] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.234577 17973 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:19.297163 17974 ts_tablet_manager.cc:1403] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.491s	user 0.033s	sys 0.019s
W20260709 12:30:19.286330 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 118065us
I20260709 12:30:19.307193 17971 tablet_bootstrap.cc:654] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.223878 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:19.219247 17920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.227669 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.218325 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.222890 17849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:19.223692 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.213569 18152 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.327801 17598 catalog_manager.cc:5697] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:19.301276 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 68468.5us
W20260709 12:30:19.296432 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 74011.1us
I20260709 12:30:19.356921 18152 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:19.336340 17889 leader_election.cc:343] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.334153 17924 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:19.406361 17597 catalog_manager.cc:5697] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.432001 17974 raft_consensus.cc:359] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.439667 17974 raft_consensus.cc:385] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:19.387123 17744 leader_election.cc:343] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.437938 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:19.440303 17974 raft_consensus.cc:740] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.454071 17974 consensus_queue.cc:260] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:19.444906 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 61868.8us
I20260709 12:30:19.466150 17742 leader_election.cc:304] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:19.476514 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:19.484561 17744 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:19.488277 17973 tablet_bootstrap.cc:654] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.480476 17921 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.472613 18338 consensus_queue.cc:1048] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:30:19.499639 17974 ts_tablet_manager.cc:1434] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.200s	user 0.010s	sys 0.000s
I20260709 12:30:19.500994 17974 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:19.508589 18263 raft_consensus.cc:2749] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.527467 17972 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:19.528160 17972 ts_tablet_manager.cc:1403] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.441s	user 0.008s	sys 0.028s
I20260709 12:30:19.539113 18147 consensus_queue.cc:1048] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:19.545637 18343 raft_consensus.cc:2804] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.546237 18343 raft_consensus.cc:493] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.546614 18343 raft_consensus.cc:3060] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.549721 17972 raft_consensus.cc:359] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.550644 17972 raft_consensus.cc:385] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:19.557511 17667 consensus_peers.cc:597] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:19.551097 17972 raft_consensus.cc:740] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.565964 17971 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:19.566664 17971 ts_tablet_manager.cc:1403] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.429s	user 0.017s	sys 0.030s
I20260709 12:30:19.560148 17744 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:19.571326 17890 leader_election.cc:343] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.563720 17972 consensus_queue.cc:260] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.572045 17890 leader_election.cc:304] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:19.573935 18329 raft_consensus.cc:2749] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.573654 17971 raft_consensus.cc:359] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.574584 17971 raft_consensus.cc:385] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.574903 17971 raft_consensus.cc:740] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.581769 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:19.575474 17972 ts_tablet_manager.cc:1434] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.003s
I20260709 12:30:19.571707 17921 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:19.577739 17974 tablet_bootstrap.cc:654] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.581611 17973 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:19.589771 17973 ts_tablet_manager.cc:1403] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.356s	user 0.008s	sys 0.017s
I20260709 12:30:19.592145 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.598419 17778 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:19.588173 17972 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:19.597052 18256 raft_consensus.cc:2749] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:19.582798 17971 consensus_queue.cc:260] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.605703 17818 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
W20260709 12:30:19.606832 17667 leader_election.cc:343] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.607468 17971 ts_tablet_manager.cc:1434] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:30:19.607733 18281 raft_consensus.cc:2804] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.608120 18281 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.608549 18281 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.608701 17971 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:19.607481 17667 leader_election.cc:304] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:19.611972 17972 tablet_bootstrap.cc:654] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.612355 18214 raft_consensus.cc:2749] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.612931 17667 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:19.613996 18214 raft_consensus.cc:2749] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:19.621290 18329 raft_consensus.cc:493] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.621986 18329 raft_consensus.cc:515] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.624429 18329 leader_election.cc:290] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:19.635073 17973 raft_consensus.cc:359] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:19.635653 17818 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:19.636227 17973 raft_consensus.cc:385] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.636372 17777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.636821 18152 raft_consensus.cc:2804] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.637460 18152 raft_consensus.cc:697] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
W20260709 12:30:19.638224 17889 leader_election.cc:343] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.638489 18152 consensus_queue.cc:237] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.633005 18257 raft_consensus.cc:493] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.636843 17973 raft_consensus.cc:740] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.651819 17973 consensus_queue.cc:260] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:19.663146 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:19.664707 17816 leader_election.cc:343] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.644640 18257 raft_consensus.cc:515] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.669981 18257 leader_election.cc:290] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:19.688585 17973 ts_tablet_manager.cc:1434] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.002s
I20260709 12:30:19.690006 17973 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:19.689710 18281 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.692106 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:19.693065 17924 raft_consensus.cc:2468] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
W20260709 12:30:19.717060 17669 consensus_peers.cc:597] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 -> Peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Couldn't send request to peer 30d17e55b097491f895c84a3443cb250. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:19.735880 17971 tablet_bootstrap.cc:654] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.745676 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:19.724339 17744 leader_election.cc:304] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:19.760068 17972 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:19.760711 17972 ts_tablet_manager.cc:1403] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.173s	user 0.025s	sys 0.011s
I20260709 12:30:19.760835 18281 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:19.764276 17972 raft_consensus.cc:359] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.765085 17972 raft_consensus.cc:385] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.765395 17972 raft_consensus.cc:740] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.766141 17972 consensus_queue.cc:260] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:19.767750 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:19.768512 17778 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.769137 17972 ts_tablet_manager.cc:1434] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:30:19.770467 17972 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:19.773375 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:19.743652 17890 leader_election.cc:343] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.774194 17921 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.774569 17890 leader_election.cc:304] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:19.777114 18332 raft_consensus.cc:2749] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.777477 17972 tablet_bootstrap.cc:654] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.781347 18343 raft_consensus.cc:515] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:19.782588 17742 leader_election.cc:343] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.784281 18281 consensus_queue.cc:1048] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:19.791890 18271 consensus_queue.cc:1048] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:19.795575 18257 raft_consensus.cc:2804] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.796190 18257 raft_consensus.cc:493] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.796572 18257 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.808010 17973 tablet_bootstrap.cc:654] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:19.809166 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 90947.2us
I20260709 12:30:19.823212 17778 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:19.836071 17815 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:19.838441 18012 raft_consensus.cc:2804] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.839592 18012 raft_consensus.cc:697] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:19.841126 18012 consensus_queue.cc:237] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.862655 17921 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:19.863488 17974 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:19.864111 17974 ts_tablet_manager.cc:1403] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.363s	user 0.014s	sys 0.016s
I20260709 12:30:19.900473 17972 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:19.901109 17972 ts_tablet_manager.cc:1403] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.131s	user 0.026s	sys 0.024s
I20260709 12:30:19.903119 17974 raft_consensus.cc:359] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.904011 17974 raft_consensus.cc:385] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.904312 17974 raft_consensus.cc:740] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.904223 17972 raft_consensus.cc:359] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:19.905071 17972 raft_consensus.cc:385] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.905390 17972 raft_consensus.cc:740] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.905174 17974 consensus_queue.cc:260] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:19.906216 17972 consensus_queue.cc:260] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:19.907684 18343 leader_election.cc:290] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:19.908931 17972 ts_tablet_manager.cc:1434] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:30:19.909899 17974 ts_tablet_manager.cc:1434] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.002s
I20260709 12:30:19.916007 17974 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
W20260709 12:30:19.925205 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:19.925881 17974 tablet_bootstrap.cc:654] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.949901 18281 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.950577 18281 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:19.984969 17698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:19.987488 18281 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:19.991708 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:19.998981 17700 raft_consensus.cc:3060] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.014173 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:19.981489 18147 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.019289 18147 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.034833 18147 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:20.041247 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:20.045735 17744 leader_election.cc:343] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.050155 17598 catalog_manager.cc:5697] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.052286 17598 catalog_manager.cc:5697] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.058996 17816 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:20.062849 18271 raft_consensus.cc:2749] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.069461 18320 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.070217 18320 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.072992 18320 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:20.077440 17851 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.077879 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:20.085562 17972 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:20.011197 17973 tablet_bootstrap.cc:492] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:20.086612 17973 ts_tablet_manager.cc:1403] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.397s	user 0.016s	sys 0.005s
I20260709 12:30:20.108366 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:20.124364 17971 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:20.124984 17971 ts_tablet_manager.cc:1403] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.517s	user 0.023s	sys 0.009s
I20260709 12:30:20.126150 18255 raft_consensus.cc:493] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.127068 18255 raft_consensus.cc:515] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.143357 18255 leader_election.cc:290] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:20.144848 17973 raft_consensus.cc:359] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.145722 17973 raft_consensus.cc:385] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.146075 17973 raft_consensus.cc:740] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.148818 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:20.149657 18257 raft_consensus.cc:515] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.152769 17973 consensus_queue.cc:260] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.136848 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:20.159405 17669 leader_election.cc:343] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.165915 17973 ts_tablet_manager.cc:1434] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 12:30:20.176153 17924 raft_consensus.cc:2393] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:20.192513 17973 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:20.194986 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:20.197026 17669 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
W20260709 12:30:20.200989 17666 leader_election.cc:343] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.201378 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:20.202147 17848 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.202867 18255 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.206605 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:20.209540 17700 raft_consensus.cc:2468] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.210597 18271 consensus_queue.cc:1048] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:20.215476 17669 leader_election.cc:343] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.216060 17669 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:20.219732 17973 tablet_bootstrap.cc:654] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.222975 18320 raft_consensus.cc:2749] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:20.226753 17667 leader_election.cc:343] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.227497 17667 leader_election.cc:304] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:20.234496 17742 leader_election.cc:304] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:20.237169 18033 raft_consensus.cc:2804] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.237548 17974 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:20.238221 17974 ts_tablet_manager.cc:1403] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.323s	user 0.054s	sys 0.096s
I20260709 12:30:20.242125 17972 tablet_bootstrap.cc:654] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.239496 18033 raft_consensus.cc:697] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:20.233878 18320 raft_consensus.cc:2749] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.246436 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:20.232910 18257 leader_election.cc:290] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:20.252812 17971 raft_consensus.cc:359] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.254005 17971 raft_consensus.cc:385] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.254362 17971 raft_consensus.cc:740] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.261606 17971 consensus_queue.cc:260] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.259375 17921 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.275491 17974 raft_consensus.cc:359] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.277168 17974 raft_consensus.cc:385] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.277585 17974 raft_consensus.cc:740] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.279620 18033 consensus_queue.cc:237] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.282867 17971 ts_tablet_manager.cc:1434] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.158s	user 0.009s	sys 0.000s
I20260709 12:30:20.284353 17971 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:20.293358 18255 raft_consensus.cc:2749] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.302069 18271 consensus_queue.cc:1048] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.311668 17974 consensus_queue.cc:260] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.329830 17974 ts_tablet_manager.cc:1434] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.005s
I20260709 12:30:20.320910 18292 raft_consensus.cc:493] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.339742 18292 raft_consensus.cc:515] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.345160 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:20.354657 17742 leader_election.cc:343] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.358479 17973 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:20.359900 17973 ts_tablet_manager.cc:1403] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.178s	user 0.014s	sys 0.011s
I20260709 12:30:20.368958 17973 raft_consensus.cc:359] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.369956 17973 raft_consensus.cc:385] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.370352 17973 raft_consensus.cc:740] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.372835 18292 leader_election.cc:290] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:20.379783 17973 consensus_queue.cc:260] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.373806 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:20.390906 17924 raft_consensus.cc:2468] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:20.392719 17744 leader_election.cc:304] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:20.400012 17973 ts_tablet_manager.cc:1434] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.004s
I20260709 12:30:20.398649 18033 raft_consensus.cc:2804] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.401178 17974 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:20.404567 17848 raft_consensus.cc:2468] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.406177 18033 raft_consensus.cc:493] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.423466 17973 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:20.423128 17598 catalog_manager.cc:5697] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:20.428031 17742 leader_election.cc:304] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:20.429795 18257 raft_consensus.cc:2804] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.430393 18257 raft_consensus.cc:697] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:20.431392 18257 consensus_queue.cc:237] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.437922 18329 raft_consensus.cc:493] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.438604 18329 raft_consensus.cc:515] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.446905 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:20.449995 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f87c2ee9429444babb65d1c8a1ec7c1" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:20.451169 17700 raft_consensus.cc:2393] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:20.455305 18152 consensus_queue.cc:1048] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.455283 17972 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:20.458520 17972 ts_tablet_manager.cc:1403] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.373s	user 0.020s	sys 0.013s
I20260709 12:30:20.422742 18033 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.462332 17972 raft_consensus.cc:359] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.463424 17972 raft_consensus.cc:385] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.444015 18329 leader_election.cc:290] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:20.463864 17972 raft_consensus.cc:740] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.465065 17972 consensus_queue.cc:260] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.472508 18332 raft_consensus.cc:493] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.473194 18332 raft_consensus.cc:515] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.475694 17889 leader_election.cc:304] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:20.477488 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:20.500607 17972 ts_tablet_manager.cc:1434] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.003s
I20260709 12:30:20.495997 17921 raft_consensus.cc:2468] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.503696 18332 leader_election.cc:290] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:20.480356 18230 raft_consensus.cc:3060] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.512104 18012 consensus_queue.cc:1048] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:20.497305 18368 log.cc:927] Time spent T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:30:20.525429 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:20.526293 17702 raft_consensus.cc:2468] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:20.528141 18033 raft_consensus.cc:515] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.531363 18033 leader_election.cc:290] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:20.532315 17890 leader_election.cc:343] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.533412 17890 leader_election.cc:304] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:20.535290 18332 raft_consensus.cc:2804] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.535804 18332 raft_consensus.cc:493] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.536221 18332 raft_consensus.cc:3060] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.541954 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:20.560470 17971 tablet_bootstrap.cc:654] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.582551 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:20.583431 17925 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:20.550772 17742 leader_election.cc:343] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.592952 17974 tablet_bootstrap.cc:654] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.593743 17973 tablet_bootstrap.cc:654] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.599671 17972 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:20.621240 18230 raft_consensus.cc:2749] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.655040 17599 catalog_manager.cc:5697] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.664726 18292 raft_consensus.cc:493] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.665441 18292 raft_consensus.cc:515] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.677043 17971 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:20.677748 17971 ts_tablet_manager.cc:1403] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.394s	user 0.019s	sys 0.051s
I20260709 12:30:20.678174 18281 raft_consensus.cc:493] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.683554 18281 raft_consensus.cc:515] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.685915 17972 tablet_bootstrap.cc:654] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.687167 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:20.687254 17973 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:20.693141 17973 ts_tablet_manager.cc:1403] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.285s	user 0.010s	sys 0.012s
I20260709 12:30:20.696214 18332 raft_consensus.cc:515] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.698787 18332 leader_election.cc:290] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:20.701263 18281 leader_election.cc:290] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:20.703557 17925 raft_consensus.cc:2468] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.704170 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:20.705978 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:20.715164 18292 leader_election.cc:290] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:20.715353 17702 raft_consensus.cc:3060] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.720166 17971 raft_consensus.cc:359] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.721102 17971 raft_consensus.cc:385] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.721428 17971 raft_consensus.cc:740] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.722196 17971 consensus_queue.cc:260] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.706228 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:20.723897 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:20.724602 17701 raft_consensus.cc:2468] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:20.723991 17744 leader_election.cc:304] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:20.729136 18256 raft_consensus.cc:2804] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.730243 18256 raft_consensus.cc:697] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:20.731546 18256 consensus_queue.cc:237] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:20.707171 17973 raft_consensus.cc:359] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.734808 17973 raft_consensus.cc:385] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.739257 17921 raft_consensus.cc:2468] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:20.740463 17816 leader_election.cc:304] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:20.741984 18271 raft_consensus.cc:2804] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.742568 18271 raft_consensus.cc:493] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.743032 18271 raft_consensus.cc:3060] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.745129 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:20.746485 17701 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:20.755527 17974 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:20.756140 17974 ts_tablet_manager.cc:1403] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.360s	user 0.031s	sys 0.035s
I20260709 12:30:20.759676 17974 raft_consensus.cc:359] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.760512 17974 raft_consensus.cc:385] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.760824 17974 raft_consensus.cc:740] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.761726 17742 leader_election.cc:304] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:20.762746 18292 raft_consensus.cc:2804] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.763290 18292 raft_consensus.cc:493] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.765076 18292 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.761581 17974 consensus_queue.cc:260] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:20.769107 17974 ts_tablet_manager.cc:1434] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 12:30:20.772686 17890 leader_election.cc:343] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.773917 17971 ts_tablet_manager.cc:1434] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 12:30:20.780267 17974 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:20.735198 17973 raft_consensus.cc:740] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.782785 17973 consensus_queue.cc:260] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.784075 17851 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:20.785764 18292 raft_consensus.cc:515] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.788446 18292 leader_election.cc:290] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:20.783208 17702 raft_consensus.cc:2468] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:20.790498 17890 leader_election.cc:304] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:20.791592 18329 raft_consensus.cc:2804] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.792178 18329 raft_consensus.cc:697] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:20.793104 18329 consensus_queue.cc:237] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.796906 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:20.797732 17702 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.797907 17974 tablet_bootstrap.cc:654] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.788535 17973 ts_tablet_manager.cc:1434] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.002s
I20260709 12:30:20.818892 17971 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:20.823933 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:20.824781 17851 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.828238 17972 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:20.828958 17972 ts_tablet_manager.cc:1403] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.234s	user 0.011s	sys 0.006s
I20260709 12:30:20.829419 17597 catalog_manager.cc:5697] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.835037 17972 raft_consensus.cc:359] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.858322 17972 raft_consensus.cc:385] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.858827 17972 raft_consensus.cc:740] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.851902 17599 catalog_manager.cc:5697] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.855763 18271 raft_consensus.cc:515] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.879555 17851 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.882280 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:20.883106 17925 raft_consensus.cc:3060] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.883817 17973 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:20.887712 17702 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:20.895284 18343 consensus_queue.cc:1048] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:30:20.893186 17972 consensus_queue.cc:260] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.923338 17973 tablet_bootstrap.cc:654] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.899704 18271 leader_election.cc:290] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:20.945147 17971 tablet_bootstrap.cc:654] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.948640 17972 ts_tablet_manager.cc:1434] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.102s	user 0.004s	sys 0.006s
I20260709 12:30:20.898756 17925 raft_consensus.cc:2468] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:20.950088 17818 leader_election.cc:304] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:20.951061 18271 raft_consensus.cc:2804] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.951642 18271 raft_consensus.cc:697] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:20.952670 18271 consensus_queue.cc:237] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:20.899541 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea37968d44c4ec5a4320301e4173d8b" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:20.954082 17703 raft_consensus.cc:3060] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.905676 17974 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:20.955731 17974 ts_tablet_manager.cc:1403] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.186s	user 0.043s	sys 0.035s
I20260709 12:30:20.955854 17972 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:20.969974 17974 raft_consensus.cc:359] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.970991 17974 raft_consensus.cc:385] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.971345 17974 raft_consensus.cc:740] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
W20260709 12:30:20.960297 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 46800.3us
I20260709 12:30:20.981144 17742 leader_election.cc:304] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:20.982496 18292 raft_consensus.cc:2804] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.983068 18292 raft_consensus.cc:697] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:20.984122 18292 consensus_queue.cc:237] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:20.993625 18343 consensus_queue.cc:1048] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.994457 17703 raft_consensus.cc:2468] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:21.002816 17971 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
W20260709 12:30:21.003283 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:21.003489 17971 ts_tablet_manager.cc:1403] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.185s	user 0.015s	sys 0.010s
I20260709 12:30:20.972154 17974 consensus_queue.cc:260] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.007397 17971 raft_consensus.cc:359] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.008492 17971 raft_consensus.cc:385] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.008908 17971 raft_consensus.cc:740] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.010205 17971 consensus_queue.cc:260] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.012668 17972 tablet_bootstrap.cc:654] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.057413 17971 ts_tablet_manager.cc:1434] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.053s	user 0.028s	sys 0.015s
I20260709 12:30:21.047859 17973 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:21.058305 17973 ts_tablet_manager.cc:1403] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.175s	user 0.053s	sys 0.022s
I20260709 12:30:21.059641 17971 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:21.026476 18257 consensus_queue.cc:1048] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:30:21.067246 17973 raft_consensus.cc:359] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.068478 17973 raft_consensus.cc:385] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.067961 17599 catalog_manager.cc:5697] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:21.068856 17973 raft_consensus.cc:740] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.069792 17973 consensus_queue.cc:260] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.073745 18343 consensus_queue.cc:1048] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.017493 17974 ts_tablet_manager.cc:1434] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 12:30:21.077654 17971 tablet_bootstrap.cc:654] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.078317 17973 ts_tablet_manager.cc:1434] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:30:21.077939 17598 catalog_manager.cc:5697] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:21.128569 17972 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:21.129359 17972 ts_tablet_manager.cc:1403] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.174s	user 0.017s	sys 0.000s
I20260709 12:30:21.131832 17973 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:21.132819 18257 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.141952 18257 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.145898 18257 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:21.156522 17972 raft_consensus.cc:359] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.157408 17972 raft_consensus.cc:385] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.157770 17972 raft_consensus.cc:740] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.161157 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:21.163028 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:21.163424 17971 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:21.164652 17971 ts_tablet_manager.cc:1403] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.105s	user 0.018s	sys 0.004s
W20260709 12:30:21.168431 17742 leader_election.cc:343] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:21.169752 17742 leader_election.cc:343] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.170356 17742 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:21.170846 17974 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:21.171155 18257 raft_consensus.cc:2749] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.172626 17972 consensus_queue.cc:260] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.175437 17971 raft_consensus.cc:359] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.185158 17971 raft_consensus.cc:385] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.192224 17973 tablet_bootstrap.cc:654] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.187247 17971 raft_consensus.cc:740] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.194020 17971 consensus_queue.cc:260] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.194697 17972 ts_tablet_manager.cc:1434] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.065s	user 0.010s	sys 0.000s
I20260709 12:30:21.196439 17972 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:21.196862 17971 ts_tablet_manager.cc:1434] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:30:21.198181 17971 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:21.222970 17972 tablet_bootstrap.cc:654] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.219435 18257 consensus_queue.cc:1048] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:30:21.242673 18329 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.260236 17971 tablet_bootstrap.cc:654] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.248978 18329 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.262997 18329 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:21.264638 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:21.266475 18343 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.266721 17701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:21.267275 18343 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.275101 17890 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:21.276559 18377 raft_consensus.cc:2749] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.279322 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:21.289000 18304 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.283797 18343 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:21.301734 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:21.301972 18304 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.303728 17703 raft_consensus.cc:2468] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:21.305824 18304 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:21.298925 17744 leader_election.cc:343] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.315294 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:21.316766 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:21.317592 17667 leader_election.cc:343] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.317659 17923 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:21.320216 17851 raft_consensus.cc:3060] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.320756 17669 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:21.322191 18304 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.322741 18304 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.323334 18304 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:21.329185 17742 consensus_peers.cc:597] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:21.331845 18147 raft_consensus.cc:493] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.332747 18147 raft_consensus.cc:515] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.330336 17742 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:21.336735 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:21.337750 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:21.338562 17669 leader_election.cc:343] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.339254 18147 leader_election.cc:290] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:21.337967 18256 raft_consensus.cc:2804] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.342396 18256 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.343070 18256 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.343313 18304 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.346570 18230 consensus_queue.cc:1048] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:21.348272 17973 tablet_bootstrap.cc:492] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:21.348912 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:21.348943 17973 ts_tablet_manager.cc:1403] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.217s	user 0.018s	sys 0.007s
W20260709 12:30:21.365746 17667 leader_election.cc:343] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.370100 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:21.371354 17923 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.360620 18304 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:21.385844 18271 consensus_queue.cc:1048] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.408452 17972 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:21.409296 17972 ts_tablet_manager.cc:1403] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.213s	user 0.016s	sys 0.015s
I20260709 12:30:21.411737 17974 tablet_bootstrap.cc:654] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.409983 17973 raft_consensus.cc:359] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.414474 17973 raft_consensus.cc:385] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.414851 17973 raft_consensus.cc:740] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.420269 17973 consensus_queue.cc:260] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.423333 17923 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:21.429883 18281 consensus_queue.cc:1048] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:21.430713 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 73631.7us
W20260709 12:30:21.409688 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 64377.1us
W20260709 12:30:21.435074 17666 leader_election.cc:343] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.435755 17666 leader_election.cc:304] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:21.446188 18396 raft_consensus.cc:2749] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.447803 17669 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:21.452694 17973 ts_tablet_manager.cc:1434] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.001s
I20260709 12:30:21.476068 17973 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:21.480952 17971 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:21.481712 17971 ts_tablet_manager.cc:1403] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.284s	user 0.040s	sys 0.016s
I20260709 12:30:21.472805 17972 raft_consensus.cc:359] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.485596 17972 raft_consensus.cc:385] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.485929 17972 raft_consensus.cc:740] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.486661 17972 consensus_queue.cc:260] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.493031 18271 raft_consensus.cc:493] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.493718 18271 raft_consensus.cc:515] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.495954 17972 ts_tablet_manager.cc:1434] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.003s
I20260709 12:30:21.498142 18256 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.500029 18230 consensus_queue.cc:1048] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.492810 17971 raft_consensus.cc:359] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.511948 17971 raft_consensus.cc:385] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.485267 18304 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:21.512919 17971 raft_consensus.cc:740] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.527920 17971 consensus_queue.cc:260] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.528756 18033 consensus_queue.cc:1048] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:21.529430 18304 raft_consensus.cc:697] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:21.536924 18304 consensus_queue.cc:237] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.538643 17972 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:21.545782 17971 ts_tablet_manager.cc:1434] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 12:30:21.551563 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:21.543944 18271 leader_election.cc:290] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:21.564209 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:21.566350 17818 leader_election.cc:304] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:21.579039 17971 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:21.579589 17974 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:21.583987 17973 tablet_bootstrap.cc:654] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.584925 17974 ts_tablet_manager.cc:1403] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.414s	user 0.017s	sys 0.014s
I20260709 12:30:21.599355 18256 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:21.601871 18271 raft_consensus.cc:2749] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:21.617225 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:21.623018 17744 leader_election.cc:343] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.625003 17974 raft_consensus.cc:359] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.625862 17974 raft_consensus.cc:385] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.626158 17974 raft_consensus.cc:740] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.636082 17972 tablet_bootstrap.cc:654] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.637023 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:21.648046 17703 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.649178 18405 consensus_queue.cc:1048] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.650533 17924 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.662317 17974 consensus_queue.cc:260] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.674916 18230 raft_consensus.cc:493] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.675683 18230 raft_consensus.cc:515] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.678493 18230 leader_election.cc:290] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:21.682245 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:21.684544 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:21.685408 17778 raft_consensus.cc:2468] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
W20260709 12:30:21.694020 17890 leader_election.cc:343] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.697888 17889 leader_election.cc:304] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:21.699738 18332 raft_consensus.cc:2804] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.703079 18332 raft_consensus.cc:493] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.704612 18332 raft_consensus.cc:3060] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.708644 17971 tablet_bootstrap.cc:654] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.700711 17974 ts_tablet_manager.cc:1434] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.115s	user 0.008s	sys 0.002s
W20260709 12:30:21.721379 18411 log.cc:927] Time spent T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:30:21.722666 17973 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:21.734966 17973 ts_tablet_manager.cc:1403] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.259s	user 0.029s	sys 0.010s
I20260709 12:30:21.748737 18012 raft_consensus.cc:493] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.749884 18012 raft_consensus.cc:515] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.753170 17599 catalog_manager.cc:5697] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:21.780519 17974 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:21.785064 17973 raft_consensus.cc:359] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.786314 17973 raft_consensus.cc:385] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.786702 17973 raft_consensus.cc:740] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.802460 17973 consensus_queue.cc:260] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:21.826364 17971 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:21.827296 17971 ts_tablet_manager.cc:1403] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.249s	user 0.026s	sys 0.023s
I20260709 12:30:21.829504 18012 leader_election.cc:290] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:21.842612 17972 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:21.843573 17972 ts_tablet_manager.cc:1403] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.315s	user 0.030s	sys 0.018s
I20260709 12:30:21.846663 17972 raft_consensus.cc:359] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.848073 17972 raft_consensus.cc:385] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.848587 17972 raft_consensus.cc:740] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.853070 17972 consensus_queue.cc:260] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.868510 17972 ts_tablet_manager.cc:1434] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:30:21.851328 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:21.870321 17972 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:21.876709 17703 raft_consensus.cc:2468] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:21.881438 17742 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:21.882876 18402 raft_consensus.cc:2804] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:21.883546 18402 raft_consensus.cc:697] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:21.884629 18402 consensus_queue.cc:237] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.869388 18332 raft_consensus.cc:515] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:21.900588 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:21.900849 18332 leader_election.cc:290] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:21.901467 17774 raft_consensus.cc:3060] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.904714 18304 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.905411 18304 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.905323 17974 tablet_bootstrap.cc:654] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.924327 17972 tablet_bootstrap.cc:654] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.926918 17852 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.921485 18304 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:21.925957 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:21.943349 17973 ts_tablet_manager.cc:1434] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.208s	user 0.017s	sys 0.005s
I20260709 12:30:21.932716 18410 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.946449 17849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:21.947110 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:21.947577 17973 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:21.937075 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:21.954133 17666 leader_election.cc:343] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:21.970489 17667 leader_election.cc:343] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.971364 17667 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:21.952363 18410 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:21.974681 18396 raft_consensus.cc:2749] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.976445 18410 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:21.979169 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:21.980106 17776 raft_consensus.cc:2468] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:21.981884 17666 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:21.986372 18396 raft_consensus.cc:2804] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.987437 18396 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.987942 18396 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.990767 17971 raft_consensus.cc:359] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.991685 17971 raft_consensus.cc:385] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.992846 17971 raft_consensus.cc:740] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.993649 17971 consensus_queue.cc:260] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:21.994365 17818 leader_election.cc:304] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:21.991781 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:21.998216 17667 leader_election.cc:343] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.001554 17971 ts_tablet_manager.cc:1434] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.174s	user 0.005s	sys 0.003s
W20260709 12:30:22.002295 17890 leader_election.cc:343] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.002888 17971 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:22.028985 17974 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:22.030897 17974 ts_tablet_manager.cc:1403] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.254s	user 0.011s	sys 0.007s
I20260709 12:30:22.025382 18396 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.035331 18410 consensus_queue.cc:1048] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:22.001588 17973 tablet_bootstrap.cc:654] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.011672 17774 raft_consensus.cc:2468] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:22.016017 18422 raft_consensus.cc:2749] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:22.038676 17889 leader_election.cc:304] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:22.035908 17974 raft_consensus.cc:359] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.041126 17974 raft_consensus.cc:385] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.047917 18332 raft_consensus.cc:2804] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:22.053857 18332 raft_consensus.cc:697] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:22.056253 18332 consensus_queue.cc:237] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.041515 17974 raft_consensus.cc:740] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.067142 17972 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:22.068222 17972 ts_tablet_manager.cc:1403] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.198s	user 0.024s	sys 0.009s
I20260709 12:30:22.073221 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:22.074023 17776 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.067535 17974 consensus_queue.cc:260] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.076803 17972 raft_consensus.cc:359] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.084789 17972 raft_consensus.cc:385] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.085083 17972 raft_consensus.cc:740] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.085790 17972 consensus_queue.cc:260] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.091688 17974 ts_tablet_manager.cc:1434] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.003s
I20260709 12:30:22.099478 17972 ts_tablet_manager.cc:1434] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:30:22.101096 17972 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:22.112078 17972 tablet_bootstrap.cc:654] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.115068 18396 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:22.124714 17974 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:22.127696 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
W20260709 12:30:22.129596 17667 leader_election.cc:343] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.140128 17971 tablet_bootstrap.cc:654] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.143872 17973 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:22.147403 17848 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.148352 17973 ts_tablet_manager.cc:1403] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.201s	user 0.033s	sys 0.022s
I20260709 12:30:22.157812 17776 raft_consensus.cc:2468] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:22.161247 17666 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:22.150115 17974 tablet_bootstrap.cc:654] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.171793 18396 raft_consensus.cc:2804] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:22.177558 18396 raft_consensus.cc:697] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:22.196265 18332 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.205021 18332 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.178558 18396 consensus_queue.cc:237] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.206315 17598 catalog_manager.cc:5697] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:22.211884 17973 raft_consensus.cc:359] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.215493 17973 raft_consensus.cc:385] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.215879 17973 raft_consensus.cc:740] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.216846 17973 consensus_queue.cc:260] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:22.249436 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:22.252789 18332 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:22.254361 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:22.254992 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.256043 17703 raft_consensus.cc:2393] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:22.257496 17890 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:22.257689 18422 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.258366 18422 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.254889 17599 catalog_manager.cc:5697] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:22.265885 17973 ts_tablet_manager.cc:1434] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.002s
I20260709 12:30:22.267319 18377 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.276688 18422 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.279907 17974 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:22.280550 17974 ts_tablet_manager.cc:1403] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.156s	user 0.015s	sys 0.007s
I20260709 12:30:22.281402 17971 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:22.282102 17971 ts_tablet_manager.cc:1403] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.279s	user 0.014s	sys 0.005s
I20260709 12:30:22.284315 17974 raft_consensus.cc:359] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.285202 17974 raft_consensus.cc:385] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.284940 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.285801 17974 raft_consensus.cc:740] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.286783 17974 consensus_queue.cc:260] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.285117 17971 raft_consensus.cc:359] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.287340 17920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:22.287832 17971 raft_consensus.cc:385] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.289217 17974 ts_tablet_manager.cc:1434] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:22.291370 17818 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:22.292161 17973 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:22.293340 17972 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:22.293959 17972 ts_tablet_manager.cc:1403] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.193s	user 0.023s	sys 0.005s
I20260709 12:30:22.288215 17971 raft_consensus.cc:740] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.298352 17971 consensus_queue.cc:260] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.296486 18422 raft_consensus.cc:2749] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:22.300344 18304 consensus_queue.cc:1048] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:22.306833 17974 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:22.319368 17974 tablet_bootstrap.cc:654] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.325151 17972 raft_consensus.cc:359] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.328372 18377 raft_consensus.cc:2749] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:22.330013 17972 raft_consensus.cc:385] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.331292 17972 raft_consensus.cc:740] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.333810 17972 consensus_queue.cc:260] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.337080 17973 tablet_bootstrap.cc:654] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.342550 17971 ts_tablet_manager.cc:1434] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:30:22.355616 17974 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:22.358891 17601 catalog_manager.cc:5697] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:22.366753 17974 ts_tablet_manager.cc:1403] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.017s
I20260709 12:30:22.351280 17971 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:22.350612 17972 ts_tablet_manager.cc:1434] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.056s	user 0.011s	sys 0.000s
I20260709 12:30:22.372325 17974 raft_consensus.cc:359] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.374316 17974 raft_consensus.cc:385] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.374651 17974 raft_consensus.cc:740] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.375150 17972 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:22.375617 17974 consensus_queue.cc:260] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.379755 17974 ts_tablet_manager.cc:1434] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260709 12:30:22.381515 17974 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:22.370544 18402 consensus_queue.cc:1048] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:30:22.402498 18402 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.408438 17974 tablet_bootstrap.cc:654] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.411439 17971 tablet_bootstrap.cc:654] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.403169 18402 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.420608 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.421458 17703 raft_consensus.cc:2468] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:22.422935 17742 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:22.431506 18405 raft_consensus.cc:2804] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:22.432046 18405 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:22.432444 18405 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.436110 17972 tablet_bootstrap.cc:654] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.436317 18402 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:22.446952 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:22.450192 17972 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:22.450897 17972 ts_tablet_manager.cc:1403] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.000s
W20260709 12:30:22.452052 17742 leader_election.cc:343] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.456234 17973 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:22.456889 17973 ts_tablet_manager.cc:1403] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.177s	user 0.010s	sys 0.025s
I20260709 12:30:22.460134 17973 raft_consensus.cc:359] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.464120 17973 raft_consensus.cc:385] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.464529 17973 raft_consensus.cc:740] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.466109 17974 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:22.466780 17974 ts_tablet_manager.cc:1403] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.007s
I20260709 12:30:22.467011 18404 raft_consensus.cc:493] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.467509 17973 consensus_queue.cc:260] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.467715 18404 raft_consensus.cc:515] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.472038 18404 leader_election.cc:290] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.473440 17973 ts_tablet_manager.cc:1434] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:30:22.474867 17973 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:22.477981 17974 raft_consensus.cc:359] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.478256 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.478904 17974 raft_consensus.cc:385] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.479293 17974 raft_consensus.cc:740] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.479501 17703 raft_consensus.cc:2468] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:22.480103 17974 consensus_queue.cc:260] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.466794 17972 raft_consensus.cc:359] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.486531 17972 raft_consensus.cc:385] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.486876 17972 raft_consensus.cc:740] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.487681 18377 consensus_queue.cc:1048] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:22.496894 18422 raft_consensus.cc:493] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.497557 18422 raft_consensus.cc:515] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.497983 17974 ts_tablet_manager.cc:1434] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:30:22.499770 17974 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: Bootstrap starting.
I20260709 12:30:22.500732 18422 leader_election.cc:290] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.501744 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:22.503490 17818 leader_election.cc:343] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.504907 17742 leader_election.cc:304] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:22.510072 17698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.511073 17698 raft_consensus.cc:2468] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 0.
I20260709 12:30:22.507071 18402 raft_consensus.cc:2804] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:22.516480 18402 raft_consensus.cc:493] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:22.516970 18402 raft_consensus.cc:3060] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.487668 17972 consensus_queue.cc:260] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:22.522141 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 50833.9us
I20260709 12:30:22.524892 17816 leader_election.cc:304] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:22.528414 18256 consensus_queue.cc:1048] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:30:22.535439 17974 tablet_bootstrap.cc:654] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.528856 18422 raft_consensus.cc:2804] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:22.536481 17890 consensus_peers.cc:597] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:22.536886 18422 raft_consensus.cc:493] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:22.537319 18422 raft_consensus.cc:3060] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.529110 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:22.545715 17924 raft_consensus.cc:2468] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:22.547322 17972 ts_tablet_manager.cc:1434] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.001s
I20260709 12:30:22.550827 17973 tablet_bootstrap.cc:654] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.570794 17972 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:22.572353 18422 raft_consensus.cc:515] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.587134 17972 tablet_bootstrap.cc:654] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.596063 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:22.598457 17818 leader_election.cc:343] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.599356 18396 raft_consensus.cc:493] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.601820 18396 raft_consensus.cc:515] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.604058 17973 tablet_bootstrap.cc:492] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:22.604877 17973 ts_tablet_manager.cc:1403] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.130s	user 0.009s	sys 0.005s
I20260709 12:30:22.607923 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:22.609980 17667 leader_election.cc:343] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.610491 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:22.611377 17703 raft_consensus.cc:3060] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.612424 17972 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: No bootstrap required, opened a new log
I20260709 12:30:22.613047 17972 ts_tablet_manager.cc:1403] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.001s
I20260709 12:30:22.614048 18422 leader_election.cc:290] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.616073 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:22.616902 17776 raft_consensus.cc:2468] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 0.
I20260709 12:30:22.618613 17666 leader_election.cc:304] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:22.619621 18320 raft_consensus.cc:2804] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:22.620369 18320 raft_consensus.cc:493] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:22.620815 18320 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.624527 18396 leader_election.cc:290] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:22.625773 18405 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.627036 17972 raft_consensus.cc:359] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.632076 18320 raft_consensus.cc:515] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.633364 17971 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:22.634534 17972 raft_consensus.cc:385] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.634964 17972 raft_consensus.cc:740] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.635038 17971 ts_tablet_manager.cc:1403] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.284s	user 0.027s	sys 0.018s
I20260709 12:30:22.627765 17973 raft_consensus.cc:359] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.636986 17973 raft_consensus.cc:385] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.637352 17973 raft_consensus.cc:740] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.638378 17973 consensus_queue.cc:260] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.633581 17703 raft_consensus.cc:2468] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:22.566608 18402 raft_consensus.cc:515] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.640746 17816 leader_election.cc:304] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
I20260709 12:30:22.642623 18421 raft_consensus.cc:2804] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:22.643256 18421 raft_consensus.cc:697] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:22.644248 18421 consensus_queue.cc:237] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.644728 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:22.646265 17776 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.646492 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
W20260709 12:30:22.649092 17667 leader_election.cc:343] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.644268 17971 raft_consensus.cc:359] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.653699 17971 raft_consensus.cc:385] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.654609 17971 raft_consensus.cc:740] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.656397 17971 consensus_queue.cc:260] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.666267 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:22.667201 17924 raft_consensus.cc:3060] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.673923 18147 consensus_queue.cc:1048] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:22.692592 17972 consensus_queue.cc:260] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.700613 17971 ts_tablet_manager.cc:1434] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 12:30:22.708909 17971 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:22.728826 17776 raft_consensus.cc:2468] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 1.
I20260709 12:30:22.732314 17924 raft_consensus.cc:2468] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:22.735423 17973 ts_tablet_manager.cc:1434] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.000s
I20260709 12:30:22.736758 17744 leader_election.cc:304] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:22.737797 17973 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:22.738662 18405 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:22.740327 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
W20260709 12:30:22.742605 17742 leader_election.cc:343] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:22.740149 18033 raft_consensus.cc:2804] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:22.743831 18033 raft_consensus.cc:697] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:22.744805 18033 consensus_queue.cc:237] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.645011 18320 leader_election.cc:290] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:22.756090 18451 log.cc:927] Time spent T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:30:22.797252 17972 ts_tablet_manager.cc:1434] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.184s	user 0.014s	sys 0.003s
I20260709 12:30:22.804593 17974 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: No bootstrap required, opened a new log
I20260709 12:30:22.805634 17974 ts_tablet_manager.cc:1403] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: Time spent bootstrapping tablet: real 0.306s	user 0.023s	sys 0.017s
I20260709 12:30:22.815079 17974 raft_consensus.cc:359] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.817909 17974 raft_consensus.cc:385] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:22.818336 17974 raft_consensus.cc:740] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Initialized, Role: FOLLOWER
I20260709 12:30:22.819211 17974 consensus_queue.cc:260] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.834244 17666 leader_election.cc:304] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:22.837004 17974 ts_tablet_manager.cc:1434] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:30:22.858204 17971 tablet_bootstrap.cc:654] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.861857 18304 raft_consensus.cc:2804] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:22.868646 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:22.869422 17703 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.868997 17973 tablet_bootstrap.cc:654] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:22.871691 18256 raft_consensus.cc:493] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.839360 18402 leader_election.cc:290] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.872385 18256 raft_consensus.cc:515] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:22.881332 18256 leader_election.cc:290] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:22.882350 18304 raft_consensus.cc:697] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:22.883630 18304 consensus_queue.cc:237] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:22.900731 17703 raft_consensus.cc:2468] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:22.904585 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bda730805b64dda88c94c3072246211" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:22.905395 17703 raft_consensus.cc:3060] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:22.905344 17924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:22.922935 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:22.923941 17700 raft_consensus.cc:2468] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:22.945389 18459 raft_consensus.cc:493] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.946341 18459 raft_consensus.cc:515] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:22.951263 18422 raft_consensus.cc:493] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.952196 18422 raft_consensus.cc:515] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:22.956342 18422 leader_election.cc:290] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:22.940061 17703 raft_consensus.cc:2468] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
W20260709 12:30:22.983498 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 50684.2us
I20260709 12:30:22.984534 17742 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6
W20260709 12:30:22.980595 17744 leader_election.cc:343] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:23.005918 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 70427.8us
I20260709 12:30:23.005919 18463 raft_consensus.cc:2804] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:23.010150 18421 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:23.063619 17973 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:23.040128 18470 raft_consensus.cc:493] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:23.064767 17973 ts_tablet_manager.cc:1403] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.327s	user 0.013s	sys 0.004s
I20260709 12:30:23.064944 18470 raft_consensus.cc:515] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:23.068766 18470 leader_election.cc:290] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:23.077061 18464 log.cc:927] Time spent T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:30:23.053728 18421 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:23.093452 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 61706.5us
I20260709 12:30:23.097306 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:23.104920 17852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:23.110828 17971 tablet_bootstrap.cc:492] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:23.111552 17971 ts_tablet_manager.cc:1403] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.403s	user 0.009s	sys 0.010s
I20260709 12:30:23.108250 17698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3be2bb16cb74f8fb343216495dc07a3" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:23.155812 17702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:23.142859 18463 raft_consensus.cc:697] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:23.172302 18404 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:23.174728 17973 raft_consensus.cc:359] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:23.178999 17973 raft_consensus.cc:385] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:23.162951 17698 raft_consensus.cc:2393] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:23.172324 17846 raft_consensus.cc:3060] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:23.184311 17667 leader_election.cc:343] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:23.186013 18421 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:23.189363 18404 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:23.195854 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:23.198140 18459 leader_election.cc:290] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:23.203526 17890 leader_election.cc:304] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:23.204952 18404 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:23.205063 18230 raft_consensus.cc:3060] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:23.179484 17973 raft_consensus.cc:740] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:23.206466 17973 consensus_queue.cc:260] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:23.207437 17742 leader_election.cc:304] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250
W20260709 12:30:23.208498 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 199786us
I20260709 12:30:23.212225 17971 raft_consensus.cc:359] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:23.213124 17971 raft_consensus.cc:385] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:23.213429 17971 raft_consensus.cc:740] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:23.214169 17971 consensus_queue.cc:260] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:23.215760 18256 raft_consensus.cc:2804] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:23.216315 18256 raft_consensus.cc:493] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:23.216691 18256 raft_consensus.cc:3060] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:23.221866 17973 ts_tablet_manager.cc:1434] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.157s	user 0.010s	sys 0.000s
I20260709 12:30:23.214532 17850 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:23.229523 17971 ts_tablet_manager.cc:1434] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.118s	user 0.002s	sys 0.005s
I20260709 12:30:23.235728 17921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:23.185094 18463 consensus_queue.cc:237] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:23.248225 17669 leader_election.cc:343] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:23.258013 17971 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:23.274636 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:23.275568 17851 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:23.258438 17669 leader_election.cc:304] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:23.280874 18147 raft_consensus.cc:2749] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:23.296991 17601 catalog_manager.cc:5697] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:23.254772 18230 raft_consensus.cc:2749] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:23.323185 18256 raft_consensus.cc:515] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:23.327451 18458 consensus_queue.cc:1048] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:23.326911 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:23.333173 17667 consensus_peers.cc:597] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:23.358179 17973 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:23.343703 17598 catalog_manager.cc:5697] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:23.406546 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:23.434779 18256 leader_election.cc:290] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:23.435422 18425 consensus_queue.cc:1048] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:30:23.438340 18420 consensus_queue.cc:1048] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:23.444726 17971 tablet_bootstrap.cc:654] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:23.449519 17973 tablet_bootstrap.cc:654] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:23.456658 17925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:23.457501 17925 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 0.
I20260709 12:30:23.465020 17815 leader_election.cc:304] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:23.480710 17744 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:23.484005 17700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:23.484766 17700 raft_consensus.cc:3060] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:23.470412 18337 raft_consensus.cc:2749] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:23.499536 18476 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:23.519726 18476 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:23.520141 18476 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:23.527395 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:23.541692 17700 raft_consensus.cc:2468] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:23.704682 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:23.965108 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 348218us
I20260709 12:30:23.966044 17742 leader_election.cc:304] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
W20260709 12:30:23.974397 17815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 357888us
I20260709 12:30:23.954788 18476 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:24.003566 18256 raft_consensus.cc:2804] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:24.004722 17973 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:24.005379 17973 ts_tablet_manager.cc:1403] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.687s	user 0.020s	sys 0.013s
I20260709 12:30:24.005234 18256 raft_consensus.cc:697] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:24.006692 18256 consensus_queue.cc:237] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:24.008982 17973 raft_consensus.cc:359] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:24.029614 17973 raft_consensus.cc:385] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:24.016140 17815 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:24.030311 17973 raft_consensus.cc:740] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:24.044559 17973 consensus_queue.cc:260] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:24.007512 18488 log.cc:927] Time spent T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Append to log took a long time: real 0.392s	user 0.006s	sys 0.006s
W20260709 12:30:24.065583 17815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 49592.5us
I20260709 12:30:24.024773 18476 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:24.074031 18471 raft_consensus.cc:2749] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:24.130926 17973 ts_tablet_manager.cc:1434] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.125s	user 0.022s	sys 0.008s
I20260709 12:30:24.148300 17973 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:24.155169 17601 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50444 (request call id 87) took 1034 ms. Trace:
I20260709 12:30:24.161537 17597 catalog_manager.cc:5697] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:24.155807 17601 rpcz_store.cc:276] 0709 12:30:23.120482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:30:23.120744 (+   262us) service_pool.cc:224] Handling call
0709 12:30:24.155071 (+1034327us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:30:23.370845 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:30:23.371510 (+   665us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:30:23.371543 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:30:23.371562 (+    19us) tablet.cc:662] Decoding operations
0709 12:30:23.371952 (+   390us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:30:23.372051 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:30:23.372088 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:30:23.372303 (+   215us) tablet.cc:701] Row locks acquired
0709 12:30:23.372320 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:30:23.372718 (+   398us) write_op.cc:270] Start()
0709 12:30:23.372842 (+   124us) write_op.cc:276] Timestamp: P: 1783600223372655 usec, L: 0
0709 12:30:23.372866 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:30:23.373703 (+   837us) log.cc:844] Serialized 537 byte log entry
0709 12:30:23.953778 (+580075us) op_driver.cc:464] REPLICATION: finished
0709 12:30:24.063826 (+110048us) write_op.cc:301] APPLY: starting
0709 12:30:24.072977 (+  9151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:30:24.086261 (+ 13284us) tablet.cc:1370] finished BulkCheckPresence
0709 12:30:24.098028 (+ 11767us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:30:24.098287 (+   259us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:30:24.098430 (+   143us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:30:24.098465 (+    35us) write_op.cc:312] APPLY: finished
0709 12:30:24.098652 (+   187us) log.cc:844] Serialized 22 byte log entry
0709 12:30:24.098806 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:30:24.098871 (+    65us) write_op.cc:454] Released schema lock
0709 12:30:24.098909 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:30:24.099428 (+   519us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:30:24.099460 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":63666,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":73776,"apply.run_cpu_time_us":2258,"apply.run_wall_time_us":54510,"num_ops":1,"prepare.queue_time_us":23665,"prepare.run_cpu_time_us":3244,"prepare.run_wall_time_us":3246,"replication_time_us":338668,"thread_start_us":36823,"threads_started":2}]]}
I20260709 12:30:24.170703 17597 catalog_manager.cc:5697] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:24.182548 17971 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:24.183187 17971 ts_tablet_manager.cc:1403] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.952s	user 0.025s	sys 0.024s
I20260709 12:30:24.188791 17971 raft_consensus.cc:359] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:24.189754 17971 raft_consensus.cc:385] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:24.190111 17971 raft_consensus.cc:740] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:24.191012 17971 consensus_queue.cc:260] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:24.192469 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:24.194806 17847 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:24.203759 18405 consensus_queue.cc:1048] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:24.205433 18497 consensus_queue.cc:1048] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:24.194077 17973 tablet_bootstrap.cc:654] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:24.193782 18396 consensus_queue.cc:1048] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:24.208425 17742 consensus_peers.cc:597] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.217473 17971 ts_tablet_manager.cc:1434] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 12:30:24.232827 17971 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: Bootstrap starting.
I20260709 12:30:24.287101 17973 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:24.287957 17847 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
W20260709 12:30:24.225940 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 45631.5us
I20260709 12:30:24.292865 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:24.304394 17922 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:24.305262 17744 leader_election.cc:343] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:24.305366 17598 catalog_manager.cc:5697] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:24.307054 18477 consensus_queue.cc:1048] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:24.308090 17973 ts_tablet_manager.cc:1403] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.003s
I20260709 12:30:24.316936 17922 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:24.299690 17971 tablet_bootstrap.cc:654] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:24.328827 17742 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:24.348023 18476 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:24.348709 18476 raft_consensus.cc:697] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:24.349772 18476 consensus_queue.cc:237] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:24.332700 17973 raft_consensus.cc:359] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:24.356511 17973 raft_consensus.cc:385] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:24.357762 17973 raft_consensus.cc:740] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:24.360514 17973 consensus_queue.cc:260] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:24.382555 17971 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: No bootstrap required, opened a new log
I20260709 12:30:24.385962 17971 ts_tablet_manager.cc:1403] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: Time spent bootstrapping tablet: real 0.153s	user 0.017s	sys 0.000s
I20260709 12:30:24.390040 17971 raft_consensus.cc:359] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:24.395509 17971 raft_consensus.cc:385] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:24.396425 17971 raft_consensus.cc:740] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Initialized, Role: FOLLOWER
I20260709 12:30:24.398573 17973 ts_tablet_manager.cc:1434] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.090s	user 0.006s	sys 0.002s
I20260709 12:30:24.401683 17973 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: Bootstrap starting.
I20260709 12:30:24.405118 17971 consensus_queue.cc:260] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:24.409399 17971 ts_tablet_manager.cc:1434] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:30:24.422909 17973 tablet_bootstrap.cc:654] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:24.445113 17844 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:24.471553 18496 raft_consensus.cc:493] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:24.472335 18496 raft_consensus.cc:515] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:24.477310 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:24.480837 18496 leader_election.cc:290] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:24.473534 17599 catalog_manager.cc:5697] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:24.505524 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:24.506881 17703 raft_consensus.cc:2468] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:24.514456 17890 leader_election.cc:304] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:24.480890 17847 raft_consensus.cc:2468] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 0.
I20260709 12:30:24.529814 18498 raft_consensus.cc:2804] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:24.571491 18498 raft_consensus.cc:493] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:24.627036 18498 raft_consensus.cc:3060] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:24.739193 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 96948.9us
W20260709 12:30:24.742540 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 110258us
W20260709 12:30:24.745249 17815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 116657us
W20260709 12:30:24.738801 18520 log.cc:927] Time spent T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:30:24.803601 17973 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: No bootstrap required, opened a new log
I20260709 12:30:24.804980 17973 ts_tablet_manager.cc:1403] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: Time spent bootstrapping tablet: real 0.404s	user 0.055s	sys 0.063s
I20260709 12:30:24.810432 18470 raft_consensus.cc:493] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:24.811254 18470 raft_consensus.cc:515] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:24.813717 18470 leader_election.cc:290] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:24.815644 18518 log.cc:927] Time spent T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:30:24.843748 17699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:24.845026 17699 raft_consensus.cc:2393] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30d17e55b097491f895c84a3443cb250 in current term 1: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:24.858330 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae4ca22644a4128bc2662c6bb502884" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:24.877151 17666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 131387us
I20260709 12:30:24.884438 17890 leader_election.cc:304] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:24.891037 18496 raft_consensus.cc:3060] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:24.948235 17973 raft_consensus.cc:359] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:24.949254 17973 raft_consensus.cc:385] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:24.948616 18256 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:24.951880 17973 raft_consensus.cc:740] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Initialized, Role: FOLLOWER
I20260709 12:30:24.953435 17973 consensus_queue.cc:260] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:24.980155 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 95832.4us
W20260709 12:30:24.993520 17649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:30:25.001786 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 154396us
I20260709 12:30:25.003257 17973 ts_tablet_manager.cc:1434] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: Time spent starting tablet: real 0.197s	user 0.009s	sys 0.001s
W20260709 12:30:24.977905 17667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 48177.7us
I20260709 12:30:25.039465 17844 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:25.043915 18499 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:25.072563 18498 raft_consensus.cc:515] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:24.981575 18476 consensus_queue.cc:1048] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.079s
W20260709 12:30:25.096442 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 106558us
I20260709 12:30:25.100849 18230 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:25.104068 18230 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:25.106523 18230 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:25.110339 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:25.075249 18498 leader_election.cc:290] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:25.120674 17699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
I20260709 12:30:25.121459 17699 raft_consensus.cc:3060] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:25.122951 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:25.123773 17847 raft_consensus.cc:3060] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:25.129585 17703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1fbecc92da4549828813c3a7185b1a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" is_pre_election: true
I20260709 12:30:25.139266 17890 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:25.154352 18332 raft_consensus.cc:2749] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.137439 17847 raft_consensus.cc:2468] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:25.162376 17890 leader_election.cc:304] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:25.199118 18535 raft_consensus.cc:2804] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:25.211992 18535 raft_consensus.cc:697] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:25.213063 18535 consensus_queue.cc:237] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:25.205248 18496 raft_consensus.cc:2749] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:25.276432 17699 raft_consensus.cc:2468] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:25.456682 18256 consensus_queue.cc:1048] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:25.541862 17600 catalog_manager.cc:5697] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 1 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:25.651436 18498 consensus_queue.cc:1048] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:25.688387 18498 consensus_queue.cc:1048] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:26.327441 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 78615.1us
I20260709 12:30:26.363857  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
W20260709 12:30:26.430434 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 76885.8us
W20260709 12:30:26.467994 17816 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 155 similar messages]
W20260709 12:30:26.517584 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 58290.2us
W20260709 12:30:26.521792 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 62160.7us
W20260709 12:30:26.631516 17890 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.633816 17816 consensus_peers.cc:597] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.636104 17816 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.636039 17890 consensus_peers.cc:597] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.639865 17816 consensus_peers.cc:597] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.641909 17816 consensus_peers.cc:597] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.642763 17816 consensus_peers.cc:597] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.645128 17816 consensus_peers.cc:597] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.646878 17890 consensus_peers.cc:597] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.656544 17890 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.658938 17816 consensus_peers.cc:597] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.668457 17816 consensus_peers.cc:597] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.669780 17816 consensus_peers.cc:597] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.685614 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.699134 17742 consensus_peers.cc:597] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.700335 17742 consensus_peers.cc:597] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.701171 17742 consensus_peers.cc:597] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.701954 17742 consensus_peers.cc:597] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.702714 17742 consensus_peers.cc:597] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.703478 17742 consensus_peers.cc:597] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.721060 17816 consensus_peers.cc:597] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.722375 17742 consensus_peers.cc:597] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.725317 17742 consensus_peers.cc:597] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.726663 17742 consensus_peers.cc:597] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.727568 17742 consensus_peers.cc:597] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.728252 17742 consensus_peers.cc:597] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.731711 17742 consensus_peers.cc:597] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.759114 17816 consensus_peers.cc:597] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.764117 17890 consensus_peers.cc:597] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.784754  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:26.786410  6451 tablet_replica.cc:333] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.787945  6451 raft_consensus.cc:2243] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.789430  6451 raft_consensus.cc:2272] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:26.793401 17890 consensus_peers.cc:597] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.794131  6451 tablet_replica.cc:333] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.795127  6451 raft_consensus.cc:2243] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.796486  6451 raft_consensus.cc:2272] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.799678  6451 tablet_replica.cc:333] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.800642  6451 raft_consensus.cc:2243] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.801380  6451 raft_consensus.cc:2272] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.804955  6451 tablet_replica.cc:333] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.805941  6451 raft_consensus.cc:2243] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.806603  6451 raft_consensus.cc:2272] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.809572  6451 tablet_replica.cc:333] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.810546  6451 raft_consensus.cc:2243] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.811306  6451 raft_consensus.cc:2272] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.814553  6451 tablet_replica.cc:333] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.815399  6451 raft_consensus.cc:2243] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.816059  6451 raft_consensus.cc:2272] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.818980  6451 tablet_replica.cc:333] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.820369  6451 raft_consensus.cc:2243] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.822070  6451 raft_consensus.cc:2272] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.825246  6451 tablet_replica.cc:333] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.826414  6451 raft_consensus.cc:2243] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.828212  6451 raft_consensus.cc:2272] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.832569  6451 tablet_replica.cc:333] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.833822  6451 raft_consensus.cc:2243] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.835887  6451 raft_consensus.cc:2272] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.840268  6451 tablet_replica.cc:333] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.841550  6451 raft_consensus.cc:2243] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.842424  6451 raft_consensus.cc:2272] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.847846  6451 tablet_replica.cc:333] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.849213  6451 raft_consensus.cc:2243] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.851106  6451 raft_consensus.cc:2272] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.855283  6451 tablet_replica.cc:333] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.856566  6451 raft_consensus.cc:2243] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.857431  6451 raft_consensus.cc:2272] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:26.857514 17742 consensus_peers.cc:597] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.862195  6451 tablet_replica.cc:333] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.863690  6451 raft_consensus.cc:2243] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.864886  6451 raft_consensus.cc:2272] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.869148  6451 tablet_replica.cc:333] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.870046  6451 raft_consensus.cc:2243] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.870746  6451 raft_consensus.cc:2272] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.874791  6451 tablet_replica.cc:333] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.875864  6451 raft_consensus.cc:2243] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.876981  6451 raft_consensus.cc:2272] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.881417  6451 tablet_replica.cc:333] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.882637  6451 raft_consensus.cc:2243] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.883728  6451 raft_consensus.cc:2272] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.888036  6451 tablet_replica.cc:333] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.889274  6451 raft_consensus.cc:2243] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.890172  6451 raft_consensus.cc:2272] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.894714  6451 tablet_replica.cc:333] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.896075  6451 raft_consensus.cc:2243] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.897675  6451 raft_consensus.cc:2272] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.902031  6451 tablet_replica.cc:333] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.903524  6451 raft_consensus.cc:2243] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.904639  6451 raft_consensus.cc:2272] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.909042  6451 tablet_replica.cc:333] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.910154  6451 raft_consensus.cc:2243] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.911259  6451 raft_consensus.cc:2272] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.915787  6451 tablet_replica.cc:333] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.916926  6451 raft_consensus.cc:2243] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.917963  6451 raft_consensus.cc:2272] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.925240  6451 tablet_replica.cc:333] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.926718  6451 raft_consensus.cc:2243] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.927906  6451 raft_consensus.cc:2272] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.932626  6451 tablet_replica.cc:333] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.933880  6451 raft_consensus.cc:2243] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.935310  6451 raft_consensus.cc:2272] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.939754  6451 tablet_replica.cc:333] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.940800  6451 raft_consensus.cc:2243] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.942342  6451 raft_consensus.cc:2272] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.946471  6451 tablet_replica.cc:333] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.947576  6451 raft_consensus.cc:2243] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.948429  6451 raft_consensus.cc:2272] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.952594  6451 tablet_replica.cc:333] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.953706  6451 raft_consensus.cc:2243] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.954479  6451 raft_consensus.cc:2272] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.957839  6451 tablet_replica.cc:333] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.958725  6451 raft_consensus.cc:2243] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.959419  6451 raft_consensus.cc:2272] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.963950  6451 tablet_replica.cc:333] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.964883  6451 raft_consensus.cc:2243] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.965740  6451 raft_consensus.cc:2272] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.969563  6451 tablet_replica.cc:333] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.970523  6451 raft_consensus.cc:2243] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.971393  6451 raft_consensus.cc:2272] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.974855  6451 tablet_replica.cc:333] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.975914  6451 raft_consensus.cc:2243] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.977553  6451 raft_consensus.cc:2272] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.981091  6451 tablet_replica.cc:333] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.982187  6451 raft_consensus.cc:2243] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:26.983078  6451 raft_consensus.cc:2272] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.986989  6451 tablet_replica.cc:333] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.988114  6451 raft_consensus.cc:2243] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.989560  6451 raft_consensus.cc:2272] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:26.993705  6451 tablet_replica.cc:333] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:26.994814  6451 raft_consensus.cc:2243] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:26.996397  6451 raft_consensus.cc:2272] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.000131  6451 tablet_replica.cc:333] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.001300  6451 raft_consensus.cc:2243] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:27.002759  6451 raft_consensus.cc:2272] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.006837  6451 tablet_replica.cc:333] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.007813  6451 raft_consensus.cc:2243] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.008697  6451 raft_consensus.cc:2272] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.011932  6451 tablet_replica.cc:333] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.012639  6451 raft_consensus.cc:2243] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:27.013718  6451 raft_consensus.cc:2272] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.016742  6451 tablet_replica.cc:333] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.017422  6451 raft_consensus.cc:2243] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.017943  6451 raft_consensus.cc:2272] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.020653  6451 tablet_replica.cc:333] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.021548  6451 raft_consensus.cc:2243] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:27.022760  6451 raft_consensus.cc:2272] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.026190  6451 tablet_replica.cc:333] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.026831  6451 raft_consensus.cc:2243] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.027525  6451 raft_consensus.cc:2272] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.030367  6451 tablet_replica.cc:333] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.031306  6451 raft_consensus.cc:2243] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.032148  6451 raft_consensus.cc:2272] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.035950  6451 tablet_replica.cc:333] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.036917  6451 raft_consensus.cc:2243] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.037770  6451 raft_consensus.cc:2272] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.041826  6451 tablet_replica.cc:333] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.042826  6451 raft_consensus.cc:2243] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.043694  6451 raft_consensus.cc:2272] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.047366  6451 tablet_replica.cc:333] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.048223  6451 raft_consensus.cc:2243] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:27.049702  6451 raft_consensus.cc:2272] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.053362  6451 tablet_replica.cc:333] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.054312  6451 raft_consensus.cc:2243] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:27.055244  6451 raft_consensus.cc:2272] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:27.059528  6451 tablet_replica.cc:333] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: stopping tablet replica
I20260709 12:30:27.060463  6451 raft_consensus.cc:2243] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:27.061981  6451 raft_consensus.cc:2272] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:27.181689 17890 connection.cc:573] client connection to 127.6.76.193:37959 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:30:27.187580 17816 connection.cc:573] client connection to 127.6.76.193:37959 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:30:27.224124  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
I20260709 12:30:27.355180  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:27.365900 18573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:27.367774 18572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:27.371031 18575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:27.377667  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:27.385952  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:27.386392  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:27.386693  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600227386671 us; error 0 us; skew 500 ppm
I20260709 12:30:27.387524  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:27.391494  6451 webserver.cc:533] Webserver started at http://127.6.76.193:37929/ using document root <none> and password file <none>
I20260709 12:30:27.392514  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:27.392879  6451 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:30:27.410699  6451 fs_manager.cc:714] Time spent opening directory manager: real 0.015s	user 0.005s	sys 0.008s
I20260709 12:30:27.425061 18580 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:27.426365  6451 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:27.426893  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-n447"
I20260709 12:30:27.427647  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:27.454490  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:27.456152  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:27.458300  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:27.465375 18587 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:30:27.842345  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:30:27.842721  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.379s	user 0.010s	sys 0.001s
I20260709 12:30:27.843109  6451 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:30:27.852909 18587 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:27.863281 18565 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:27.864039 18565 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:27.867805 18565 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:27.869830 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:27.874842 17890 leader_election.cc:336] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:27.874917 18587 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.875464 17890 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:27.876331 18587 tablet_bootstrap.cc:492] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:27.876288 18565 raft_consensus.cc:2749] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:27.877499 18587 ts_tablet_manager.cc:1403] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 12:30:27.881023 18587 raft_consensus.cc:359] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.881877 18587 raft_consensus.cc:740] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.882730 18587 consensus_queue.cc:260] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.884193 18565 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:27.884909 18565 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:27.889568 18565 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:27.890728 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:27.909437 18587 ts_tablet_manager.cc:1434] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.020s
I20260709 12:30:27.911270 18587 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:27.928021 18563 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:27.928843 18563 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:27.932693 17890 leader_election.cc:336] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:27.934675 17890 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:27.938957 18563 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:27.939888 18565 raft_consensus.cc:2749] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:27.946556 17816 leader_election.cc:336] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:27.948495 18587 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.949664 18587 tablet_bootstrap.cc:492] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:27.950511 18587 ts_tablet_manager.cc:1403] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.040s	user 0.026s	sys 0.012s
I20260709 12:30:27.953660 18587 raft_consensus.cc:359] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.954515 18587 raft_consensus.cc:740] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.955386 18587 consensus_queue.cc:260] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.957772 18587 ts_tablet_manager.cc:1434] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:30:27.958719 18587 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:27.960820 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:27.965301 17815 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:27.967608 18558 raft_consensus.cc:2749] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:27.977830 18563 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:27.978629 18563 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.981140 18563 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:27.985698 18561 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:27.986948 18561 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:27.990283 18561 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:27.992832 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:28.002471 17816 leader_election.cc:336] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.015610 17815 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.016237 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.017083 17845 raft_consensus.cc:2468] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.021292 18561 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.022302 18561 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.017202 18563 raft_consensus.cc:2749] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.026428 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.034437 18587 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:30:28.057440 17742 leader_election.cc:336] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.026585 18561 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.062793 17923 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.063513 18587 tablet_bootstrap.cc:492] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:28.064497 18587 ts_tablet_manager.cc:1403] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.106s	user 0.025s	sys 0.052s
I20260709 12:30:28.064844 17744 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.066008 18561 raft_consensus.cc:2804] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.066679 18561 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.067178 17742 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.067724 18587 raft_consensus.cc:359] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.067274 18561 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.068534 18587 raft_consensus.cc:740] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
W20260709 12:30:28.071686 17742 leader_election.cc:336] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.071956 18587 consensus_queue.cc:260] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.074697 18587 ts_tablet_manager.cc:1434] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:30:28.076056 18587 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:28.072510 18592 raft_consensus.cc:2804] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.079386 18592 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.079972 18592 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.096551 18592 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.105733 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2da24cf13b4824be213c5a60b39803" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:28.107609 17845 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.101092 18594 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.111874 18594 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.113471 18587 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:28.115195 18587 tablet_bootstrap.cc:492] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:28.116123 18587 ts_tablet_manager.cc:1403] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.015s
I20260709 12:30:28.121105 18594 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.120577 18587 raft_consensus.cc:359] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.127512 18587 raft_consensus.cc:740] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:28.127738 18593 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.128392 18593 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.128789 18587 consensus_queue.cc:260] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.131637 18587 ts_tablet_manager.cc:1434] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.002s
I20260709 12:30:28.132627 18587 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:28.100708 18592 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.137624 18593 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.139339 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.140268 17845 raft_consensus.cc:2468] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.142210 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:28.147141 17742 leader_election.cc:336] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.156531 17742 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.157876 18532 raft_consensus.cc:2804] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.158496 18532 raft_consensus.cc:697] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:28.159904 18532 consensus_queue.cc:237] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.161927 18594 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.162626 18594 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.165503 18594 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.168286 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:28.188377 18593 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.189026 18593 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.191730 18593 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.194072 18592 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.194906 18595 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.195238 18592 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.195700 18595 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.198122 18598 raft_consensus.cc:493] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.199564 18598 raft_consensus.cc:515] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.203835 18594 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.204558 18594 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:28.206202 17890 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.207115 17890 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.207986 18594 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.208850 18565 raft_consensus.cc:2749] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.216259 18593 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.216969 18593 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.220005 18593 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.223563 18595 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.224341 18595 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.224864 18595 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.227496 18595 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:28.237695 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 45433.5us
I20260709 12:30:28.242121 18595 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.241993 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.243882 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:28.245857 18561 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.260018 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:28.260902 17774 raft_consensus.cc:2468] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:28.262439 17889 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.263475 18594 raft_consensus.cc:2804] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.263885 18594 raft_consensus.cc:493] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.264267 18594 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.257018 18595 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.266067 17775 raft_consensus.cc:2468] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:28.270426 18565 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.271359 18565 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.274569 18565 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.277508 18594 raft_consensus.cc:515] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.282501 18595 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.269579 18598 leader_election.cc:290] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.283591 18592 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.287374 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.287125 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.288558 17923 raft_consensus.cc:2393] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 2: Already voted for candidate 30d17e55b097491f895c84a3443cb250 in this term.
W20260709 12:30:28.289582 17890 leader_election.cc:336] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.290405 17890 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.292438 18565 raft_consensus.cc:2749] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.292539 18593 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.293453 18593 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.293870 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.294667 17919 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.297659 18593 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.298815 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.299932 17845 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
W20260709 12:30:28.300690 17890 leader_election.cc:336] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.303193 18594 leader_election.cc:290] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.303479 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c2238a9361414181c3aff69d2f9dbd" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:28.304307 17775 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:28.306021 17816 leader_election.cc:336] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.309155 17815 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
W20260709 12:30:28.313331 17742 leader_election.cc:336] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.313562 17890 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
W20260709 12:30:28.314276 17742 leader_election.cc:336] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.314975 17742 leader_election.cc:336] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.306634 17816 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.314783 18565 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.318241 18565 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.318660 18565 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.315496 17742 leader_election.cc:304] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.321897 18532 raft_consensus.cc:2749] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.323745 17598 catalog_manager.cc:5697] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:28.328729 17890 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.335947 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:28.336598 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.335883 18593 raft_consensus.cc:2804] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.337703 18593 raft_consensus.cc:493] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.338122 18593 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.336956 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:28.340992 17890 leader_election.cc:336] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.344349 18532 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.345010 18532 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.348935 18597 raft_consensus.cc:2749] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.349934 18595 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.345125 18565 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.350634 18595 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.352763 17744 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.367759 18595 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.366750 17744 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.366840 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:28.369761 17919 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.369872 17844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:28.373389 17844 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:28.350220 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 44224.3us
I20260709 12:30:28.376801 18592 raft_consensus.cc:2804] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.377328 18592 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.377700 18592 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.355736 17775 raft_consensus.cc:2468] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:28.356725 18561 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.380853 17889 leader_election.cc:304] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.351817 18532 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.364317 18587 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:28.381974 18598 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.382642 18598 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.383193 18587 tablet_bootstrap.cc:492] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:28.384171 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.384985 17926 raft_consensus.cc:2468] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
W20260709 12:30:28.343261 17742 leader_election.cc:336] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.385083 18598 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:28.385777 17742 leader_election.cc:336] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.386312 17742 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:28.386375 17816 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.387784 18598 raft_consensus.cc:2749] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.387866 17816 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.388687 18593 raft_consensus.cc:515] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:28.389500 17816 leader_election.cc:336] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.390215 17845 raft_consensus.cc:2393] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 2: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:28.384135 18587 ts_tablet_manager.cc:1403] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.252s	user 0.059s	sys 0.085s
I20260709 12:30:28.394927 18597 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.395915 18595 raft_consensus.cc:2749] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.395790 18597 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.397368 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.398231 17847 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.398298 18597 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.385232 18594 raft_consensus.cc:2804] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.400673 18587 raft_consensus.cc:359] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.401237 18594 raft_consensus.cc:697] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:28.401535 18587 raft_consensus.cc:740] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
W20260709 12:30:28.405223 17890 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.405351 17744 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.402396 18587 consensus_queue.cc:260] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.408602 18532 raft_consensus.cc:2804] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.408766 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:28.409222 18532 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.409691 17926 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:28.409818 18532 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.411906 17818 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:28.412952 18597 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.412618 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.413770 18597 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.414757 18597 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.354817 18565 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.415922 17845 raft_consensus.cc:2468] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.420173 18558 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.424990 18558 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.426882 18587 ts_tablet_manager.cc:1434] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:30:28.428031 18558 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.402388 18594 consensus_queue.cc:237] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.433202  6451 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:30:28.433890  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.591s	user 0.384s	sys 0.008s
I20260709 12:30:28.428045 18587 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:28.428371 18592 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:28.420943 17816 leader_election.cc:336] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.447880 17816 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.409263 17742 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.449474 18592 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.421048 18593 leader_election.cc:290] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.451457 18593 raft_consensus.cc:2749] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.453105 18564 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.453976 18564 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.456770 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f142d3ffe24e67b967c51897ba2e82" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:28.457793 17926 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.460043 18564 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.430688 17844 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:28.463201 18563 raft_consensus.cc:2749] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.466156 18532 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.467634 18597 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.394307 17919 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.469367 18532 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.472172 17844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:28.473174 17844 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:28.450294 18592 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.474918 18597 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.478231 17744 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.450141 17742 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.479913 17918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85189191344a46deb54f096d7ac2c06d" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:28.480187 18595 raft_consensus.cc:2804] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.484018 18601 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.484540 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:28.480847 18595 raft_consensus.cc:697] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:28.486335 17919 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.485972 18601 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.481101 18598 raft_consensus.cc:2804] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.480777 17918 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.486934 18595 consensus_queue.cc:237] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.495409 18598 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.496397 18598 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.490828 18565 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.497874 18565 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.489537 18601 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:28.500308 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 50258us
I20260709 12:30:28.501148 18565 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.502684 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:28.503300 17742 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.503693 17775 raft_consensus.cc:2468] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 1.
I20260709 12:30:28.469894 17926 raft_consensus.cc:2468] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.505605 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb98822af0c4e85aa127325141dce47" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:28.509074 18603 raft_consensus.cc:2804] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.509696 18603 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.510063 17890 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.511906 18564 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.510087 18603 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.512539 18564 raft_consensus.cc:697] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:28.513725 18564 consensus_queue.cc:237] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.516253 17889 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.516250 17890 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:28.530555 17744 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.483886 18593 raft_consensus.cc:493] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.539764 18532 raft_consensus.cc:2804] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.551944 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:28.568966 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 148060us
I20260709 12:30:28.558948 18532 raft_consensus.cc:697] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:28.549662 18598 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.551146 18593 raft_consensus.cc:515] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.581195 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:28.586333 17742 leader_election.cc:336] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.594723 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 78599.9us
I20260709 12:30:28.599610 17847 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 1.
I20260709 12:30:28.576748 17772 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:28.598843 18532 consensus_queue.cc:237] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.578084 18594 raft_consensus.cc:2804] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.603125 18598 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
W20260709 12:30:28.617424 17890 leader_election.cc:336] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.625399 17890 leader_election.cc:336] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.621382 18593 leader_election.cc:290] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.603376 18610 raft_consensus.cc:2804] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.616006 18594 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.638399 18610 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.646843 17918 raft_consensus.cc:2468] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.668780 17815 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.650327 18594 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.685712 17843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:28.686669 17843 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.689838 18587 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:28.690758 18592 raft_consensus.cc:2749] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:28.693607 18603 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.698716 18593 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.699330 18593 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.699791 18593 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.700932 18587 tablet_bootstrap.cc:492] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:28.702422 18587 ts_tablet_manager.cc:1403] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.275s	user 0.044s	sys 0.013s
W20260709 12:30:28.708184 18612 log.cc:927] Time spent T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:30:28.659483 18610 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.717952 17775 raft_consensus.cc:2468] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:28.718870 18603 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.720530 18587 raft_consensus.cc:359] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.721458 18587 raft_consensus.cc:740] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:28.724714 18587 consensus_queue.cc:260] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.736578 18587 ts_tablet_manager.cc:1434] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 12:30:28.742290 18587 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:28.750583 17919 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
W20260709 12:30:28.752081 17816 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.760447 17601 catalog_manager.cc:5697] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:28.763410 17816 leader_election.cc:336] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.764125 17816 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.765067 17816 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.761742 18610 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:28.779171 18610 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:28.781044 18593 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:28.781656 17890 consensus_peers.cc:597] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:28.787918 18594 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.789237 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:28.790170 17845 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.782665 17843 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.790884 18594 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:28.793577 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 131671us
I20260709 12:30:28.799063 17744 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.801761 18593 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:28.804575 18602 raft_consensus.cc:2804] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:30:28.824291 17816 leader_election.cc:336] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.804683 17845 raft_consensus.cc:2468] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:28.806752 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
W20260709 12:30:28.808099 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 136084us
I20260709 12:30:28.826656 17775 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.827077 17742 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.828219 18627 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.828744 18627 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.829202 18627 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:28.829090 17742 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:28.830408 18595 raft_consensus.cc:2804] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:30:28.830717 17742 leader_election.cc:336] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.832105 17742 leader_election.cc:336] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.831022 18595 raft_consensus.cc:697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:28.833050 18602 raft_consensus.cc:697] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:28.833377 18595 consensus_queue.cc:237] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.834259 18602 consensus_queue.cc:237] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.837870 17598 catalog_manager.cc:5697] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:28.839905 17742 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.840092 17598 catalog_manager.cc:5697] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:28.842857 17742 consensus_peers.cc:597] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:28.844055 17742 leader_election.cc:336] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.844837 17775 raft_consensus.cc:2468] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
W20260709 12:30:28.844933 17742 leader_election.cc:336] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.850595 17742 leader_election.cc:336] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.846361 17816 consensus_peers.cc:597] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:28.819336 17890 leader_election.cc:336] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:28.852511 17890 leader_election.cc:336] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.854612 17890 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.858060 18610 raft_consensus.cc:2804] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.859706 18610 raft_consensus.cc:697] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:28.861055 18610 consensus_queue.cc:237] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:28.852515 17816 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.896605 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:28.913781 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 61293.8us
W20260709 12:30:28.866487 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 207920us
I20260709 12:30:28.914285 17775 raft_consensus.cc:2468] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:28.883980 17815 leader_election.cc:304] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.881731 17843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff59477c0d846eaab2e41621e730061" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:28.915745 17843 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:28.906829 17742 leader_election.cc:336] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:28.905974 17889 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.905715 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:28.921561 18623 raft_consensus.cc:2804] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.922588 18616 raft_consensus.cc:2804] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:28.872999 17845 raft_consensus.cc:1275] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:28.923195 18616 raft_consensus.cc:697] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:28.924295 18616 consensus_queue.cc:237] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:28.930944 18623 raft_consensus.cc:697] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:28.903084 18627 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.934507 17773 raft_consensus.cc:2393] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 632ff234ebc74963b295e6b50ccb78e6 in current term 2: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:28.932113 18623 consensus_queue.cc:237] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:28.940250 18627 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:28.968381 18595 consensus_queue.cc:1048] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:28.976353 17890 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:28.977555 17815 leader_election.cc:304] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.979425 18563 raft_consensus.cc:2804] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:28.979995 18563 raft_consensus.cc:493] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:28.980273 17847 raft_consensus.cc:1275] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:28.980410 18563 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:28.984689 18624 log.cc:927] Time spent T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:30:28.986340 18610 consensus_queue.cc:1048] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:28.932767 17843 raft_consensus.cc:2468] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 2.
I20260709 12:30:28.990943 17815 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:28.992779 18623 raft_consensus.cc:2749] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:30:29.002038 17742 consensus_peers.cc:597] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:29.093492 18589 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:29.104449 17742 consensus_peers.cc:597] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.119009 18587 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:29.130613 17773 raft_consensus.cc:1275] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.132376 18589 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:29.172286 18564 consensus_queue.cc:1048] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:29.105106 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 102370us
W20260709 12:30:29.173745 17742 leader_election.cc:336] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:29.174485 17742 consensus_peers.cc:597] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.180184 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.189641 17890 consensus_peers.cc:597] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.178637 17601 catalog_manager.cc:5697] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:29.198359 17601 catalog_manager.cc:5697] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:29.201311 17890 consensus_peers.cc:597] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:29.213883 17818 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:29.111186 18563 raft_consensus.cc:515] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:29.220322 18587 tablet_bootstrap.cc:492] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:29.221264 18587 ts_tablet_manager.cc:1403] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.484s	user 0.030s	sys 0.011s
I20260709 12:30:29.219395 17742 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:29.223716 17816 consensus_peers.cc:597] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.242533 17742 consensus_peers.cc:597] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:29.243448 17742 consensus_peers.cc:597] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.244014 17742 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:29.231915 18645 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:29.247182 18603 raft_consensus.cc:2804] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:29.271274 17923 raft_consensus.cc:1275] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.257958 18589 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:29.281785 18603 raft_consensus.cc:697] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:29.285727 18603 consensus_queue.cc:237] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:29.289155 17890 consensus_peers.cc:597] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:29.290058 17890 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.324119 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 105601us
I20260709 12:30:29.326457 18587 raft_consensus.cc:359] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:29.345712 18587 raft_consensus.cc:740] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:29.346863 18587 consensus_queue.cc:260] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:29.282493 18595 consensus_queue.cc:1048] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
W20260709 12:30:29.352005 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 74004.2us
I20260709 12:30:29.364727 18587 ts_tablet_manager.cc:1434] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.143s	user 0.009s	sys 0.000s
I20260709 12:30:29.359248 17923 raft_consensus.cc:1275] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.325109 17843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:29.371433 18587 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:29.352854 18643 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:29.411976 17843 raft_consensus.cc:2393] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 753c9642e55f4b819d92519d568c2d94 in current term 2: Already voted for candidate 632ff234ebc74963b295e6b50ccb78e6 in this term.
I20260709 12:30:29.405781 18563 leader_election.cc:290] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:29.434355 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1607aea0063f40f0a8d248cf7860749d" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:29.440315 17778 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:29.448249 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 85386.1us
I20260709 12:30:29.406658 18643 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:29.503830 18647 log.cc:927] Time spent T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Append to log took a long time: real 0.233s	user 0.002s	sys 0.000s
I20260709 12:30:29.505584 17851 raft_consensus.cc:1275] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.508997 18662 consensus_queue.cc:1048] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:29.510555 17816 consensus_peers.cc:597] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.511186 17816 consensus_peers.cc:597] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.513764 17816 consensus_peers.cc:597] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.523790 17778 raft_consensus.cc:2468] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
W20260709 12:30:29.527652 17816 consensus_peers.cc:597] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.548848 17774 raft_consensus.cc:1275] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:29.551083 17816 consensus_peers.cc:597] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.555974 17816 leader_election.cc:336] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:29.558245 17816 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.561172 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 117679us
W20260709 12:30:29.577473 17816 consensus_peers.cc:597] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.581141 17816 consensus_peers.cc:597] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:29.611073 18645 raft_consensus.cc:697] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:29.610496 17598 catalog_manager.cc:5697] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:29.612733 18645 consensus_queue.cc:237] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:29.573154 18662 consensus_queue.cc:1048] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:30:29.586503 18592 consensus_queue.cc:1048] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:30:29.645274 18589 raft_consensus.cc:493] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:29.646034 18589 raft_consensus.cc:515] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:29.629518 17919 raft_consensus.cc:1275] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.649310 18643 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:29.706700 18589 leader_election.cc:290] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:29.726981 18589 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:29.727622 18589 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:29.733529 17890 consensus_peers.cc:597] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.734350 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 174969us
W20260709 12:30:29.735131 17890 consensus_peers.cc:597] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:29.736193 17890 consensus_peers.cc:597] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:29.739629 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:29.747241 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:29.749555 17775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
W20260709 12:30:29.757261 17742 consensus_peers.cc:597] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.730103 18589 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:29.769673 18643 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:29.770412 18643 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:29.773197 18643 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:29.770996 17600 catalog_manager.cc:5697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:29.778947 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:29.778388 17600 catalog_manager.cc:5697] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:29.819996 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 61804.6us
I20260709 12:30:29.830341 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:29.832376 17851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:29.838848 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:29.834966 17843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:29.841127 17843 raft_consensus.cc:2393] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b96ea338e2a940f9b606d5d0fdd10e85 in current term 2: Already voted for candidate 753c9642e55f4b819d92519d568c2d94 in this term.
I20260709 12:30:29.844273 18582 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:29.846624 18582 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:29.847453 18589 raft_consensus.cc:2749] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:29.865185 17815 leader_election.cc:304] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:29.906284 18602 consensus_queue.cc:1048] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:29.886404 18623 raft_consensus.cc:2804] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:29.890784 18589 raft_consensus.cc:2749] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:29.915652 17742 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:29.919401 17776 raft_consensus.cc:1275] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.920828 18623 raft_consensus.cc:697] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:29.921829 17847 raft_consensus.cc:1275] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:29.922130 18623 consensus_queue.cc:237] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:29.916836 18582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 70315.7us
I20260709 12:30:29.927356 18582 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:29.928488 18589 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:29.928700 17816 consensus_peers.cc:597] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.939121 18587 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:29.939291 18603 raft_consensus.cc:2749] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:30:29.943316 18587 tablet_bootstrap.cc:492] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:29.946202 18587 ts_tablet_manager.cc:1403] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.577s	user 0.027s	sys 0.003s
I20260709 12:30:29.959512 18587 raft_consensus.cc:359] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:29.961484 18587 raft_consensus.cc:740] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:29.968907 18587 consensus_queue.cc:260] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:29.973624 18587 ts_tablet_manager.cc:1434] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
W20260709 12:30:29.973989 17742 consensus_peers.cc:597] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:29.987149 17742 consensus_peers.cc:597] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.987546 18587 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:29.987650 17742 consensus_peers.cc:597] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.978001 18601 consensus_queue.cc:1048] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:29.977576 17601 catalog_manager.cc:5697] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:29.994422 17843 raft_consensus.cc:1275] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:29.982795 17816 consensus_peers.cc:597] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:29.985708 18589 raft_consensus.cc:2749] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:30.012952 17600 catalog_manager.cc:5697] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.071584 18623 consensus_queue.cc:1048] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:30.074057 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 158705us
W20260709 12:30:30.080160 17890 consensus_peers.cc:597] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:30.085637 18601 consensus_queue.cc:1048] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:30.113494 17601 catalog_manager.cc:5697] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:30.119637 17917 raft_consensus.cc:1275] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:30.111969 17600 catalog_manager.cc:5697] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 1 to 2, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:30.137501 17742 consensus_peers.cc:597] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:30.138429 17742 consensus_peers.cc:597] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:30.140816 18614 consensus_queue.cc:1048] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
W20260709 12:30:30.149025 17742 consensus_peers.cc:597] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:30.165105 17742 consensus_peers.cc:597] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:30.166280 17742 consensus_peers.cc:597] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:30.213714 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edef181ff5d249908318aaf871e8b47e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:30.220148 17918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:30.223369 18587 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:30.230459 18584 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:30.232179 18584 leader_election.cc:304] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:30.232452 18589 raft_consensus.cc:2749] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:30.233299 18589 raft_consensus.cc:2749] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:30.241003 17816 consensus_peers.cc:597] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:30.244850 18643 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.245596 18643 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:30.249114 18643 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:30.256847 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:30.283550 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:30.298396 17742 consensus_peers.cc:597] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:30.305689 17742 consensus_peers.cc:597] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:30.306430 17742 consensus_peers.cc:597] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:30.315042 18587 tablet_bootstrap.cc:492] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:30.316085 18587 ts_tablet_manager.cc:1403] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.329s	user 0.029s	sys 0.045s
I20260709 12:30:30.333261 18584 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:30.371680 18587 raft_consensus.cc:359] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:30.372658 18587 raft_consensus.cc:740] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
W20260709 12:30:30.378883 18584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 54191.1us
I20260709 12:30:30.379324 18643 raft_consensus.cc:2749] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:30.391726 18587 consensus_queue.cc:260] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:30.396914 18587 ts_tablet_manager.cc:1434] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.080s	user 0.006s	sys 0.001s
I20260709 12:30:30.398365 18587 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:30.481931 18587 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:30.483752 17776 raft_consensus.cc:1275] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:30.484675 18587 tablet_bootstrap.cc:492] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:30.487188 18587 ts_tablet_manager.cc:1403] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.089s	user 0.024s	sys 0.012s
I20260709 12:30:30.491427 18587 raft_consensus.cc:359] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:30.496959 18587 raft_consensus.cc:740] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.500093 18587 consensus_queue.cc:260] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:30.513545 18587 ts_tablet_manager.cc:1434] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.003s
I20260709 12:30:30.514612 18587 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:30.553591 18645 consensus_queue.cc:1048] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:30.569912 17816 consensus_peers.cc:597] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:30.700523 18587 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:30.702189 18587 tablet_bootstrap.cc:492] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:30.704208 18587 ts_tablet_manager.cc:1403] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.190s	user 0.047s	sys 0.052s
I20260709 12:30:30.713555 18587 raft_consensus.cc:359] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:30.716285 18587 raft_consensus.cc:740] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.719740 18587 consensus_queue.cc:260] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:30.731417 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 47620.9us
I20260709 12:30:30.749586 18587 ts_tablet_manager.cc:1434] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 12:30:30.755707 18587 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:30.864611 18587 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:30.866546 18587 tablet_bootstrap.cc:492] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:30.867690 18587 ts_tablet_manager.cc:1403] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.116s	user 0.015s	sys 0.013s
I20260709 12:30:30.873075 18587 raft_consensus.cc:359] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:30.873136 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 69051.3us
I20260709 12:30:30.891717 18587 raft_consensus.cc:740] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.895409 18587 consensus_queue.cc:260] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:30.904003 18587 ts_tablet_manager.cc:1434] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:30:30.912897 18587 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:30.930361 18651 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:30.931109 17742 consensus_peers.cc:597] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:30.931185 18651 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:30.943207 18651 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:30.949591 17816 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:30.972174 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:30.973230 17773 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:30.981132 17815 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:30.982060 18623 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:30.982530 18623 raft_consensus.cc:493] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.982909 18623 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:31.008574 18587 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.009843 18587 tablet_bootstrap.cc:492] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.010684 18587 ts_tablet_manager.cc:1403] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.098s	user 0.019s	sys 0.001s
I20260709 12:30:31.017220 18623 raft_consensus.cc:515] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:31.028497 17816 leader_election.cc:336] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:31.029192 18587 raft_consensus.cc:359] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.030097 18587 raft_consensus.cc:740] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.040549 18587 consensus_queue.cc:260] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.044937 18587 ts_tablet_manager.cc:1434] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 12:30:31.047326 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cd8f570354b1ea762c251df7bbc19" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:31.048135 17776 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:31.051286 18587 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.058046 17776 raft_consensus.cc:2468] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:31.062104 17815 leader_election.cc:304] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:31.065125 18617 raft_consensus.cc:2804] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:31.065834 18617 raft_consensus.cc:697] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:31.066915 18617 consensus_queue.cc:237] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.058924 18623 leader_election.cc:290] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Requested vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:31.115899 17601 catalog_manager.cc:5697] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 3, leader changed from b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:31.134568 18587 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.135896 18587 tablet_bootstrap.cc:492] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.136724 18587 ts_tablet_manager.cc:1403] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.091s	user 0.022s	sys 0.015s
I20260709 12:30:31.139801 18587 raft_consensus.cc:359] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.148146 18587 raft_consensus.cc:740] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.149366 18587 consensus_queue.cc:260] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.152082 18587 ts_tablet_manager.cc:1434] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:30:31.153507 18643 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.155754 18643 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.159974 18643 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:31.160250 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:31.160560 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
I20260709 12:30:31.161381 18587 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.168258 18582 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:31.169978 18643 raft_consensus.cc:2749] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:31.214234 17890 consensus_peers.cc:597] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.215505 18587 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.227838 18587 tablet_bootstrap.cc:492] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.228880 18587 ts_tablet_manager.cc:1403] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.004s
I20260709 12:30:31.233742 18587 raft_consensus.cc:359] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.234622 18587 raft_consensus.cc:740] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.235507 18587 consensus_queue.cc:260] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.250584 18587 ts_tablet_manager.cc:1434] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:30:31.256095 18587 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.264356  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:37959
I20260709 12:30:31.257011 18764 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:37959 every 8 connection(s)
I20260709 12:30:31.309522 18587 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.310976 18587 tablet_bootstrap.cc:492] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.314635 18587 ts_tablet_manager.cc:1403] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.000s
I20260709 12:30:31.318046 18587 raft_consensus.cc:359] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.337093 18587 raft_consensus.cc:740] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.339056 18587 consensus_queue.cc:260] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.342921 18587 ts_tablet_manager.cc:1434] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:30:31.345366 18587 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.381145 18765 heartbeater.cc:344] Connected to a master server at 127.6.76.254:33695
I20260709 12:30:31.381737 18765 heartbeater.cc:461] Registering TS with master...
W20260709 12:30:31.382251 17742 consensus_peers.cc:597] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.382587 18765 heartbeater.cc:507] Master 127.6.76.254:33695 requested a full tablet report, sending...
I20260709 12:30:31.384675 18587 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.385903 18587 tablet_bootstrap.cc:492] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.386831 18587 ts_tablet_manager.cc:1403] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.008s
I20260709 12:30:31.390190 18587 raft_consensus.cc:359] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.391153 18587 raft_consensus.cc:740] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.392009 18587 consensus_queue.cc:260] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.394443 18587 ts_tablet_manager.cc:1434] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:31.397157 18587 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.412124 17601 ts_manager.cc:194] Re-registered known tserver with Master: b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:31.418607 18587 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.419921 18587 tablet_bootstrap.cc:492] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.420743 18587 ts_tablet_manager.cc:1403] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 12:30:31.424422 18587 raft_consensus.cc:359] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.425455 18587 raft_consensus.cc:740] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.426404 18587 consensus_queue.cc:260] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.428998 18587 ts_tablet_manager.cc:1434] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:31.430145 18587 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.434370 18724 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.518838 18778 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.519588 18778 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:31.522351 17816 consensus_peers.cc:597] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:31.523110 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753c9642e55f4b819d92519d568c2d94" is_pre_election: true
I20260709 12:30:31.530826 18778 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:31.535635 18587 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.537031 18587 tablet_bootstrap.cc:492] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.537895 18587 ts_tablet_manager.cc:1403] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.108s	user 0.044s	sys 0.034s
I20260709 12:30:31.541087 18587 raft_consensus.cc:359] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.543412 18587 raft_consensus.cc:740] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.547323 18587 consensus_queue.cc:260] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.554230 18727 raft_consensus.cc:3060] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.561576 18587 ts_tablet_manager.cc:1434] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.006s
W20260709 12:30:31.561865 17816 consensus_peers.cc:597] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.562938 18587 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.563737 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f292bf1109ec493d9111ad805d777330" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "632ff234ebc74963b295e6b50ccb78e6" is_pre_election: true
W20260709 12:30:31.565469 17816 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.567627 18582 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96ea338e2a940f9b606d5d0fdd10e85; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:31.568614 18778 raft_consensus.cc:2749] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:31.573988 18728 raft_consensus.cc:3060] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.587875 18726 raft_consensus.cc:3060] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.603745 18587 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.611294 18587 tablet_bootstrap.cc:492] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.612112 18587 ts_tablet_manager.cc:1403] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.008s
I20260709 12:30:31.615512 18587 raft_consensus.cc:359] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.616460 18587 raft_consensus.cc:740] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.617473 18587 consensus_queue.cc:260] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.619411 18587 ts_tablet_manager.cc:1434] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:31.620491 18587 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.633271 17601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34232
I20260709 12:30:31.646034 18765 heartbeater.cc:499] Master 127.6.76.254:33695 was elected leader, sending a full tablet report...
I20260709 12:30:31.658947 18587 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.660341 18587 tablet_bootstrap.cc:492] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.661217 18587 ts_tablet_manager.cc:1403] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.012s
I20260709 12:30:31.664402 18587 raft_consensus.cc:359] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.666778 18587 raft_consensus.cc:740] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
W20260709 12:30:31.672187 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.670225 18587 consensus_queue.cc:260] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:31.672833 17816 consensus_peers.cc:597] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.675444 18726 raft_consensus.cc:3060] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.675874 18587 ts_tablet_manager.cc:1434] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:30:31.679358 17776 raft_consensus.cc:1275] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:31.680511 18587 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.690373 18780 consensus_queue.cc:1048] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:31.727480 18587 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.728678 18587 tablet_bootstrap.cc:492] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.729446 18587 ts_tablet_manager.cc:1403] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.000s
W20260709 12:30:31.736814 17890 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.737867 18587 raft_consensus.cc:359] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.738759 18587 raft_consensus.cc:740] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.739743 18587 consensus_queue.cc:260] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:31.745904 18587 ts_tablet_manager.cc:1434] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:30:31.748535 18587 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:31.761279 17742 consensus_peers.cc:597] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:31.769183 18587 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.770625 18587 tablet_bootstrap.cc:492] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.771566 18587 ts_tablet_manager.cc:1403] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.008s
I20260709 12:30:31.774600 18587 raft_consensus.cc:359] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.775581 18587 raft_consensus.cc:740] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.776306 18587 consensus_queue.cc:260] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.778185 18587 ts_tablet_manager.cc:1434] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:31.779129 18587 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.801613 18587 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.803069 18587 tablet_bootstrap.cc:492] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
W20260709 12:30:31.803686 17816 consensus_peers.cc:597] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.803872 18587 ts_tablet_manager.cc:1403] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:30:31.807060 18587 raft_consensus.cc:359] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.807940 18587 raft_consensus.cc:740] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.808735 18587 consensus_queue.cc:260] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:31.811800 17816 consensus_peers.cc:597] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.812405 18587 ts_tablet_manager.cc:1434] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:31.813846 18587 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.836103 18587 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.837195 18587 tablet_bootstrap.cc:492] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.838107 18587 ts_tablet_manager.cc:1403] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.000s
I20260709 12:30:31.840986 18587 raft_consensus.cc:359] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.841703 18587 raft_consensus.cc:740] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.842736 18587 consensus_queue.cc:260] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.844952 18587 ts_tablet_manager.cc:1434] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:31.846678 18587 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.863502 18587 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.866283 18587 tablet_bootstrap.cc:492] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.871075 18587 ts_tablet_manager.cc:1403] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:30:31.874125 18587 raft_consensus.cc:359] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.874935 18587 raft_consensus.cc:740] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.875849 18587 consensus_queue.cc:260] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:31.878587 18587 ts_tablet_manager.cc:1434] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 12:30:31.880870 18587 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.905640 18587 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.907922 18587 tablet_bootstrap.cc:492] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.908880 18587 ts_tablet_manager.cc:1403] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.001s
I20260709 12:30:31.911501 18587 raft_consensus.cc:359] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.912446 18587 raft_consensus.cc:740] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.913355 18587 consensus_queue.cc:260] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.915673 18587 ts_tablet_manager.cc:1434] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:30:31.917306 18587 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:31.927385 18726 raft_consensus.cc:3060] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.962060 18587 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:31.963330 18587 tablet_bootstrap.cc:492] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:31.964143 18587 ts_tablet_manager.cc:1403] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.047s	user 0.029s	sys 0.012s
I20260709 12:30:31.967322 18587 raft_consensus.cc:359] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:31.968132 18587 raft_consensus.cc:740] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:31.969246 18587 consensus_queue.cc:260] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:31.969842 17742 consensus_peers.cc:597] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:31.971899 18587 ts_tablet_manager.cc:1434] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:31.972960 18587 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:31.993064 17890 consensus_peers.cc:597] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:32.016311 18587 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.017716 18587 tablet_bootstrap.cc:492] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.018637 18587 ts_tablet_manager.cc:1403] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.007s
I20260709 12:30:32.019064 18726 raft_consensus.cc:3060] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:30:32.022218 18587 raft_consensus.cc:359] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.023193 18587 raft_consensus.cc:740] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.024081 18587 consensus_queue.cc:260] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.026407 18587 ts_tablet_manager.cc:1434] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 12:30:32.027411 18587 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
W20260709 12:30:32.027683 17816 consensus_peers.cc:597] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:30:32.097546 17816 consensus_peers.cc:597] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:32.103910 18587 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.105705 18587 tablet_bootstrap.cc:492] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.106552 18587 ts_tablet_manager.cc:1403] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.079s	user 0.022s	sys 0.008s
I20260709 12:30:32.110436 18587 raft_consensus.cc:359] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:32.111179 17890 consensus_peers.cc:597] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:32.111297 18587 raft_consensus.cc:740] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.112773 18587 consensus_queue.cc:260] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.115484 18587 ts_tablet_manager.cc:1434] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 12:30:32.115578 17742 consensus_peers.cc:597] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:32.116585 18587 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.145262 18587 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.146395 18587 tablet_bootstrap.cc:492] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.147176 18587 ts_tablet_manager.cc:1403] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.001s
I20260709 12:30:32.149870 18587 raft_consensus.cc:359] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.152887 18587 raft_consensus.cc:740] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.153710 18587 consensus_queue.cc:260] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.157614 18587 ts_tablet_manager.cc:1434] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:30:32.158641 18587 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.179517 18587 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.180837 18587 tablet_bootstrap.cc:492] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.181633 18587 ts_tablet_manager.cc:1403] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 12:30:32.184478 18587 raft_consensus.cc:359] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.185245 18587 raft_consensus.cc:740] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.186003 18587 consensus_queue.cc:260] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.188427 18587 ts_tablet_manager.cc:1434] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:30:32.189369 18587 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.215207 18587 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.216354 18587 tablet_bootstrap.cc:492] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.217103 18587 ts_tablet_manager.cc:1403] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.000s
I20260709 12:30:32.219730 18587 raft_consensus.cc:359] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.220279 18587 raft_consensus.cc:740] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.220929 18587 consensus_queue.cc:260] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.222654 18587 ts_tablet_manager.cc:1434] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:32.224009 18587 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.250720 18587 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.251875 18587 tablet_bootstrap.cc:492] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.252687 18587 ts_tablet_manager.cc:1403] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.015s
I20260709 12:30:32.255524 18587 raft_consensus.cc:359] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.256325 18587 raft_consensus.cc:740] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.256405 18726 raft_consensus.cc:3060] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:32.257208 18587 consensus_queue.cc:260] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.260753 18587 ts_tablet_manager.cc:1434] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:32.262900 18587 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.300173 18587 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.301477 18587 tablet_bootstrap.cc:492] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.302379 18587 ts_tablet_manager.cc:1403] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.040s	user 0.027s	sys 0.004s
I20260709 12:30:32.305577 18587 raft_consensus.cc:359] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.306416 18587 raft_consensus.cc:740] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.307554 18587 consensus_queue.cc:260] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.310484 18587 ts_tablet_manager.cc:1434] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:30:32.313220 18587 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.324803 18726 raft_consensus.cc:3060] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:32.331350 18587 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.332630 18587 tablet_bootstrap.cc:492] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.333479 18587 ts_tablet_manager.cc:1403] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:30:32.336421 18587 raft_consensus.cc:359] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.337699 18587 raft_consensus.cc:740] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.338547 18587 consensus_queue.cc:260] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.342324 18587 ts_tablet_manager.cc:1434] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:30:32.343205 18587 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.363509 18726 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:32.382223 18587 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.383663 18587 tablet_bootstrap.cc:492] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.386415 18587 ts_tablet_manager.cc:1403] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.000s
I20260709 12:30:32.390483 18587 raft_consensus.cc:359] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.391161 18587 raft_consensus.cc:740] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.392023 18587 consensus_queue.cc:260] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.395154 18587 ts_tablet_manager.cc:1434] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:30:32.396246 18587 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.420317 18587 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.421337 18587 tablet_bootstrap.cc:492] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.422191 18587 ts_tablet_manager.cc:1403] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.000s
I20260709 12:30:32.424467 18587 raft_consensus.cc:359] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.425156 18587 raft_consensus.cc:740] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.425865 18587 consensus_queue.cc:260] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.427299 18587 ts_tablet_manager.cc:1434] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:32.428592 18587 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.459765 18587 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.461202 18587 tablet_bootstrap.cc:492] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.462183 18587 ts_tablet_manager.cc:1403] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.008s
I20260709 12:30:32.465854 18587 raft_consensus.cc:359] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.466676 18587 raft_consensus.cc:740] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.467573 18587 consensus_queue.cc:260] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:32.469988 18587 ts_tablet_manager.cc:1434] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:32.472070 18587 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.498725 18587 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.499933 18587 tablet_bootstrap.cc:492] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.500826 18587 ts_tablet_manager.cc:1403] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.006s
I20260709 12:30:32.504381 18587 raft_consensus.cc:359] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.506198 18587 raft_consensus.cc:740] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.506994 18587 consensus_queue.cc:260] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.509456 18587 ts_tablet_manager.cc:1434] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:30:32.510414 18587 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.529052 18587 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.530313 18587 tablet_bootstrap.cc:492] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.531242 18587 ts_tablet_manager.cc:1403] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 12:30:32.534503 18587 raft_consensus.cc:359] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.535326 18587 raft_consensus.cc:740] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.536099 18587 consensus_queue.cc:260] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.538604 18587 ts_tablet_manager.cc:1434] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:32.539748 18587 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.573130 18587 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.574705 18587 tablet_bootstrap.cc:492] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.576114 18587 ts_tablet_manager.cc:1403] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.009s
I20260709 12:30:32.594110 18587 raft_consensus.cc:359] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.595175 18587 raft_consensus.cc:740] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.596293 18587 consensus_queue.cc:260] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.599961 18587 ts_tablet_manager.cc:1434] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.005s
I20260709 12:30:32.601243 18587 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.626633 18587 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.627990 18587 tablet_bootstrap.cc:492] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.628800 18587 ts_tablet_manager.cc:1403] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.003s
I20260709 12:30:32.631910 18587 raft_consensus.cc:359] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.632859 18587 raft_consensus.cc:740] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.633839 18587 consensus_queue.cc:260] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:32.636873 18587 ts_tablet_manager.cc:1434] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:32.637884 18587 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.648412 18728 raft_consensus.cc:3060] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:32.668540 18587 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.669778 18587 tablet_bootstrap.cc:492] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.670625 18587 ts_tablet_manager.cc:1403] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.009s
I20260709 12:30:32.674572 18587 raft_consensus.cc:359] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.675922 18587 raft_consensus.cc:740] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.676769 18587 consensus_queue.cc:260] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.680507 18587 ts_tablet_manager.cc:1434] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.002s
I20260709 12:30:32.681675 18587 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap starting.
I20260709 12:30:32.705817 18587 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:32.706988 18587 tablet_bootstrap.cc:492] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Bootstrap complete.
I20260709 12:30:32.707789 18587 ts_tablet_manager.cc:1403] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 12:30:32.710757 18587 raft_consensus.cc:359] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.711596 18587 raft_consensus.cc:740] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Initialized, Role: FOLLOWER
I20260709 12:30:32.712419 18587 consensus_queue.cc:260] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:32.714632 18587 ts_tablet_manager.cc:1434] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:32.756356 18728 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:33.030581 18726 raft_consensus.cc:3060] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:36.487569 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 56106.9us
W20260709 12:30:39.793623 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 60506.6us
W20260709 12:30:40.725449 18761 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8040 after lost signal to thread 6454
W20260709 12:30:40.727030 18761 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3e80 after lost signal to thread 17635
W20260709 12:30:40.728637 18761 debug-util.cc:398] Leaking SignalData structure 0x7b08000afe00 after lost signal to thread 17802
W20260709 12:30:40.730420 18761 debug-util.cc:398] Leaking SignalData structure 0x7b080027cba0 after lost signal to thread 17876
W20260709 12:30:40.732641 18761 debug-util.cc:398] Leaking SignalData structure 0x7b08003468c0 after lost signal to thread 17950
W20260709 12:30:40.734644 18761 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2a00 after lost signal to thread 18764
I20260709 12:30:41.566463  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:41.669694 17847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "051771ffcb694816b9aff49434d710aa"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:41.670763 17847 raft_consensus.cc:606] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:41.693724 17774 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43b78182e2a746608309739058b74474"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:41.694440 17774 raft_consensus.cc:606] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:41.698181 17848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:41.698930 17848 raft_consensus.cc:606] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:41.703889 17848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73f4f834976044f1be00f6e31249489e"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:41.704619 17848 raft_consensus.cc:606] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:41.712385 17848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:41.714120 17848 raft_consensus.cc:606] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:41.717801 17847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:41.718521 17847 raft_consensus.cc:606] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:41.721212  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:30:41.723711 18858 raft_consensus.cc:993] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.724237  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:41.724864 18837 raft_consensus.cc:1081] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.725651  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:41.727156 18727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:41.727964 18727 raft_consensus.cc:493] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:41.728400 18727 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.751621 18727 raft_consensus.cc:515] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:41.754766 18727 leader_election.cc:290] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:41.756780 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:41.757818 17774 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.756783 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b12a0adfe741248970463752f3b30e" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:41.760195 17847 raft_consensus.cc:3055] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:41.760497 17847 raft_consensus.cc:740] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:41.761521 17847 consensus_queue.cc:260] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:41.774226 17847 raft_consensus.cc:3060] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.797775 17774 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:41.807631 18581 leader_election.cc:304] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:41.807734 17847 raft_consensus.cc:2468] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:41.812230 18866 raft_consensus.cc:2804] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:41.825181 18866 raft_consensus.cc:697] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:41.825604 18829 raft_consensus.cc:993] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.826277 18829 raft_consensus.cc:1081] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.826474 18866 consensus_queue.cc:237] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:41.835405 18728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "051771ffcb694816b9aff49434d710aa"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:41.838544 18728 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:41.854296 18829 raft_consensus.cc:993] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:41.855015 18858 raft_consensus.cc:1081] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:41.859925 17926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73f4f834976044f1be00f6e31249489e"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54184
I20260709 12:30:41.860611 17926 raft_consensus.cc:493] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:41.861056 17926 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.860785 17601 catalog_manager.cc:5697] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:41.849481 18728 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:41.912402 17926 raft_consensus.cc:515] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:41.916388 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:41.917543 17776 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.919100 17926 leader_election.cc:290] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:41.920758 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:41.922106 17847 raft_consensus.cc:3055] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:41.922550 17847 raft_consensus.cc:740] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:41.923522 17847 consensus_queue.cc:260] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:41.924587 17847 raft_consensus.cc:3060] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:41.939780 17847 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:41.941370 17890 leader_election.cc:304] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:41.942518 18845 raft_consensus.cc:2804] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:41.944459 18845 raft_consensus.cc:697] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:41.945374 18845 consensus_queue.cc:237] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:41.969220 18728 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:41.973104 18728 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:41.978662 18860 raft_consensus.cc:993] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.979323 18860 raft_consensus.cc:1081] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:41.981000 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:41.981899 17850 raft_consensus.cc:3055] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:41.982307 17850 raft_consensus.cc:740] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:41.983347 17850 consensus_queue.cc:260] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:41.985090 17850 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:41.987512 17776 raft_consensus.cc:2468] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:41.988471 17774 raft_consensus.cc:1240] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Rejecting Update request from peer 632ff234ebc74963b295e6b50ccb78e6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:41.991532 17601 catalog_manager.cc:5697] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:41.995018 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:41.995878 17926 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:42.005187 17850 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:42.006497 17926 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:42.018721 18724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43b78182e2a746608309739058b74474"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:42.019469 18724 raft_consensus.cc:493] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:42.019922 18724 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.006629 18582 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:42.028077 18866 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:42.028743 18866 raft_consensus.cc:697] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:42.029919 18866 consensus_queue.cc:237] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:42.092562 17601 catalog_manager.cc:5697] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 2 to 3, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:42.100185 17889 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 43877.5us
I20260709 12:30:42.112027 18841 raft_consensus.cc:993] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:42.112890 18837 raft_consensus.cc:1081] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:42.117486 17917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54184
I20260709 12:30:42.118171 17917 raft_consensus.cc:493] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:42.118552 17917 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.126900 18724 raft_consensus.cc:515] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:42.127872 17917 raft_consensus.cc:515] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:42.130280 17917 leader_election.cc:290] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:42.131645 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:42.131949 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:42.132594 17776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:42.137054 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b78182e2a746608309739058b74474" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:42.274250 17774 raft_consensus.cc:3055] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:42.274660 17774 raft_consensus.cc:740] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:42.275583 17774 consensus_queue.cc:260] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:42.276882 17774 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.187871 18724 leader_election.cc:290] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:42.284169 17776 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.266325 17850 raft_consensus.cc:3055] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:42.288103 17850 raft_consensus.cc:740] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:42.289228 17850 consensus_queue.cc:260] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:42.290328 18841 raft_consensus.cc:993] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:42.144021 17847 raft_consensus.cc:3060] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.290944 18841 raft_consensus.cc:1081] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:42.318377 17774 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:42.321090 17850 raft_consensus.cc:3060] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.354000 18728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:42.356307 18728 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:42.362869 18728 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.369318 17776 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:42.376392 17847 raft_consensus.cc:2468] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:42.381300 18582 leader_election.cc:304] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:42.387009 17850 raft_consensus.cc:2468] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:42.383262 18867 raft_consensus.cc:2804] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:42.400975 17890 leader_election.cc:304] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:42.422602 18889 raft_consensus.cc:2804] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:42.425988 17844 raft_consensus.cc:1275] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:42.438498 18889 raft_consensus.cc:697] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:42.445192 18866 consensus_queue.cc:1048] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:30:42.442889 18867 raft_consensus.cc:697] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
W20260709 12:30:42.470309 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 48745.1us
I20260709 12:30:42.476801 17844 raft_consensus.cc:1275] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:42.476641 18889 consensus_queue.cc:237] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:42.510140 18728 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:42.550953 18728 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:42.561560 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 60204us
I20260709 12:30:42.561724 18867 consensus_queue.cc:237] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:42.566111 17845 raft_consensus.cc:1275] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:42.572552 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:42.573340 17850 raft_consensus.cc:3055] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:42.573694 17850 raft_consensus.cc:740] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:42.580888 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:42.583554 17850 consensus_queue.cc:260] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:42.581941 17923 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.567317 18877 consensus_queue.cc:1048] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:30:42.607131 17850 raft_consensus.cc:3060] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:42.607539 18891 consensus_queue.cc:1048] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:30:42.654614 17850 raft_consensus.cc:2468] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:42.663067 17923 raft_consensus.cc:2468] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:42.673115 17774 raft_consensus.cc:1275] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:42.673591 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 244083us
I20260709 12:30:42.690061 18582 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:42.710177 17600 catalog_manager.cc:5697] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:42.730840 18882 consensus_queue.cc:1048] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:42.710310 18902 raft_consensus.cc:2804] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:42.737129 18902 raft_consensus.cc:697] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:42.737157 17774 raft_consensus.cc:1275] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:42.738423 18902 consensus_queue.cc:237] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:42.778061 17926 raft_consensus.cc:1275] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:42.814990 18903 consensus_queue.cc:1048] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:42.818022 18903 consensus_queue.cc:1048] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:42.866957  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:42.961208 17601 catalog_manager.cc:5697] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:42.972813 17601 catalog_manager.cc:5697] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:43.076057 17771 raft_consensus.cc:1275] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.077322 17847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:43.078243 17847 raft_consensus.cc:606] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:43.081485 18845 consensus_queue.cc:1048] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:43.089736 17847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:43.090583 17847 raft_consensus.cc:606] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:43.100095 17773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:43.100986 17773 raft_consensus.cc:606] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:43.112296 17773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fc8a907276843128987dab0a2f52235"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:43.116209 17773 raft_consensus.cc:606] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:43.115401 18883 raft_consensus.cc:993] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.118219 18883 raft_consensus.cc:1081] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.120852 18722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:43.121544 18722 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:43.121954 18722 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.134382 17771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:43.138010 17847 raft_consensus.cc:1275] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.142580 18886 consensus_queue.cc:1048] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:43.160719 18722 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.165081 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:43.165791 17845 raft_consensus.cc:3055] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:43.166148 17845 raft_consensus.cc:740] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:43.167049 17845 consensus_queue.cc:260] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.168383 17845 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.170601 17926 raft_consensus.cc:1275] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.172561 18903 consensus_queue.cc:1048] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:43.172854 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:43.173564 17926 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.184504 17771 raft_consensus.cc:606] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:43.185851 17772 raft_consensus.cc:1275] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.189999  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:30:43.191141 18902 consensus_queue.cc:1048] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:43.192107  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:43.193045  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:43.195926 17850 raft_consensus.cc:1275] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.198020 18867 consensus_queue.cc:1048] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:43.204288 17848 raft_consensus.cc:1275] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.207626 18867 consensus_queue.cc:1048] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:43.224872 18895 raft_consensus.cc:993] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:43.225715 18896 raft_consensus.cc:1081] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:43.235584 17919 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54200
I20260709 12:30:43.236359 17919 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:43.236861 17919 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.253755 17919 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:43.254760 17926 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.267676 17919 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:43.269160 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:43.271551 17773 raft_consensus.cc:3055] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:43.271922 17773 raft_consensus.cc:740] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:43.272842 17773 consensus_queue.cc:260] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:43.274137 17773 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.270437 18584 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:43.278107 18867 raft_consensus.cc:2804] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:43.284721 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:43.285640 17850 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.286865 18867 raft_consensus.cc:697] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:43.315733 18867 consensus_queue.cc:237] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.295451 17773 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:43.328612 17889 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:43.349856 18877 raft_consensus.cc:2804] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:43.350518 18877 raft_consensus.cc:697] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:43.353205 18877 consensus_queue.cc:237] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:43.360716 17600 catalog_manager.cc:5697] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:43.204964 17845 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.410548 18722 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:43.432122 17850 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:43.451463 18851 raft_consensus.cc:993] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.452204 18851 raft_consensus.cc:1081] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.507891 18728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fc8a907276843128987dab0a2f52235"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:43.508575 18728 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:43.508976 18728 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.544713 18728 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:43.552181 18883 raft_consensus.cc:993] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.564627 18883 raft_consensus.cc:1081] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.567001 18722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:43.567672 18722 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:43.568080 18722 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.570684 17600 catalog_manager.cc:5697] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:43.556334 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 46725.1us
I20260709 12:30:43.578065 18722 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.594141 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:43.611805 17845 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.583320 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc8a907276843128987dab0a2f52235" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:43.613204 17773 raft_consensus.cc:3055] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:43.613655 17773 raft_consensus.cc:740] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:43.614997 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:43.615818 17923 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.615324 17773 consensus_queue.cc:260] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:43.616907 17773 raft_consensus.cc:3060] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.595739 18728 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:43.620281 17844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:43.621061 17844 raft_consensus.cc:3055] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:43.621399 17844 raft_consensus.cc:740] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:43.622252 17844 consensus_queue.cc:260] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.624529 17844 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:43.647532 17923 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.648553 18722 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:43.663632 17845 raft_consensus.cc:2468] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.660352 18584 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:43.666818 18867 raft_consensus.cc:2804] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:43.667589 18867 raft_consensus.cc:697] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:43.668634 18867 consensus_queue.cc:237] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:43.651908 17844 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.682654 18582 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:43.683775 18867 raft_consensus.cc:2804] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:43.684643 18867 raft_consensus.cc:697] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:43.685730 18867 consensus_queue.cc:237] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:43.694059 18851 raft_consensus.cc:993] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.703437 18881 raft_consensus.cc:1081] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:43.717408 17601 catalog_manager.cc:5697] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:43.690436 17773 raft_consensus.cc:2468] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:43.731642 18722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:43.734463 18722 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:43.735100 18722 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:43.738667 17773 raft_consensus.cc:1275] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.741457 18877 consensus_queue.cc:1048] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:43.759600 18722 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:43.750625 17601 catalog_manager.cc:5697] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:43.772370 18722 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:43.833549 17844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
W20260709 12:30:43.833971 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 60123us
I20260709 12:30:43.834316 17844 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:30:43.837672 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 53472.3us
I20260709 12:30:43.840274 17847 raft_consensus.cc:1275] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:43.857892 17773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4872dc0c0f3c4bf69e5618fae1ac5f49" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:43.858719 17773 raft_consensus.cc:3055] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:43.859134 17773 raft_consensus.cc:740] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:43.863482 17773 consensus_queue.cc:260] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:43.865193 17773 raft_consensus.cc:3060] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:43.877800 18891 consensus_queue.cc:1048] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:43.900949 17773 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:43.908911 17844 raft_consensus.cc:2468] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:43.931885 18581 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:43.940236 18891 raft_consensus.cc:2804] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:43.940829 18891 raft_consensus.cc:697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:43.942423 18891 consensus_queue.cc:237] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:43.979964 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 64478.7us
I20260709 12:30:44.043108 18722 raft_consensus.cc:1240] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Rejecting Update request from peer 753c9642e55f4b819d92519d568c2d94 for earlier term 2. Current term is 3. Ops: []
W20260709 12:30:44.050087 18928 log.cc:927] Time spent T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 12:30:44.050446 17844 raft_consensus.cc:1275] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.121090 17922 raft_consensus.cc:1275] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.140748 17600 catalog_manager.cc:5697] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 2 to 3, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:44.143029 18902 consensus_queue.cc:1048] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:44.147855 18933 consensus_queue.cc:1048] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:44.194235 17845 raft_consensus.cc:1275] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.197525 18902 consensus_queue.cc:1048] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:44.229769 17922 raft_consensus.cc:1275] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.232067 18902 consensus_queue.cc:1048] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:44.258940 17847 raft_consensus.cc:1275] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.267419 17773 raft_consensus.cc:1275] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:44.267750 18935 consensus_queue.cc:1048] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:44.269193 18891 consensus_queue.cc:1048] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:44.410394 17845 raft_consensus.cc:1275] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:44.415130 18867 consensus_queue.cc:1048] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:30:44.429579 17773 raft_consensus.cc:1275] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:44.439316 18891 consensus_queue.cc:1048] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:30:44.440948  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:44.612651 17771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:44.613461 17771 raft_consensus.cc:606] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:44.644238 17922 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:44610
I20260709 12:30:44.645256 17922 raft_consensus.cc:606] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:44.649641 17771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:44.650460 17771 raft_consensus.cc:606] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:44.657807 17771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:44.658558 17771 raft_consensus.cc:606] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:44.662886 17771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:44.663731 17771 raft_consensus.cc:606] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:44.668051  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:30:44.670253  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:44.671331  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:44.674496 18881 raft_consensus.cc:993] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:44.675150 18881 raft_consensus.cc:1081] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:44.681849 18728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:44.682518 18728 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:44.682946 18728 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.693246 18728 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:44.699170 18728 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:44.701130 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:44.701895 17847 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.699124 17771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f591a6d82c496ab6a106341934b97d" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:44.704591 17771 raft_consensus.cc:3055] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:44.705462 17771 raft_consensus.cc:740] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:44.706355 17771 consensus_queue.cc:260] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:44.707504 17771 raft_consensus.cc:3060] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.726857 17771 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:44.728925 18581 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:44.730571 18891 raft_consensus.cc:2804] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:44.731132 18891 raft_consensus.cc:697] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:44.731796 17847 raft_consensus.cc:2468] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:44.732169 18891 consensus_queue.cc:237] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:44.750566 17598 catalog_manager.cc:5697] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:44.862718 18851 raft_consensus.cc:993] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:44.863346 18881 raft_consensus.cc:1081] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:44.865748 18722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:44.866458 18722 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:44.866852 18722 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.876569 18722 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:44.880313 18722 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:44.881480 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:44.881814 17771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0b2be2ba8b41579122c1d5d17a6b26" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:44.882254 17922 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.882416 17771 raft_consensus.cc:3055] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:44.882882 17771 raft_consensus.cc:740] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:44.883996 17771 consensus_queue.cc:260] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:44.886122 17771 raft_consensus.cc:3060] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.907542 17922 raft_consensus.cc:2468] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:44.908128 17771 raft_consensus.cc:2468] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:44.910924 18581 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:44.912565 18891 raft_consensus.cc:2804] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:44.913148 18891 raft_consensus.cc:697] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:44.914214 18891 consensus_queue.cc:237] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:44.928035 18844 raft_consensus.cc:993] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:44.928661 18851 raft_consensus.cc:1081] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:44.931278 17922 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54200
I20260709 12:30:44.931883 17922 raft_consensus.cc:493] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:44.932240 17922 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.941033 17922 raft_consensus.cc:515] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:44.943879 17922 leader_election.cc:290] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:44.944345 17771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:44.945185 17771 raft_consensus.cc:3055] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:44.945588 17771 raft_consensus.cc:740] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:44.946498 17771 consensus_queue.cc:260] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:44.947039 17845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:44.947790 17845 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.948225 17771 raft_consensus.cc:3060] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:44.952617 17598 catalog_manager.cc:5697] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:44.960415 17771 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:44.962186 17889 leader_election.cc:304] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:44.963327 18939 raft_consensus.cc:2804] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:44.963928 18939 raft_consensus.cc:697] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:44.965065 18939 consensus_queue.cc:237] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:44.962575 17845 raft_consensus.cc:2468] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:44.998133 17598 catalog_manager.cc:5697] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:45.154460 18845 raft_consensus.cc:993] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:45.155126 18939 raft_consensus.cc:1081] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:45.159636 18728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:30:45.160269 18728 raft_consensus.cc:493] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:45.160620 18728 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.171509 18728 raft_consensus.cc:515] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:45.175027 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:45.175805 17922 raft_consensus.cc:3055] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:45.176131 17922 raft_consensus.cc:740] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:45.176414 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2aef671e0349d6ab0d618271ae26a8" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:45.177050 17778 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.176934 17922 consensus_queue.cc:260] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:45.177505 18728 leader_election.cc:290] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:45.178220 17922 raft_consensus.cc:3060] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.185716 17922 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:45.185930 17778 raft_consensus.cc:2468] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:45.187076 18584 leader_election.cc:304] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:45.188290 18891 raft_consensus.cc:2804] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:45.191095 18891 raft_consensus.cc:697] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:45.192227 18891 consensus_queue.cc:237] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:45.194344 18851 raft_consensus.cc:993] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower 632ff234ebc74963b295e6b50ccb78e6 to start an election
I20260709 12:30:45.194976 18851 raft_consensus.cc:1081] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer 632ff234ebc74963b295e6b50ccb78e6 to start an election
I20260709 12:30:45.196986 17847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:59570
I20260709 12:30:45.197630 17847 raft_consensus.cc:493] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:45.198017 17847 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.217482 17845 raft_consensus.cc:1275] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.219892 18935 consensus_queue.cc:1048] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:45.217918 17847 raft_consensus.cc:515] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:45.230405 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:45.231431 17778 raft_consensus.cc:3055] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:45.232003 17778 raft_consensus.cc:740] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:45.233147 17847 leader_election.cc:290] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:45.233283 17778 consensus_queue.cc:260] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:45.235635 17778 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.260538 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc87abcb4d04cbb8196b5b8ecc06ec5" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:45.261300 17923 raft_consensus.cc:3060] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:45.269928 17923 raft_consensus.cc:2468] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:45.276429 17598 catalog_manager.cc:5697] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 30d17e55b097491f895c84a3443cb250 (127.6.76.196) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:45.277371 17818 leader_election.cc:304] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:45.290197 18937 raft_consensus.cc:2804] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:45.291184 18937 raft_consensus.cc:697] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:45.291558 17778 raft_consensus.cc:2468] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:45.292269 18937 consensus_queue.cc:237] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:45.401358 17778 raft_consensus.cc:1275] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.413020 17778 raft_consensus.cc:1275] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.413785 18966 consensus_queue.cc:1048] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:45.414866 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 98134.4us
I20260709 12:30:45.423627 18967 consensus_queue.cc:1048] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:45.461599 17845 raft_consensus.cc:1275] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.470983 17773 raft_consensus.cc:1275] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.469676 17598 catalog_manager.cc:5697] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:45.505138 18845 consensus_queue.cc:1048] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:45.578413 17889 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 96723.7us
I20260709 12:30:45.626657 17923 raft_consensus.cc:1275] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.628633 18966 consensus_queue.cc:1048] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:45.595268 18899 consensus_queue.cc:1048] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:30:45.642675 17772 raft_consensus.cc:1275] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.663722 18935 consensus_queue.cc:1048] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:45.682950 17922 raft_consensus.cc:1275] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.687568 18966 consensus_queue.cc:1048] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:45.822971 18581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 104753us
W20260709 12:30:45.846359 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 136726us
W20260709 12:30:45.849951 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 132736us
I20260709 12:30:45.854759  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:45.886363 17922 raft_consensus.cc:1275] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:45.906780 18985 log.cc:927] Time spent T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: Append to log took a long time: real 0.050s	user 0.000s	sys 0.002s
I20260709 12:30:45.907130 17771 raft_consensus.cc:1275] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:45.911798 18829 consensus_queue.cc:1048] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:45.917778 18937 consensus_queue.cc:1048] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:46.145980 17926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
mode: GRACEFUL
new_leader_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:44610
I20260709 12:30:46.146767 17926 raft_consensus.cc:606] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Received request to transfer leadership to TS 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:46.155139 17773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:46.155920 17773 raft_consensus.cc:606] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:46.161650 17772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:46.162503 17772 raft_consensus.cc:606] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:46.164747  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:30:46.176970  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:46.178004  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:46.343096 18881 raft_consensus.cc:993] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:46.343832 18851 raft_consensus.cc:1081] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:46.345604 17926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54200
I20260709 12:30:46.346346 17926 raft_consensus.cc:493] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:46.346747 17926 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.356309 17926 raft_consensus.cc:515] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:46.365746 17772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:46.366374 17926 leader_election.cc:290] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:46.366545 17772 raft_consensus.cc:3055] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:46.367151 17772 raft_consensus.cc:740] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:46.368178 17772 consensus_queue.cc:260] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:46.369818 17772 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.370558 17850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90406ef14e034b2daea6aaf73e737745" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:46.371491 17850 raft_consensus.cc:3060] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.381515 17850 raft_consensus.cc:2468] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:46.382145 17772 raft_consensus.cc:2468] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:46.383917 17889 leader_election.cc:304] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:46.385028 18971 raft_consensus.cc:2804] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:46.385622 18971 raft_consensus.cc:697] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:46.386651 18971 consensus_queue.cc:237] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:46.478765 17601 catalog_manager.cc:5697] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:46.633857 18975 raft_consensus.cc:993] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: : Instructing follower 632ff234ebc74963b295e6b50ccb78e6 to start an election
I20260709 12:30:46.661692 18899 raft_consensus.cc:1081] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Signalling peer 632ff234ebc74963b295e6b50ccb78e6 to start an election
W20260709 12:30:46.658631 18581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 118938us
W20260709 12:30:46.727595 17890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 90389.7us
W20260709 12:30:46.730424 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 45998.2us
I20260709 12:30:46.740281 18851 raft_consensus.cc:993] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:46.741251 18851 raft_consensus.cc:1081] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:46.786468 17771 raft_consensus.cc:1275] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:46.788060 17843 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:59566
I20260709 12:30:46.789193 17843 raft_consensus.cc:493] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:46.789732 17843 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.809810 18996 consensus_queue.cc:1048] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:46.817761 17850 raft_consensus.cc:1275] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:46.824548 17744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 70832.2us
I20260709 12:30:46.830180 17926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54200
I20260709 12:30:46.830994 17926 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:46.831470 17926 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.867385 17926 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:46.868345 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 136723us
I20260709 12:30:46.879489 17843 raft_consensus.cc:515] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:46.890458 18939 consensus_queue.cc:1048] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:46.894199 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:46.895955 17843 leader_election.cc:290] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:46.895090 17919 raft_consensus.cc:3055] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:46.897035 17919 raft_consensus.cc:740] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:46.898257 17919 consensus_queue.cc:260] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:46.899809 17919 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.921108 17926 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:46.939486 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:46.940363 17847 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:46.964653 17771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:46.994762 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a8ca1913d34de8bdacce55ee0fa4da" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:47.012650 17919 raft_consensus.cc:2468] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:47.012619 17778 raft_consensus.cc:3060] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:47.016193 17926 raft_consensus.cc:1240] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Rejecting Update request from peer 753c9642e55f4b819d92519d568c2d94 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:47.021205 17771 raft_consensus.cc:3055] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:47.021875 17771 raft_consensus.cc:740] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:47.025720 17771 consensus_queue.cc:260] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
W20260709 12:30:47.042811 19000 consensus_queue.cc:1175] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "30d17e55b097491f895c84a3443cb250" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 753c9642e55f4b819d92519d568c2d94 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:30:47.062011 17818 leader_election.cc:304] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:47.063977 18837 raft_consensus.cc:2804] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:47.064566 18837 raft_consensus.cc:697] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:47.065726 18837 consensus_queue.cc:237] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.095168 17778 raft_consensus.cc:2468] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:47.095736 17847 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:47.107139 17771 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:47.105957 17890 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:47.117858 18939 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:47.118584 18939 raft_consensus.cc:697] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:47.120767 18939 consensus_queue.cc:237] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.131879 17771 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:47.178398 17601 catalog_manager.cc:5697] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 1 to 2, leader changed from 30d17e55b097491f895c84a3443cb250 (127.6.76.196) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.206696 17600 catalog_manager.cc:5697] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.240767  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:47.361397 17919 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
mode: GRACEFUL
new_leader_uuid: "753c9642e55f4b819d92519d568c2d94"
 from {username='slave'} at 127.0.0.1:44610
I20260709 12:30:47.362196 17919 raft_consensus.cc:606] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Received request to transfer leadership to TS 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:47.368099 17919 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:44610
I20260709 12:30:47.368978 17919 raft_consensus.cc:606] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:47.373082 17847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:47.374886 17847 raft_consensus.cc:606] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS 30d17e55b097491f895c84a3443cb250
I20260709 12:30:47.378152  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:30:47.380616  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:47.381947  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:47.395725 18939 raft_consensus.cc:993] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:47.396435 18939 raft_consensus.cc:1081] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:47.399758 18722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:30:47.400379 18722 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:47.400749 18722 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.411017 18722 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:47.414239 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:47.414985 17848 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.429620 17848 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:47.430953 18722 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:47.434043 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:47.434549 17847 raft_consensus.cc:1240] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Rejecting Update request from peer 30d17e55b097491f895c84a3443cb250 for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:47.434818 17919 raft_consensus.cc:3055] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:47.435055 18582 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:47.435406 17919 raft_consensus.cc:740] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:47.436911 17919 consensus_queue.cc:260] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:47.438412 17919 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.439757 18935 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:47.440740 18935 raft_consensus.cc:697] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:47.442020 18935 consensus_queue.cc:237] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:47.443647 18971 consensus_queue.cc:1175] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "632ff234ebc74963b295e6b50ccb78e6" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 30d17e55b097491f895c84a3443cb250 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:30:47.464718 17919 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:47.499629 17600 catalog_manager.cc:5697] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 2 to 3, leader changed from 30d17e55b097491f895c84a3443cb250 (127.6.76.196) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.506251 18829 raft_consensus.cc:993] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:47.507036 18829 raft_consensus.cc:1081] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer 30d17e55b097491f895c84a3443cb250 to start an election
I20260709 12:30:47.510340 17919 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
 from {username='slave'} at 127.0.0.1:54184
I20260709 12:30:47.511142 17919 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:47.511624 17919 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:47.550275 17778 raft_consensus.cc:1275] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:47.552423 18998 consensus_queue.cc:1048] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:47.538753 17919 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.558398 17778 raft_consensus.cc:1275] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:47.560957 17778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:47.561756 17778 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:47.562280 17847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "30d17e55b097491f895c84a3443cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:47.562901 17847 raft_consensus.cc:3055] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:47.563108 17847 raft_consensus.cc:740] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:47.560678 19009 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:30:47.564200 17847 consensus_queue.cc:260] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.571911 17778 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:47.581162 19009 raft_consensus.cc:993] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: : Instructing follower 753c9642e55f4b819d92519d568c2d94 to start an election
I20260709 12:30:47.581766 19009 raft_consensus.cc:1081] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Signalling peer 753c9642e55f4b819d92519d568c2d94 to start an election
I20260709 12:30:47.586859 17926 raft_consensus.cc:1275] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:47.589103 18998 consensus_queue.cc:1048] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:47.585762 17889 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:47.595466 19009 raft_consensus.cc:2804] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:47.596310 19009 raft_consensus.cc:697] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:47.597857 19009 consensus_queue.cc:237] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.605039 17848 raft_consensus.cc:1275] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:47.611480 17771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
 from {username='slave'} at 127.0.0.1:52392
I20260709 12:30:47.613971 17847 raft_consensus.cc:3060] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:47.561113 17919 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:47.624751 18939 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:47.671924 17771 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:47.672618 17771 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.677829 17847 raft_consensus.cc:2468] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d17e55b097491f895c84a3443cb250 in term 2.
I20260709 12:30:47.696770 17771 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.741568 17771 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 3 election: Requested vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:47.741439 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:47.742723 17919 raft_consensus.cc:3055] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:47.743410 17919 raft_consensus.cc:740] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:47.744520 17919 consensus_queue.cc:260] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.746184 17919 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.741444 17848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "753c9642e55f4b819d92519d568c2d94" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:47.750893 17848 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:47.755152 17919 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 3.
I20260709 12:30:47.754361 17600 catalog_manager.cc:5697] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to 30d17e55b097491f895c84a3443cb250 (127.6.76.196). New cstate: current_term: 2 leader_uuid: "30d17e55b097491f895c84a3443cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.756652 17744 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:47.766769 19022 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:47.762924 17848 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753c9642e55f4b819d92519d568c2d94 in term 3.
I20260709 12:30:47.788489 19022 raft_consensus.cc:697] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 3 LEADER]: Becoming Leader. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:47.789574 19022 consensus_queue.cc:237] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:47.808808 17600 catalog_manager.cc:5697] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 reported cstate change: term changed from 2 to 3, leader changed from 30d17e55b097491f895c84a3443cb250 (127.6.76.196) to 753c9642e55f4b819d92519d568c2d94 (127.6.76.194). New cstate: current_term: 3 leader_uuid: "753c9642e55f4b819d92519d568c2d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.854545 17919 raft_consensus.cc:1275] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:47.856400 19023 consensus_queue.cc:1048] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:47.879802 17848 raft_consensus.cc:1275] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:47.885627 19023 consensus_queue.cc:1048] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:47.989202 17771 raft_consensus.cc:1275] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:47.991415 18939 consensus_queue.cc:1048] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:48.014220 17847 raft_consensus.cc:1275] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer 30d17e55b097491f895c84a3443cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:48.016937 18971 consensus_queue.cc:1048] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:48.296312 17919 raft_consensus.cc:1275] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:48.297816 19022 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:48.312958 17848 raft_consensus.cc:1275] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Refusing update from remote peer 753c9642e55f4b819d92519d568c2d94: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:48.321694 19001 consensus_queue.cc:1048] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:30:48.384610 17742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:45767, user_credentials={real_user=slave}} blocked reactor thread for 63302.4us
I20260709 12:30:48.481161  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:48.559413 17774 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510"
dest_uuid: "753c9642e55f4b819d92519d568c2d94"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:39242
I20260709 12:30:48.560226 17774 raft_consensus.cc:606] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:48.563917 17917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f"
dest_uuid: "30d17e55b097491f895c84a3443cb250"
mode: GRACEFUL
new_leader_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:44610
I20260709 12:30:48.564635 17917 raft_consensus.cc:606] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Received request to transfer leadership to TS 632ff234ebc74963b295e6b50ccb78e6
I20260709 12:30:48.566947  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:48.569442  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:48.570467  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:48.645692 19031 raft_consensus.cc:993] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:48.646235 19001 raft_consensus.cc:1081] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:48.648228 18727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55800
I20260709 12:30:48.648697 18727 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:48.648938 18727 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.656247 18727 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:48.658710 18727 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:48.660116 17917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:48.661691 17917 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.660477 17774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173b1a41d3a1403e92f3345206f45510" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:48.664700 17774 raft_consensus.cc:3055] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:48.665114 17774 raft_consensus.cc:740] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 753c9642e55f4b819d92519d568c2d94, State: Running, Role: LEADER
I20260709 12:30:48.666666 17774 consensus_queue.cc:260] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:48.667873 17774 raft_consensus.cc:3060] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.672320 17917 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:48.673842 18584 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:48.674809 19033 raft_consensus.cc:2804] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:48.675585 19033 raft_consensus.cc:697] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:48.675717 17774 raft_consensus.cc:2468] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 3.
I20260709 12:30:48.676644 19033 consensus_queue.cc:237] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:48.690213 17601 catalog_manager.cc:5697] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 2 to 3, leader changed from 753c9642e55f4b819d92519d568c2d94 (127.6.76.194) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 3 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:48.842710 18971 raft_consensus.cc:993] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: : Instructing follower 632ff234ebc74963b295e6b50ccb78e6 to start an election
I20260709 12:30:48.843506 18996 raft_consensus.cc:1081] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Signalling peer 632ff234ebc74963b295e6b50ccb78e6 to start an election
I20260709 12:30:48.845598 17848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
 from {username='slave'} at 127.0.0.1:59566
I20260709 12:30:48.846217 17848 raft_consensus.cc:493] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:48.846625 17848 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.853360 17848 raft_consensus.cc:515] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:48.855253 17848 leader_election.cc:290] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:48.856468 17917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:48.856480 17771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb7403698d84ba789b17c0ef904859f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "753c9642e55f4b819d92519d568c2d94"
I20260709 12:30:48.857172 17917 raft_consensus.cc:3055] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:48.857518 17771 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.857627 17917 raft_consensus.cc:740] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d17e55b097491f895c84a3443cb250, State: Running, Role: LEADER
I20260709 12:30:48.858736 17917 consensus_queue.cc:260] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:48.860136 17917 raft_consensus.cc:3060] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:48.868109 17771 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:48.869837 17815 leader_election.cc:304] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94; no voters: 
I20260709 12:30:48.872452 18829 raft_consensus.cc:2804] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:48.873025 18829 raft_consensus.cc:697] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:48.873883 18829 consensus_queue.cc:237] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:48.877863 17917 raft_consensus.cc:2468] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:48.890823 17601 catalog_manager.cc:5697] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 reported cstate change: term changed from 2 to 3, leader changed from 30d17e55b097491f895c84a3443cb250 (127.6.76.196) to 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "632ff234ebc74963b295e6b50ccb78e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:49.262331 17774 raft_consensus.cc:1275] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:49.263984 19033 consensus_queue.cc:1048] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:49.327425 17917 raft_consensus.cc:1275] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:49.330658 19042 consensus_queue.cc:1048] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:49.333206 17919 raft_consensus.cc:1275] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:49.335326 18829 consensus_queue.cc:1048] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:49.358489 17774 raft_consensus.cc:1275] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Refusing update from remote peer 632ff234ebc74963b295e6b50ccb78e6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:49.366441 18998 consensus_queue.cc:1048] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:49.632335  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:49.690490 17843 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3"
dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
mode: GRACEFUL
new_leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:57308
I20260709 12:30:49.691324 17843 raft_consensus.cc:606] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Received request to transfer leadership to TS b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:49.693064  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:30:49.695305  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:49.696269  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:50.115116 19046 raft_consensus.cc:993] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: : Instructing follower b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:50.115803 19054 raft_consensus.cc:1081] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Signalling peer b96ea338e2a940f9b606d5d0fdd10e85 to start an election
I20260709 12:30:50.118093 18727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3"
dest_uuid: "b96ea338e2a940f9b606d5d0fdd10e85"
 from {username='slave'} at 127.0.0.1:55790
I20260709 12:30:50.118695 18727 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:50.119067 18727 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:50.125634 18727 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:50.127786 18727 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:50.128968 17843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "632ff234ebc74963b295e6b50ccb78e6"
I20260709 12:30:50.129371 17917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:50.129737 17843 raft_consensus.cc:3055] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:50.130059 17917 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:50.130084 17843 raft_consensus.cc:740] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:50.131283 17843 consensus_queue.cc:260] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:50.133276 17843 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:50.139787 17917 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:50.142041 18584 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85; no voters: 
I20260709 12:30:50.142930 18995 raft_consensus.cc:2804] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:50.144212 17843 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96ea338e2a940f9b606d5d0fdd10e85 in term 2.
I20260709 12:30:50.156939 18995 raft_consensus.cc:697] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Becoming Leader. State: Replica: b96ea338e2a940f9b606d5d0fdd10e85, State: Running, Role: LEADER
I20260709 12:30:50.158511 18995 consensus_queue.cc:237] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:50.174687 17601 catalog_manager.cc:5697] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 reported cstate change: term changed from 1 to 2, leader changed from 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195) to b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:50.670607 17917 raft_consensus.cc:1275] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:50.672215 19042 consensus_queue.cc:1048] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:50.689318 17848 raft_consensus.cc:1275] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Refusing update from remote peer b96ea338e2a940f9b606d5d0fdd10e85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:50.692375 19067 consensus_queue.cc:1048] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:50.740938  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:50.802122  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:50.803756  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:50.804589  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:50.848778  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:50.897454  6451 tablet_server.cc:179] TabletServer@127.6.76.193:37959 shutting down...
W20260709 12:30:50.906049 17742 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 372 similar messages]
W20260709 12:30:50.912350 17742 consensus_peers.cc:597] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:50.919479 17816 consensus_peers.cc:597] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:50.957765  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:50.959019  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.960391  6451 raft_consensus.cc:2243] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:50.961090  6451 raft_consensus.cc:2272] T d3be2bb16cb74f8fb343216495dc07a3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.966267  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.967852  6451 raft_consensus.cc:2243] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:50.968469  6451 raft_consensus.cc:2272] T 09de1ae34c854240a05f695427ac7c94 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.971808  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.972954  6451 raft_consensus.cc:2243] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:50.973631  6451 raft_consensus.cc:2272] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.977392  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.978488  6451 raft_consensus.cc:2243] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:50.979101  6451 raft_consensus.cc:2272] T ffeea622a1d642ffb92c2fe4b4dcaa57 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.982599  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.983790  6451 raft_consensus.cc:2243] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:50.985327  6451 raft_consensus.cc:2272] T 173b1a41d3a1403e92f3345206f45510 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:50.985774 17816 consensus_peers.cc:597] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:50.989070  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.990293  6451 raft_consensus.cc:2243] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:50.990918  6451 raft_consensus.cc:2272] T 59353fd7c83841ea9e358df44da38a6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:50.993767 17742 consensus_peers.cc:597] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:50.995076  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:50.996471  6451 raft_consensus.cc:2243] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:50.997849  6451 raft_consensus.cc:2272] T 6d2aef671e0349d6ab0d618271ae26a8 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.002013  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.003159  6451 raft_consensus.cc:2243] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.004680  6451 raft_consensus.cc:2272] T 43b78182e2a746608309739058b74474 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.006430 17890 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.008965  6451 tablet_replica.cc:333] stopping tablet replica
W20260709 12:30:51.009260 17816 consensus_peers.cc:597] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.010262  6451 raft_consensus.cc:2243] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.011026  6451 raft_consensus.cc:2272] T 1607aea0063f40f0a8d248cf7860749d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.015421  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.016597  6451 raft_consensus.cc:2243] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.017329  6451 raft_consensus.cc:2272] T 85189191344a46deb54f096d7ac2c06d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.021152  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.022258  6451 raft_consensus.cc:2243] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.022912  6451 raft_consensus.cc:2272] T 952cd8f570354b1ea762c251df7bbc19 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.026854  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.027971  6451 raft_consensus.cc:2243] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:30:51.028108 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.029316  6451 raft_consensus.cc:2272] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.034075  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.035064  6451 raft_consensus.cc:2243] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.035754  6451 raft_consensus.cc:2272] T 21c2238a9361414181c3aff69d2f9dbd P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.038897  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.039685  6451 raft_consensus.cc:2243] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.040227  6451 raft_consensus.cc:2272] T fbb98822af0c4e85aa127325141dce47 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.042863  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.043668  6451 raft_consensus.cc:2243] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.044157  6451 raft_consensus.cc:2272] T d8101eccca0c484c9d5b29aaff2497db P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.047344  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.048276  6451 raft_consensus.cc:2243] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.049043  6451 raft_consensus.cc:2272] T ffede44de4554930b68441a774c0dd06 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.052466  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.053184  6451 raft_consensus.cc:2243] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.054175  6451 raft_consensus.cc:2272] T 2933687c848f48e2b01b32ede660a4c9 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.058013  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.059008  6451 raft_consensus.cc:2243] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.060284  6451 raft_consensus.cc:2272] T c953b9330799457cb8d5c4ea252f5b30 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.060375 17742 consensus_peers.cc:597] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.064169  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.065140  6451 raft_consensus.cc:2243] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.066390  6451 raft_consensus.cc:2272] T 16e7941692924de28d3fb53cf3cd833a P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.069537  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.070358  6451 raft_consensus.cc:2243] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.071705  6451 raft_consensus.cc:2272] T d7b12a0adfe741248970463752f3b30e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.074905  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.076108  6451 raft_consensus.cc:2243] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.076725  6451 raft_consensus.cc:2272] T 2f8bdb4d10d0433c909031a9340f6c1b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.080289  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.081271  6451 raft_consensus.cc:2243] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.081939  6451 raft_consensus.cc:2272] T 20637ddae560402b81988ca96fc61b1e P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.085793  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.086745  6451 raft_consensus.cc:2243] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.087448  6451 raft_consensus.cc:2272] T ea1fbecc92da4549828813c3a7185b1a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.087018 17742 consensus_peers.cc:597] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.091660  6451 tablet_replica.cc:333] stopping tablet replica
W20260709 12:30:51.092599 17742 consensus_peers.cc:597] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.092866  6451 raft_consensus.cc:2243] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.093621  6451 raft_consensus.cc:2272] T 4b6bcf23aeb44b0b8aea5466500e7f8d P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.097045  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.097820  6451 raft_consensus.cc:2243] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.098330  6451 raft_consensus.cc:2272] T 2f87c2ee9429444babb65d1c8a1ec7c1 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.100688  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.101390  6451 raft_consensus.cc:2243] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.101858  6451 raft_consensus.cc:2272] T 6874bb3d07a84cc1b883375c3e83bcfe P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.104681  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.105415  6451 raft_consensus.cc:2243] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.106580  6451 raft_consensus.cc:2272] T 051771ffcb694816b9aff49434d710aa P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.110332  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.111161  6451 raft_consensus.cc:2243] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.112396  6451 raft_consensus.cc:2272] T 6e0b2be2ba8b41579122c1d5d17a6b26 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.115360  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.116470  6451 raft_consensus.cc:2243] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.117200  6451 raft_consensus.cc:2272] T f3120efae4a24a319065841f0dff065a P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.120844  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.121829  6451 raft_consensus.cc:2243] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.122421  6451 raft_consensus.cc:2272] T 4bda730805b64dda88c94c3072246211 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.123658 17816 consensus_peers.cc:597] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.123947 17742 consensus_peers.cc:597] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.125754  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.126731  6451 raft_consensus.cc:2243] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.127419  6451 raft_consensus.cc:2272] T f211414d5e584164a01d70059b214a4b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.128142 17816 consensus_peers.cc:597] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.131229  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.132277  6451 raft_consensus.cc:2243] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.132910  6451 raft_consensus.cc:2272] T edef181ff5d249908318aaf871e8b47e P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.136802  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.137756  6451 raft_consensus.cc:2243] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.138356  6451 raft_consensus.cc:2272] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.142120  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.143057  6451 raft_consensus.cc:2243] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.144375  6451 raft_consensus.cc:2272] T bb2695d8605c4716960fd53bdc4eb899 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.146448 17816 consensus_peers.cc:597] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.148037 17742 consensus_peers.cc:597] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.148092  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.149500  6451 raft_consensus.cc:2243] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.150352  6451 raft_consensus.cc:2272] T af2da24cf13b4824be213c5a60b39803 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.154357  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.155320  6451 raft_consensus.cc:2243] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.156029  6451 raft_consensus.cc:2272] T 0ff59477c0d846eaab2e41621e730061 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.159987  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.160955  6451 raft_consensus.cc:2243] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.161576  6451 raft_consensus.cc:2272] T f292bf1109ec493d9111ad805d777330 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.162249 17742 consensus_peers.cc:597] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.164443  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.165222  6451 raft_consensus.cc:2243] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.165674  6451 raft_consensus.cc:2272] T 5ae4ca22644a4128bc2662c6bb502884 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.168457  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.169411  6451 raft_consensus.cc:2243] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.170009  6451 raft_consensus.cc:2272] T 9a387a07df404a3a8cdec9c49a791e6f P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.173864  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.174736  6451 raft_consensus.cc:2243] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:30:51.174749 17890 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.175904 17890 consensus_peers.cc:597] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.176201  6451 raft_consensus.cc:2272] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P b96ea338e2a940f9b606d5d0fdd10e85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.179690  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.180635  6451 raft_consensus.cc:2243] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.181298  6451 raft_consensus.cc:2272] T 49f142d3ffe24e67b967c51897ba2e82 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.185210  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.185907  6451 raft_consensus.cc:2243] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.187077  6451 raft_consensus.cc:2272] T 6fc8a907276843128987dab0a2f52235 P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.187476 17890 consensus_peers.cc:597] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.190791  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.191682  6451 raft_consensus.cc:2243] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.192305  6451 raft_consensus.cc:2272] T 9ea37968d44c4ec5a4320301e4173d8b P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.195631  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.196293  6451 raft_consensus.cc:2243] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.196803  6451 raft_consensus.cc:2272] T 79b1348052e745648f1cba4c11a7e863 P b96ea338e2a940f9b606d5d0fdd10e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.199149  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:51.199772  6451 raft_consensus.cc:2243] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.200800  6451 raft_consensus.cc:2272] T 85f591a6d82c496ab6a106341934b97d P b96ea338e2a940f9b606d5d0fdd10e85 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.227882 17742 consensus_peers.cc:597] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.258944 17742 consensus_peers.cc:597] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.263435 17890 consensus_peers.cc:597] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.306907 17890 consensus_peers.cc:597] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.326237  6451 tablet_server.cc:196] TabletServer@127.6.76.193:37959 shutdown complete.
W20260709 12:30:51.356003 17816 consensus_peers.cc:597] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.368600 17742 consensus_peers.cc:597] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.373191 17890 consensus_peers.cc:597] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.388443 17816 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.400225 17890 consensus_peers.cc:597] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.456882  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
W20260709 12:30:51.476976 17816 consensus_peers.cc:597] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.480576 17742 consensus_peers.cc:597] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.517078 17815 consensus_peers.cc:597] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.519598 17742 consensus_peers.cc:597] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.564605 17815 consensus_peers.cc:597] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.590670 17889 consensus_peers.cc:597] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.599840  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:51.601255  6451 tablet_replica.cc:333] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.602634  6451 raft_consensus.cc:2243] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.603895  6451 raft_consensus.cc:2272] T 4bda730805b64dda88c94c3072246211 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.606370  6451 tablet_replica.cc:333] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.607066  6451 raft_consensus.cc:2243] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.607710  6451 raft_consensus.cc:2272] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.610673  6451 tablet_replica.cc:333] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.611786  6451 raft_consensus.cc:2243] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.612989  6451 raft_consensus.cc:2272] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.616490  6451 tablet_replica.cc:333] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.617336  6451 raft_consensus.cc:2243] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.617879  6451 raft_consensus.cc:2272] T 43b78182e2a746608309739058b74474 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.620831  6451 tablet_replica.cc:333] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.621587  6451 raft_consensus.cc:2243] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.622082  6451 raft_consensus.cc:2272] T 90406ef14e034b2daea6aaf73e737745 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.624663  6451 tablet_replica.cc:333] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.625737  6451 raft_consensus.cc:2243] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.626369  6451 raft_consensus.cc:2272] T 173b1a41d3a1403e92f3345206f45510 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.632200  6451 tablet_replica.cc:333] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.632959  6451 raft_consensus.cc:2243] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.633555  6451 raft_consensus.cc:2272] T de32f68d2ee24bd09e87ceae49cd7e81 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.636011  6451 tablet_replica.cc:333] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.636837  6451 raft_consensus.cc:2243] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.637467  6451 raft_consensus.cc:2272] T 67ea582b2cc3418bac58a2e7ceb32649 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.641669  6451 tablet_replica.cc:333] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
W20260709 12:30:51.642794 17889 consensus_peers.cc:597] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.642794  6451 raft_consensus.cc:2243] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.644564  6451 raft_consensus.cc:2272] T b3592a718f974e5f9f26e16b1942834f P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.648422  6451 tablet_replica.cc:333] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.650046  6451 raft_consensus.cc:2243] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.650689  6451 raft_consensus.cc:2272] T 6fc8a907276843128987dab0a2f52235 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.654222  6451 tablet_replica.cc:333] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.655108  6451 raft_consensus.cc:2243] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.656134  6451 raft_consensus.cc:2272] T 49f142d3ffe24e67b967c51897ba2e82 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.659092  6451 tablet_replica.cc:333] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.660434  6451 raft_consensus.cc:2243] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.661675  6451 raft_consensus.cc:2272] T fbb98822af0c4e85aa127325141dce47 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.665328  6451 tablet_replica.cc:333] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.666453  6451 raft_consensus.cc:2243] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.667274  6451 raft_consensus.cc:2272] T 21c2238a9361414181c3aff69d2f9dbd P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.671270  6451 tablet_replica.cc:333] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.672273  6451 raft_consensus.cc:2243] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.673643  6451 raft_consensus.cc:2272] T ea1fbecc92da4549828813c3a7185b1a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.677699  6451 tablet_replica.cc:333] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.678726  6451 raft_consensus.cc:2243] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.679433  6451 raft_consensus.cc:2272] T 952cd8f570354b1ea762c251df7bbc19 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.683060  6451 tablet_replica.cc:333] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.684176  6451 raft_consensus.cc:2243] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.685456  6451 raft_consensus.cc:2272] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.688781  6451 tablet_replica.cc:333] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.689476  6451 raft_consensus.cc:2243] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.690042  6451 raft_consensus.cc:2272] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.693318  6451 tablet_replica.cc:333] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.694281  6451 raft_consensus.cc:2243] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.695468  6451 raft_consensus.cc:2272] T 4460e907889c4b0d9b234ccf0f574eaf P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.696919 17889 consensus_peers.cc:597] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.698869  6451 tablet_replica.cc:333] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.700139  6451 raft_consensus.cc:2243] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.700974  6451 raft_consensus.cc:2272] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.704463  6451 tablet_replica.cc:333] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.705500  6451 raft_consensus.cc:2243] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.706736  6451 raft_consensus.cc:2272] T f3120efae4a24a319065841f0dff065a P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.710237  6451 tablet_replica.cc:333] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.711361  6451 raft_consensus.cc:2243] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.712661  6451 raft_consensus.cc:2272] T 6874bb3d07a84cc1b883375c3e83bcfe P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.716405  6451 tablet_replica.cc:333] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.717447  6451 raft_consensus.cc:2243] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.718647  6451 raft_consensus.cc:2272] T 5ae4ca22644a4128bc2662c6bb502884 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.722561  6451 tablet_replica.cc:333] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
W20260709 12:30:51.723598 17889 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.723718  6451 raft_consensus.cc:2243] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.729903  6451 raft_consensus.cc:2272] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.732251 17815 consensus_peers.cc:597] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.733855  6451 tablet_replica.cc:333] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.734992  6451 raft_consensus.cc:2243] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.736461  6451 raft_consensus.cc:2272] T af2da24cf13b4824be213c5a60b39803 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.745817 17815 consensus_peers.cc:597] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.746701 17889 consensus_peers.cc:597] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.747139  6451 tablet_replica.cc:333] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
W20260709 12:30:51.747197 17815 consensus_peers.cc:597] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.748704  6451 raft_consensus.cc:2243] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.749648  6451 raft_consensus.cc:2272] T 5bd790b3318a46d0a138b9649e5e447a P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.754446  6451 tablet_replica.cc:333] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.755415  6451 raft_consensus.cc:2243] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.756121  6451 raft_consensus.cc:2272] T 73f4f834976044f1be00f6e31249489e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.759778  6451 tablet_replica.cc:333] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.760764  6451 raft_consensus.cc:2243] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.761495  6451 raft_consensus.cc:2272] T 2f8bdb4d10d0433c909031a9340f6c1b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.766036  6451 tablet_replica.cc:333] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.767017  6451 raft_consensus.cc:2243] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.768287  6451 raft_consensus.cc:2272] T 20637ddae560402b81988ca96fc61b1e P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.774533 17889 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.776329  6451 tablet_replica.cc:333] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.777537  6451 raft_consensus.cc:2243] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.778262  6451 raft_consensus.cc:2272] T d7b12a0adfe741248970463752f3b30e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.782075  6451 tablet_replica.cc:333] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
W20260709 12:30:51.783027 17815 consensus_peers.cc:597] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.783136  6451 raft_consensus.cc:2243] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.784220  6451 raft_consensus.cc:2272] T f211414d5e584164a01d70059b214a4b P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.787027  6451 tablet_replica.cc:333] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.787779  6451 raft_consensus.cc:2243] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.789057  6451 raft_consensus.cc:2272] T 9cd5aff4923949cd9a68bdae18218da4 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.792266  6451 tablet_replica.cc:333] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.793183  6451 raft_consensus.cc:2243] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.793826  6451 raft_consensus.cc:2272] T edef181ff5d249908318aaf871e8b47e P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.798290  6451 tablet_replica.cc:333] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.799767  6451 raft_consensus.cc:2243] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.801832  6451 raft_consensus.cc:2272] T 1607aea0063f40f0a8d248cf7860749d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.806286  6451 tablet_replica.cc:333] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.807518  6451 raft_consensus.cc:2243] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.808555  6451 raft_consensus.cc:2272] T 85189191344a46deb54f096d7ac2c06d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.811172  6451 tablet_replica.cc:333] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.811947  6451 raft_consensus.cc:2243] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.812443  6451 raft_consensus.cc:2272] T 57e8cc31758742738306cb81f8d38d38 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.815318  6451 tablet_replica.cc:333] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.816305  6451 raft_consensus.cc:2243] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.817034  6451 raft_consensus.cc:2272] T 85f591a6d82c496ab6a106341934b97d P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.820348  6451 tablet_replica.cc:333] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.821280  6451 raft_consensus.cc:2243] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.821868  6451 raft_consensus.cc:2272] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.825845  6451 tablet_replica.cc:333] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.826785  6451 raft_consensus.cc:2243] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.827484  6451 raft_consensus.cc:2272] T 7da5cd04765b43bdaec5b9bae211aae0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.831378  6451 tablet_replica.cc:333] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.832329  6451 raft_consensus.cc:2243] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.833132  6451 raft_consensus.cc:2272] T deb7403698d84ba789b17c0ef904859f P 753c9642e55f4b819d92519d568c2d94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.837172  6451 tablet_replica.cc:333] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.838179  6451 raft_consensus.cc:2243] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.838883  6451 raft_consensus.cc:2272] T 3e90ea3001724feca1d5658aef941cf0 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.839048 17815 consensus_peers.cc:597] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.842135 17815 consensus_peers.cc:597] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.842440  6451 tablet_replica.cc:333] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.843508  6451 raft_consensus.cc:2243] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.845062  6451 raft_consensus.cc:2272] T f292bf1109ec493d9111ad805d777330 P 753c9642e55f4b819d92519d568c2d94 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.848127 17889 consensus_peers.cc:597] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.848948  6451 tablet_replica.cc:333] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.849942  6451 raft_consensus.cc:2243] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:51.851485  6451 raft_consensus.cc:2272] T 0ff59477c0d846eaab2e41621e730061 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.852233 17815 consensus_peers.cc:597] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.855050  6451 tablet_replica.cc:333] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.856093  6451 raft_consensus.cc:2243] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.856835  6451 raft_consensus.cc:2272] T 09a8ca1913d34de8bdacce55ee0fa4da P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:51.860133  6451 tablet_replica.cc:333] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94: stopping tablet replica
I20260709 12:30:51.860901  6451 raft_consensus.cc:2243] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:51.861513  6451 raft_consensus.cc:2272] T 6d2aef671e0349d6ab0d618271ae26a8 P 753c9642e55f4b819d92519d568c2d94 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:51.909533 17815 consensus_peers.cc:597] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.932301 17889 consensus_peers.cc:597] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.932590 17815 consensus_peers.cc:597] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:51.970678  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
I20260709 12:30:52.011394 19082 raft_consensus.cc:493] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.012359 19082 raft_consensus.cc:515] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:52.013756 17889 consensus_peers.cc:597] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:52.015313 19082 leader_election.cc:290] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:52.021123 17816 leader_election.cc:336] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:52.023411 17815 leader_election.cc:336] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:52.028149 17815 leader_election.cc:304] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:52.031318 19082 raft_consensus.cc:2749] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:52.069589 17889 consensus_peers.cc:597] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 -> Peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Couldn't send request to peer 753c9642e55f4b819d92519d568c2d94. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:52.074326 19074 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.075139 19074 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:52.078662 19074 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.083181 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c953b9330799457cb8d5c4ea252f5b30" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:52.169929  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
I20260709 12:30:52.779898 19054 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.780808 19054 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:52.788146 19085 raft_consensus.cc:493] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.792356 19086 raft_consensus.cc:493] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.795751 19086 raft_consensus.cc:515] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.793290 19088 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.798916 19088 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:52.800009 19086 leader_election.cc:290] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:52.804874 19088 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.807524 19087 raft_consensus.cc:493] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.808231 19087 raft_consensus.cc:515] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.812492 19054 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.815697 19085 raft_consensus.cc:515] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.818734 19085 leader_election.cc:290] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:52.846360 19085 raft_consensus.cc:493] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.847299 19085 raft_consensus.cc:515] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.850347 19085 leader_election.cc:290] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:52.825505 19091 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.826469 19086 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.884045 19091 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:52.884198 19086 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.915817 19086 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.821597 19087 leader_election.cc:290] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:52.860559 19090 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:52.921116 19090 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:52.925266 19090 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.932447 19090 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:52.933405 19090 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:52.937207 19090 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:52.964306 19090 raft_consensus.cc:493] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:52.966231 19090 raft_consensus.cc:515] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:52.972129 19090 leader_election.cc:290] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:52.980666 19091 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:52.982164 19070 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.009012 19070 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.021024 19070 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.021633 19070 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.022156 19070 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:53.024505 19070 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:53.027114 19070 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
W20260709 12:30:53.045058 17815 leader_election.cc:336] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.048060 17815 leader_election.cc:336] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.049434 17815 leader_election.cc:336] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.051622 17815 leader_election.cc:336] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.055538 17815 leader_election.cc:336] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.056382 17815 leader_election.cc:336] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.062944 19081 raft_consensus.cc:493] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.034459 19070 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.069330 19081 raft_consensus.cc:515] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.076157 19106 raft_consensus.cc:493] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.076881 19106 raft_consensus.cc:515] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.084543 19102 raft_consensus.cc:493] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.085223 19102 raft_consensus.cc:515] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:53.085511 17816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:37959, user_credentials={real_user=slave}} blocked reactor thread for 225108us
I20260709 12:30:53.085951 19101 raft_consensus.cc:493] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.086598 19101 raft_consensus.cc:515] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.082614 19100 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.115020 19102 leader_election.cc:290] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.115463 19081 leader_election.cc:290] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.117062 19102 raft_consensus.cc:493] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.117791 19102 raft_consensus.cc:515] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.122347 19103 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.130759 19102 leader_election.cc:290] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.124258 19070 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:53.132403 17816 leader_election.cc:336] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.133217 17816 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.140782 19091 raft_consensus.cc:2749] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.143563 19106 leader_election.cc:290] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.147918 19099 raft_consensus.cc:493] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.151512 19099 raft_consensus.cc:515] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.123241 19103 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.153267 19101 leader_election.cc:290] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.154655 19107 raft_consensus.cc:493] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.155470 19099 leader_election.cc:290] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.155395 19107 raft_consensus.cc:515] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.158097 19107 leader_election.cc:290] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.165638 19093 raft_consensus.cc:493] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.166515 19093 raft_consensus.cc:515] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:53.168269 17816 leader_election.cc:336] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.113459 19100 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.174582 19109 raft_consensus.cc:493] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.175750 19109 raft_consensus.cc:515] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.177359 19091 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.178344 19091 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.180531 19090 raft_consensus.cc:493] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.181516 19090 raft_consensus.cc:515] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:53.183370 19106 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.184254 19106 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.189707 19106 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.191252 19093 leader_election.cc:290] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.192122 19100 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:53.188179 19103 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.178925 19109 leader_election.cc:290] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.196278 19109 raft_consensus.cc:493] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.197038 19109 raft_consensus.cc:515] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.197870 19090 leader_election.cc:290] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.198066 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.198762 17919 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.200242 19109 leader_election.cc:290] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.201097 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.201983 17919 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
W20260709 12:30:53.213052 17815 leader_election.cc:336] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.214911 19106 raft_consensus.cc:493] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.215801 19106 raft_consensus.cc:515] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.218473 19106 leader_election.cc:290] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.219437 19103 raft_consensus.cc:493] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.221067 19103 raft_consensus.cc:515] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
W20260709 12:30:53.221455 17816 leader_election.cc:336] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.222118 17816 leader_election.cc:304] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.223253 19090 raft_consensus.cc:2749] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.223835 17816 leader_election.cc:336] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.224017 19103 leader_election.cc:290] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
W20260709 12:30:53.224575 17816 leader_election.cc:336] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.225183 17816 leader_election.cc:304] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.225988 19090 raft_consensus.cc:2749] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:30:53.226418 17816 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.227277 17816 leader_election.cc:336] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.227666 17815 leader_election.cc:336] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.227682 17816 leader_election.cc:304] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.228778 19091 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.229096 19090 raft_consensus.cc:2749] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.231884 17816 leader_election.cc:336] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.232442 17816 leader_election.cc:304] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.233078 19106 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.239063 17917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.245136 17917 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
W20260709 12:30:53.245393 17816 leader_election.cc:336] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.242198 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:53.246115 17816 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.246665 17816 leader_election.cc:336] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.247242 17816 leader_election.cc:304] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.246536 17919 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:53.248762 19082 raft_consensus.cc:2749] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.248626 19093 raft_consensus.cc:2749] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.248656 19106 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.252262 19106 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767)
I20260709 12:30:53.253635 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.254117 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.254519 17926 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.254966 17923 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.256678 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b6bcf23aeb44b0b8aea5466500e7f8d" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:53.260865 17889 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:38991, user_credentials={real_user=slave}} blocked reactor thread for 87167.6us
I20260709 12:30:53.261127 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.262042 17923 raft_consensus.cc:2468] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 1.
I20260709 12:30:53.262975 17818 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.264514 19090 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:30:53.264959 19090 raft_consensus.cc:493] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.265353 19090 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:53.268447 19091 raft_consensus.cc:493] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.269150 19091 raft_consensus.cc:515] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.269882 19093 raft_consensus.cc:493] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.270486 19093 raft_consensus.cc:515] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.271586 19091 leader_election.cc:290] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.273007 19093 leader_election.cc:290] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
W20260709 12:30:53.280092 17815 leader_election.cc:336] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.280814 17815 leader_election.cc:336] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.288375 19090 raft_consensus.cc:515] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:53.290542 17889 leader_election.cc:336] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.291452 17889 leader_election.cc:336] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.291541 17818 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.292258  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:53.293018 19091 raft_consensus.cc:2804] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:53.293743 19091 raft_consensus.cc:493] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.294193 19091 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:30:53.292203 17889 leader_election.cc:336] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.295926  6451 tablet_replica.cc:333] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
W20260709 12:30:53.296649 17889 leader_election.cc:336] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.297241  6451 raft_consensus.cc:2243] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.298125  6451 raft_consensus.cc:2272] T 6fc8a907276843128987dab0a2f52235 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.293000 19093 raft_consensus.cc:493] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.299588 19093 raft_consensus.cc:515] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.302486 19093 leader_election.cc:290] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.302753  6451 tablet_replica.cc:333] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.304311  6451 raft_consensus.cc:2243] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.305248  6451 raft_consensus.cc:2272] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.312216 17818 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.313306 19083 raft_consensus.cc:2804] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:53.313769 19083 raft_consensus.cc:493] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.314203 19083 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:30:53.297256 17889 leader_election.cc:336] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.319770 19106 raft_consensus.cc:493] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
W20260709 12:30:53.320338 17889 leader_election.cc:336] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.320815 19106 raft_consensus.cc:515] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } }
I20260709 12:30:53.321138 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4f834976044f1be00f6e31249489e" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:53.321539 17889 leader_election.cc:336] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.324723 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32f68d2ee24bd09e87ceae49cd7e81" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.327083 19090 leader_election.cc:290] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.327890 19106 leader_election.cc:290] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.328905 19091 raft_consensus.cc:515] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.329978 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2695d8605c4716960fd53bdc4eb899" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.334986 17919 raft_consensus.cc:3060] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:30:53.335827 17889 leader_election.cc:336] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.336828 17889 leader_election.cc:336] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.337376 17889 leader_election.cc:336] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.337885 17889 leader_election.cc:336] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.338368 17889 leader_election.cc:336] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.332044 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8101eccca0c484c9d5b29aaff2497db" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.340798 19083 raft_consensus.cc:515] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.343287  6451 tablet_replica.cc:333] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.344271 19082 raft_consensus.cc:493] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.344517  6451 raft_consensus.cc:2243] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.344892 19090 raft_consensus.cc:493] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.345764  6451 raft_consensus.cc:2272] T d3be2bb16cb74f8fb343216495dc07a3 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.344933 19082 raft_consensus.cc:515] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.347713 19090 raft_consensus.cc:515] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:53.349946 17889 leader_election.cc:336] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.354465 19090 leader_election.cc:290] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.355118 19093 raft_consensus.cc:493] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.355203  6451 tablet_replica.cc:333] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.359833 19093 raft_consensus.cc:515] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.360589  6451 raft_consensus.cc:2243] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.352005 17919 raft_consensus.cc:2468] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 4.
W20260709 12:30:53.362377 17815 leader_election.cc:336] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.357796 17818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.196:46335, user_credentials={real_user=slave}} blocked reactor thread for 45682.1us
W20260709 12:30:53.358409 17890 consensus_peers.cc:597] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.364111 17890 consensus_peers.cc:597] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.365130 17890 consensus_peers.cc:597] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:53.364042 17818 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.368364 19088 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:30:53.368885 19088 raft_consensus.cc:493] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.369313 19088 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:53.372936 19092 raft_consensus.cc:493] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.373623 19092 raft_consensus.cc:515] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:53.373844 17816 leader_election.cc:336] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.374732 17816 leader_election.cc:336] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.375113 19097 raft_consensus.cc:493] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.375137 19054 raft_consensus.cc:493] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.376008 19097 raft_consensus.cc:515] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:53.376124 19054 raft_consensus.cc:515] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
I20260709 12:30:53.379557 19054 leader_election.cc:290] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.379587  6451 raft_consensus.cc:2272] T 90406ef14e034b2daea6aaf73e737745 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.375243 17816 leader_election.cc:304] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.382663 19087 raft_consensus.cc:2749] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:53.379961 17890 leader_election.cc:336] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:30:53.383903 17890 leader_election.cc:336] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:30:53.386070 17890 consensus_peers.cc:597] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.388278 17890 consensus_peers.cc:597] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.388967 17890 consensus_peers.cc:597] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:53.386189 19097 leader_election.cc:290] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959), 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991)
I20260709 12:30:53.385398  6451 tablet_replica.cc:333] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
W20260709 12:30:53.389679 17890 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:53.387094 19088 raft_consensus.cc:515] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.390380  6451 raft_consensus.cc:2243] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.391158  6451 raft_consensus.cc:2272] T 43b78182e2a746608309739058b74474 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.390386 17890 leader_election.cc:336] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:53.392458 17818 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.392874 17890 consensus_peers.cc:597] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.393576 17890 consensus_peers.cc:597] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:53.394232 17890 consensus_peers.cc:597] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:53.394711 19097 raft_consensus.cc:2804] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:30:53.394884 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:53.395262 19097 raft_consensus.cc:493] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b96ea338e2a940f9b606d5d0fdd10e85)
I20260709 12:30:53.395716 19097 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:53.398119 17818 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.402751 19092 leader_election.cc:290] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.404757 19091 leader_election.cc:290] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.406968 19083 leader_election.cc:290] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.407581 19082 leader_election.cc:290] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.409559 19089 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:30:53.410096 19089 raft_consensus.cc:493] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.410540 19089 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:53.413206 19088 leader_election.cc:290] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.414573 19093 leader_election.cc:290] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.418924 19085 raft_consensus.cc:493] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.420223 19085 raft_consensus.cc:515] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
W20260709 12:30:53.395701 17890 leader_election.cc:336] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:30:53.423794 17632 debug-util.cc:398] Leaking SignalData structure 0x7b0800368660 after lost signal to thread 6454
W20260709 12:30:53.422951 17890 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:30:53.425277 17632 debug-util.cc:398] Leaking SignalData structure 0x7b08002eafc0 after lost signal to thread 17635
W20260709 12:30:53.425503 17890 leader_election.cc:336] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:53.426023 17890 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.425529 17818 leader_election.cc:304] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.427142 17632 debug-util.cc:398] Leaking SignalData structure 0x7b0800244540 after lost signal to thread 17876
W20260709 12:30:53.427408 17818 raft_consensus.cc:2686] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 632ff234ebc74963b295e6b50ccb78e6: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:30:53.428395 17818 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
W20260709 12:30:53.428913 17632 debug-util.cc:398] Leaking SignalData structure 0x7b0800271d40 after lost signal to thread 17950
I20260709 12:30:53.429353 19083 raft_consensus.cc:2804] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:53.429620 17818 leader_election.cc:304] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.429896 19083 raft_consensus.cc:493] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 753c9642e55f4b819d92519d568c2d94)
I20260709 12:30:53.430332 19083 raft_consensus.cc:3060] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.430508 19106 raft_consensus.cc:2749] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:30:53.423202 19085 leader_election.cc:290] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
W20260709 12:30:53.434916 17890 consensus_peers.cc:597] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 -> Peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Couldn't send request to peer 632ff234ebc74963b295e6b50ccb78e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:53.427197  6451 tablet_replica.cc:333] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
W20260709 12:30:53.435937 17890 leader_election.cc:336] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.443192 17890 leader_election.cc:304] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.443786  6451 raft_consensus.cc:2243] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.444032 19106 raft_consensus.cc:2749] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.445122  6451 raft_consensus.cc:2272] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.445250 17890 leader_election.cc:336] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.445752 17890 leader_election.cc:304] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.439965 19097 raft_consensus.cc:515] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:53.433137 17815 leader_election.cc:336] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.440833 19083 raft_consensus.cc:515] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.441289 19093 raft_consensus.cc:2749] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.442309 17818 leader_election.cc:304] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.421433 19089 raft_consensus.cc:515] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.450991 19083 leader_election.cc:290] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.454201 19089 leader_election.cc:290] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Requested vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.454442  6451 tablet_replica.cc:333] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.455020 19093 raft_consensus.cc:2749] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.456231 19106 raft_consensus.cc:2749] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.458253 17818 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 
I20260709 12:30:53.455662  6451 raft_consensus.cc:2243] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:30:53.459702 17890 leader_election.cc:336] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.460178 17890 leader_election.cc:304] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.461185 19106 raft_consensus.cc:2749] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:53.462833 17890 leader_election.cc:336] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.463462 17890 leader_election.cc:304] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.465126 19097 leader_election.cc:290] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Requested vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.466172 19106 raft_consensus.cc:2749] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.468080 17815 leader_election.cc:336] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.469473 17815 leader_election.cc:336] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
W20260709 12:30:53.470121 17815 leader_election.cc:336] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.473167 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e7941692924de28d3fb53cf3cd833a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.473727 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffede44de4554930b68441a774c0dd06" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.473953 17919 raft_consensus.cc:3060] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:53.485146 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2933687c848f48e2b01b32ede660a4c9" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.485795 17923 raft_consensus.cc:3060] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:53.487766 17917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67ea582b2cc3418bac58a2e7ceb32649" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.497303 17926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "051771ffcb694816b9aff49434d710aa" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.497951 17926 raft_consensus.cc:3060] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:53.501333 17919 raft_consensus.cc:2468] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.505590 17926 raft_consensus.cc:2468] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 4.
I20260709 12:30:53.506166 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59353fd7c83841ea9e358df44da38a6f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
W20260709 12:30:53.509085 17816 leader_election.cc:336] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.509853 17816 leader_election.cc:304] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.510741  6451 raft_consensus.cc:2272] T fbb98822af0c4e85aa127325141dce47 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.510865 19093 raft_consensus.cc:493] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.511677 19093 raft_consensus.cc:515] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.514365 19093 leader_election.cc:290] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335), b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959)
I20260709 12:30:53.514773  6451 tablet_replica.cc:333] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.514971 19089 raft_consensus.cc:2804] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:53.515540 19089 raft_consensus.cc:697] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.515851  6451 raft_consensus.cc:2243] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.517066  6451 raft_consensus.cc:2272] T 9ea37968d44c4ec5a4320301e4173d8b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.516480 19089 consensus_queue.cc:237] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.520639  6451 tablet_replica.cc:333] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.521605  6451 raft_consensus.cc:2243] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:53.504891 17890 leader_election.cc:336] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.522476 19097 raft_consensus.cc:2749] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:53.522858 17816 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.522218 17890 leader_election.cc:304] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.525588 17815 leader_election.cc:336] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.522235  6451 raft_consensus.cc:2272] T de32f68d2ee24bd09e87ceae49cd7e81 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.531713 17816 leader_election.cc:336] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.531993 19106 raft_consensus.cc:2749] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.532435 17816 leader_election.cc:304] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.533206 19097 raft_consensus.cc:2749] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.533427 17816 leader_election.cc:336] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.533926 17816 leader_election.cc:304] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.534190 17890 leader_election.cc:336] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.534674 17890 leader_election.cc:304] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.535166 17816 leader_election.cc:336] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.535946 19097 raft_consensus.cc:2749] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.537469 17816 leader_election.cc:304] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.538228 19097 raft_consensus.cc:2749] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.537006  6451 tablet_replica.cc:333] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.543812 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd790b3318a46d0a138b9649e5e447a" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.543849 17923 raft_consensus.cc:2468] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.546816 17919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd5aff4923949cd9a68bdae18218da4" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.548252 17919 raft_consensus.cc:3060] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:53.548694 17890 leader_election.cc:336] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.549201 17890 leader_election.cc:304] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.549907 17918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd0b912e8f4ed3a78dbc39c92aa5f3" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
W20260709 12:30:53.550155 17890 leader_election.cc:336] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.550697 17890 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.551079 17922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a387a07df404a3a8cdec9c49a791e6f" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.551594 19100 raft_consensus.cc:2749] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:53.551774 17816 leader_election.cc:336] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.547820 17923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4460e907889c4b0d9b234ccf0f574eaf" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250"
I20260709 12:30:53.552740 19100 raft_consensus.cc:2749] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:30:53.553206 17923 raft_consensus.cc:3060] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:53.550693 17918 raft_consensus.cc:3060] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:53.545132  6451 raft_consensus.cc:2243] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:53.564240 17815 leader_election.cc:336] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.552352 17816 leader_election.cc:304] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.566883 17818 leader_election.cc:304] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.570039 17923 raft_consensus.cc:2468] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 4.
I20260709 12:30:53.570546 19100 raft_consensus.cc:2749] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.570847 17890 leader_election.cc:336] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.571509 17890 leader_election.cc:304] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.572279 19100 raft_consensus.cc:2749] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:53.573570 17890 leader_election.cc:336] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.574009 17890 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.575019 17890 leader_election.cc:336] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.575407 19103 raft_consensus.cc:2749] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.576831 19097 raft_consensus.cc:2706] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Received pre-election callback for term 3 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:30:53.577684 17919 raft_consensus.cc:2468] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 2.
I20260709 12:30:53.578606  6451 raft_consensus.cc:2272] T 67ea582b2cc3418bac58a2e7ceb32649 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.578667 19085 raft_consensus.cc:2749] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:53.580145 17816 leader_election.cc:336] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.580749 17816 leader_election.cc:336] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.581290 17816 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.581959 17816 leader_election.cc:336] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
W20260709 12:30:53.582553 17816 leader_election.cc:336] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.582907  6451 tablet_replica.cc:333] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
W20260709 12:30:53.583048 17816 leader_election.cc:336] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.584137  6451 raft_consensus.cc:2243] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.585894  6451 raft_consensus.cc:2272] T 952cd8f570354b1ea762c251df7bbc19 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.575498 17890 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.584376 17818 leader_election.cc:304] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.593138 19085 raft_consensus.cc:2804] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:30:53.593281 17890 leader_election.cc:336] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.593894 19085 raft_consensus.cc:697] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.594162 17890 leader_election.cc:304] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.595185 19085 consensus_queue.cc:237] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.597752 19103 raft_consensus.cc:2749] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.585721 17918 raft_consensus.cc:2468] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 632ff234ebc74963b295e6b50ccb78e6 in term 3.
I20260709 12:30:53.599915 19100 raft_consensus.cc:2749] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.601706 17818 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.603406 19089 raft_consensus.cc:2804] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:53.604014 19089 raft_consensus.cc:697] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.604964 19089 consensus_queue.cc:237] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.609387 17818 leader_election.cc:304] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.610460 19097 raft_consensus.cc:2749] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.611367 17818 leader_election.cc:304] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.612378 19083 raft_consensus.cc:2749] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.590224  6451 tablet_replica.cc:333] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.613726  6451 raft_consensus.cc:2243] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.615024  6451 raft_consensus.cc:2272] T b3592a718f974e5f9f26e16b1942834f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.595618 17890 leader_election.cc:336] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.618381 17890 leader_election.cc:304] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.619804 17890 leader_election.cc:336] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.620435 17890 leader_election.cc:304] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.619820 19103 raft_consensus.cc:2749] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.615526 17818 leader_election.cc:304] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.621909 19104 raft_consensus.cc:2749] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:30:53.623593 17890 leader_election.cc:336] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.624639 17890 leader_election.cc:304] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, b96ea338e2a940f9b606d5d0fdd10e85
W20260709 12:30:53.626156 17890 leader_election.cc:336] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.626746 17890 leader_election.cc:304] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.627714 19083 raft_consensus.cc:2804] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:53.628304 19083 raft_consensus.cc:697] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.629302 17818 leader_election.cc:304] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.629310 19083 consensus_queue.cc:237] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } }
I20260709 12:30:53.631018 17818 leader_election.cc:304] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.631430 19093 raft_consensus.cc:2749] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.632244 19097 raft_consensus.cc:2804] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:53.633857 19097 raft_consensus.cc:697] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.635304 19097 consensus_queue.cc:237] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.633036 17818 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.642306 17818 leader_election.cc:304] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d17e55b097491f895c84a3443cb250, 632ff234ebc74963b295e6b50ccb78e6; no voters: b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.641310 19106 raft_consensus.cc:2749] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:53.631474 19104 raft_consensus.cc:2749] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:30:53.651151 19046 raft_consensus.cc:2804] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:53.655027  6451 tablet_replica.cc:333] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.659349  6451 raft_consensus.cc:2243] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.652129 19082 raft_consensus.cc:2804] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:53.667667 19046 raft_consensus.cc:697] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.668900 19046 consensus_queue.cc:237] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.653947 19093 raft_consensus.cc:493] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 30d17e55b097491f895c84a3443cb250)
I20260709 12:30:53.670135 19093 raft_consensus.cc:515] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753c9642e55f4b819d92519d568c2d94" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 38991 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } } peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } }
I20260709 12:30:53.671823  6451 raft_consensus.cc:2272] T c953b9330799457cb8d5c4ea252f5b30 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.672678 19093 leader_election.cc:290] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991), 30d17e55b097491f895c84a3443cb250 (127.6.76.196:46335)
I20260709 12:30:53.672353 19082 raft_consensus.cc:697] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Becoming Leader. State: Replica: 632ff234ebc74963b295e6b50ccb78e6, State: Running, Role: LEADER
I20260709 12:30:53.676380 17918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e90ea3001724feca1d5658aef941cf0" candidate_uuid: "632ff234ebc74963b295e6b50ccb78e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d17e55b097491f895c84a3443cb250" is_pre_election: true
I20260709 12:30:53.675426 19082 consensus_queue.cc:237] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d17e55b097491f895c84a3443cb250" member_type: VOTER last_known_addr { host: "127.6.76.196" port: 46335 } } peers { permanent_uuid: "b96ea338e2a940f9b606d5d0fdd10e85" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 37959 } } peers { permanent_uuid: "632ff234ebc74963b295e6b50ccb78e6" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 45767 } }
W20260709 12:30:53.680478 17815 leader_election.cc:336] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 753c9642e55f4b819d92519d568c2d94 (127.6.76.194:38991): Network error: Client connection negotiation failed: client connection to 127.6.76.194:38991: connect: Connection refused (error 111)
I20260709 12:30:53.681823 17818 leader_election.cc:304] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 632ff234ebc74963b295e6b50ccb78e6; no voters: 30d17e55b097491f895c84a3443cb250, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.689241  6451 tablet_replica.cc:333] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.699914  6451 raft_consensus.cc:2243] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.700685  6451 raft_consensus.cc:2272] T d7b12a0adfe741248970463752f3b30e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.705902 17890 leader_election.cc:336] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 632ff234ebc74963b295e6b50ccb78e6 (127.6.76.195:45767): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:53.689249 19089 raft_consensus.cc:2749] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.706514 17890 leader_election.cc:304] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 632ff234ebc74963b295e6b50ccb78e6, 753c9642e55f4b819d92519d568c2d94
I20260709 12:30:53.708493 19100 raft_consensus.cc:2749] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.715459  6451 tablet_replica.cc:333] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.716974  6451 raft_consensus.cc:2243] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.717888  6451 raft_consensus.cc:2272] T 20637ddae560402b81988ca96fc61b1e P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:53.724283 17890 leader_election.cc:336] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111)
I20260709 12:30:53.724920 17890 leader_election.cc:304] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d17e55b097491f895c84a3443cb250; no voters: 753c9642e55f4b819d92519d568c2d94, b96ea338e2a940f9b606d5d0fdd10e85
I20260709 12:30:53.725365  6451 tablet_replica.cc:333] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.725638 19104 raft_consensus.cc:2749] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:53.727691  6451 raft_consensus.cc:2243] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.728782  6451 pending_rounds.cc:70] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.729141  6451 pending_rounds.cc:77] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626638808674304 op_type: NO_OP noop_request { }
I20260709 12:30:53.729650  6451 raft_consensus.cc:2889] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.730099  6451 raft_consensus.cc:2272] T 16e7941692924de28d3fb53cf3cd833a P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.733969  6451 tablet_replica.cc:333] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.734967  6451 raft_consensus.cc:2243] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.735656  6451 raft_consensus.cc:2272] T 59353fd7c83841ea9e358df44da38a6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.739120  6451 tablet_replica.cc:333] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.740108  6451 raft_consensus.cc:2243] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.740836  6451 raft_consensus.cc:2272] T af2da24cf13b4824be213c5a60b39803 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.744050  6451 tablet_replica.cc:333] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.744900  6451 raft_consensus.cc:2243] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.745576  6451 raft_consensus.cc:2272] T 5bd790b3318a46d0a138b9649e5e447a P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.748732  6451 tablet_replica.cc:333] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.749681  6451 raft_consensus.cc:2243] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.750618  6451 pending_rounds.cc:70] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.750942  6451 pending_rounds.cc:77] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305626639028445184 op_type: NO_OP noop_request { }
I20260709 12:30:53.751365  6451 raft_consensus.cc:2889] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.751737  6451 raft_consensus.cc:2272] T 4460e907889c4b0d9b234ccf0f574eaf P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.756659  6451 tablet_replica.cc:333] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.757535  6451 raft_consensus.cc:2243] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.758387  6451 pending_rounds.cc:70] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.758649  6451 pending_rounds.cc:77] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305626638777225216 op_type: NO_OP noop_request { }
I20260709 12:30:53.759037  6451 raft_consensus.cc:2889] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.759451  6451 raft_consensus.cc:2272] T 051771ffcb694816b9aff49434d710aa P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.763507  6451 tablet_replica.cc:333] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.764446  6451 raft_consensus.cc:2243] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.765105  6451 raft_consensus.cc:2272] T 6874bb3d07a84cc1b883375c3e83bcfe P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.768350  6451 tablet_replica.cc:333] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.769207  6451 raft_consensus.cc:2243] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.770403  6451 raft_consensus.cc:2272] T deb7403698d84ba789b17c0ef904859f P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.773528  6451 tablet_replica.cc:333] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.774396  6451 raft_consensus.cc:2243] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.775657  6451 raft_consensus.cc:2272] T 09de1ae34c854240a05f695427ac7c94 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.779433  6451 tablet_replica.cc:333] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.780407  6451 raft_consensus.cc:2243] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.781100  6451 raft_consensus.cc:2272] T 9a387a07df404a3a8cdec9c49a791e6f P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.783838  6451 tablet_replica.cc:333] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.784554  6451 raft_consensus.cc:2243] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.785176  6451 raft_consensus.cc:2272] T 4872dc0c0f3c4bf69e5618fae1ac5f49 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.787744  6451 tablet_replica.cc:333] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.788486  6451 raft_consensus.cc:2243] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.789485  6451 raft_consensus.cc:2272] T f211414d5e584164a01d70059b214a4b P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.792992  6451 tablet_replica.cc:333] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.793753  6451 raft_consensus.cc:2243] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.794526  6451 pending_rounds.cc:70] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.794792  6451 pending_rounds.cc:77] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626639094341632 op_type: NO_OP noop_request { }
I20260709 12:30:53.795162  6451 raft_consensus.cc:2889] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.795569  6451 raft_consensus.cc:2272] T 9cd5aff4923949cd9a68bdae18218da4 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.798766  6451 tablet_replica.cc:333] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.799568  6451 raft_consensus.cc:2243] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.800336  6451 pending_rounds.cc:70] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.800586  6451 pending_rounds.cc:77] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305626638422548480 op_type: NO_OP noop_request { }
I20260709 12:30:53.800961  6451 raft_consensus.cc:2889] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.801291  6451 raft_consensus.cc:2272] T bb2695d8605c4716960fd53bdc4eb899 P 632ff234ebc74963b295e6b50ccb78e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.805104  6451 tablet_replica.cc:333] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.805960  6451 raft_consensus.cc:2243] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.807148  6451 raft_consensus.cc:2272] T 25f7db027e2c4f5186a0d0c50ed0d6d6 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.809963  6451 tablet_replica.cc:333] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.810823  6451 raft_consensus.cc:2243] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.811987  6451 raft_consensus.cc:2272] T 7da5cd04765b43bdaec5b9bae211aae0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.814894  6451 tablet_replica.cc:333] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.815642  6451 raft_consensus.cc:2243] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.816663  6451 raft_consensus.cc:2272] T 09a8ca1913d34de8bdacce55ee0fa4da P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.819139  6451 tablet_replica.cc:333] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.819840  6451 raft_consensus.cc:2243] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.820822  6451 raft_consensus.cc:2272] T 3e90ea3001724feca1d5658aef941cf0 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.823516  6451 tablet_replica.cc:333] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.825344  6451 raft_consensus.cc:2243] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.826159  6451 raft_consensus.cc:2272] T f292bf1109ec493d9111ad805d777330 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.828891  6451 tablet_replica.cc:333] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.829593  6451 raft_consensus.cc:2243] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.830135  6451 raft_consensus.cc:2272] T 0ff59477c0d846eaab2e41621e730061 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.833290  6451 tablet_replica.cc:333] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.834239  6451 raft_consensus.cc:2243] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.834936  6451 raft_consensus.cc:2272] T d8101eccca0c484c9d5b29aaff2497db P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.837466  6451 tablet_replica.cc:333] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.838421  6451 raft_consensus.cc:2243] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.839401  6451 raft_consensus.cc:2272] T ffede44de4554930b68441a774c0dd06 P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.842321  6451 tablet_replica.cc:333] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.843040  6451 raft_consensus.cc:2243] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.844159  6451 pending_rounds.cc:70] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.844571  6451 pending_rounds.cc:77] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626638996406272 op_type: NO_OP noop_request { }
I20260709 12:30:53.844889  6451 raft_consensus.cc:2889] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.845202  6451 raft_consensus.cc:2272] T 2933687c848f48e2b01b32ede660a4c9 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.848417  6451 tablet_replica.cc:333] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.849175  6451 raft_consensus.cc:2243] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.849743  6451 raft_consensus.cc:2272] T 73f4f834976044f1be00f6e31249489e P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.852337  6451 tablet_replica.cc:333] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.853044  6451 raft_consensus.cc:2243] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.853820  6451 pending_rounds.cc:70] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Trying to abort 1 pending ops.
I20260709 12:30:53.854087  6451 pending_rounds.cc:77] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626639075872768 op_type: NO_OP noop_request { }
I20260709 12:30:53.854418  6451 raft_consensus.cc:2889] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:53.854744  6451 raft_consensus.cc:2272] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 632ff234ebc74963b295e6b50ccb78e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.857903  6451 tablet_replica.cc:333] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.858727  6451 raft_consensus.cc:2243] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.859910  6451 raft_consensus.cc:2272] T 1607aea0063f40f0a8d248cf7860749d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.863193  6451 tablet_replica.cc:333] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.864107  6451 raft_consensus.cc:2243] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.865144  6451 raft_consensus.cc:2272] T 57e8cc31758742738306cb81f8d38d38 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.867699  6451 tablet_replica.cc:333] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.868319  6451 raft_consensus.cc:2243] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:53.869256  6451 raft_consensus.cc:2272] T 79b1348052e745648f1cba4c11a7e863 P 632ff234ebc74963b295e6b50ccb78e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.871757  6451 tablet_replica.cc:333] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6: stopping tablet replica
I20260709 12:30:53.872438  6451 raft_consensus.cc:2243] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:53.873234  6451 raft_consensus.cc:2272] T 85f591a6d82c496ab6a106341934b97d P 632ff234ebc74963b295e6b50ccb78e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:53.971273  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:30:54.116326  6451 tablet_server.cc:179] TabletServer@127.6.76.196:0 shutting down...
W20260709 12:30:54.174060 17890 consensus_peers.cc:597] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:54.215287 17890 consensus_peers.cc:597] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:54.233655 17890 consensus_peers.cc:597] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 -> Peer b96ea338e2a940f9b606d5d0fdd10e85 (127.6.76.193:37959): Couldn't send request to peer b96ea338e2a940f9b606d5d0fdd10e85. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:37959: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:54.242451  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:54.243551  6451 tablet_replica.cc:333] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.244336  6451 raft_consensus.cc:2243] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.244958  6451 raft_consensus.cc:2272] T d3be2bb16cb74f8fb343216495dc07a3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.248384  6451 tablet_replica.cc:333] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.249230  6451 raft_consensus.cc:2243] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.249939  6451 raft_consensus.cc:2272] T 4bda730805b64dda88c94c3072246211 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.252899  6451 tablet_replica.cc:333] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.253692  6451 raft_consensus.cc:2243] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.254667  6451 raft_consensus.cc:2272] T 3e90ea3001724feca1d5658aef941cf0 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.257409  6451 tablet_replica.cc:333] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.258190  6451 raft_consensus.cc:2243] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.260141  6451 raft_consensus.cc:2272] T 21c2238a9361414181c3aff69d2f9dbd P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.263597  6451 tablet_replica.cc:333] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.264593  6451 raft_consensus.cc:2243] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.265698  6451 raft_consensus.cc:2272] T 90406ef14e034b2daea6aaf73e737745 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.268524  6451 tablet_replica.cc:333] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.269469  6451 raft_consensus.cc:2243] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.270113  6451 raft_consensus.cc:2272] T 2f87c2ee9429444babb65d1c8a1ec7c1 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.273561  6451 tablet_replica.cc:333] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.274446  6451 raft_consensus.cc:2243] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.275120  6451 raft_consensus.cc:2272] T cdc87abcb4d04cbb8196b5b8ecc06ec5 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.278266  6451 tablet_replica.cc:333] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.279011  6451 raft_consensus.cc:2243] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.280045  6451 raft_consensus.cc:2272] T 67ea582b2cc3418bac58a2e7ceb32649 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.282863  6451 tablet_replica.cc:333] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.283685  6451 raft_consensus.cc:2243] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.284245  6451 raft_consensus.cc:2272] T 5ae4ca22644a4128bc2662c6bb502884 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.286765  6451 tablet_replica.cc:333] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.287540  6451 raft_consensus.cc:2243] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.288574  6451 raft_consensus.cc:2272] T 9a387a07df404a3a8cdec9c49a791e6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.291190  6451 tablet_replica.cc:333] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.292085  6451 raft_consensus.cc:2243] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.292826  6451 raft_consensus.cc:2272] T 9ea37968d44c4ec5a4320301e4173d8b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.295646  6451 tablet_replica.cc:333] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.296409  6451 raft_consensus.cc:2243] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.297329  6451 raft_consensus.cc:2272] T de32f68d2ee24bd09e87ceae49cd7e81 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.299646  6451 tablet_replica.cc:333] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.300463  6451 raft_consensus.cc:2243] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.301810  6451 raft_consensus.cc:2272] T 4b6bcf23aeb44b0b8aea5466500e7f8d P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.305859  6451 tablet_replica.cc:333] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.306936  6451 raft_consensus.cc:2243] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.307603  6451 raft_consensus.cc:2272] T ea1fbecc92da4549828813c3a7185b1a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.310273  6451 tablet_replica.cc:333] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.311022  6451 raft_consensus.cc:2243] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.311682  6451 raft_consensus.cc:2272] T b3592a718f974e5f9f26e16b1942834f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.314358  6451 tablet_replica.cc:333] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.315119  6451 raft_consensus.cc:2243] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.315732  6451 raft_consensus.cc:2272] T 173b1a41d3a1403e92f3345206f45510 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.318254  6451 tablet_replica.cc:333] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.319008  6451 raft_consensus.cc:2243] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.320089  6451 raft_consensus.cc:2272] T ffede44de4554930b68441a774c0dd06 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.322917  6451 tablet_replica.cc:333] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.323714  6451 raft_consensus.cc:2243] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.324285  6451 raft_consensus.cc:2272] T 2933687c848f48e2b01b32ede660a4c9 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.327335  6451 tablet_replica.cc:333] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.328104  6451 raft_consensus.cc:2243] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.329210  6451 raft_consensus.cc:2272] T d8101eccca0c484c9d5b29aaff2497db P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.332191  6451 tablet_replica.cc:333] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.332917  6451 raft_consensus.cc:2243] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.333910  6451 raft_consensus.cc:2272] T 59353fd7c83841ea9e358df44da38a6f P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.336848  6451 tablet_replica.cc:333] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.337565  6451 raft_consensus.cc:2243] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.338214  6451 raft_consensus.cc:2272] T e1a94780ceb44c84bbaa8ba8ce21b7c3 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.341202  6451 tablet_replica.cc:333] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.342067  6451 raft_consensus.cc:2243] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.343045  6451 raft_consensus.cc:2272] T ffeea622a1d642ffb92c2fe4b4dcaa57 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.346026  6451 tablet_replica.cc:333] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.346733  6451 raft_consensus.cc:2243] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.347359  6451 raft_consensus.cc:2272] T deb7403698d84ba789b17c0ef904859f P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.349823  6451 tablet_replica.cc:333] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.350544  6451 raft_consensus.cc:2243] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.351097  6451 raft_consensus.cc:2272] T 09de1ae34c854240a05f695427ac7c94 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.353538  6451 tablet_replica.cc:333] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.354242  6451 raft_consensus.cc:2243] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.355361  6451 raft_consensus.cc:2272] T 5bd790b3318a46d0a138b9649e5e447a P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.358079  6451 tablet_replica.cc:333] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.358829  6451 raft_consensus.cc:2243] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.359462  6451 raft_consensus.cc:2272] T 4460e907889c4b0d9b234ccf0f574eaf P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.361778  6451 tablet_replica.cc:333] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.362473  6451 raft_consensus.cc:2243] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.363060  6451 raft_consensus.cc:2272] T 6e0b2be2ba8b41579122c1d5d17a6b26 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.365501  6451 tablet_replica.cc:333] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.366178  6451 raft_consensus.cc:2243] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.366755  6451 raft_consensus.cc:2272] T 051771ffcb694816b9aff49434d710aa P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.369396  6451 tablet_replica.cc:333] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.370081  6451 raft_consensus.cc:2243] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.370596  6451 raft_consensus.cc:2272] T f3120efae4a24a319065841f0dff065a P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.373131  6451 tablet_replica.cc:333] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.373911  6451 raft_consensus.cc:2243] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.374465  6451 raft_consensus.cc:2272] T 16e7941692924de28d3fb53cf3cd833a P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.377326  6451 tablet_replica.cc:333] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.378132  6451 raft_consensus.cc:2243] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.379094  6451 raft_consensus.cc:2272] T 2f8bdb4d10d0433c909031a9340f6c1b P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.381947  6451 tablet_replica.cc:333] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.382630  6451 raft_consensus.cc:2243] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.383184  6451 raft_consensus.cc:2272] T c953b9330799457cb8d5c4ea252f5b30 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.385700  6451 tablet_replica.cc:333] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.386343  6451 raft_consensus.cc:2243] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.386871  6451 raft_consensus.cc:2272] T ddfd0b912e8f4ed3a78dbc39c92aa5f3 P 30d17e55b097491f895c84a3443cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.389194  6451 tablet_replica.cc:333] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.389861  6451 raft_consensus.cc:2243] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.390388  6451 raft_consensus.cc:2272] T 09a8ca1913d34de8bdacce55ee0fa4da P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.392836  6451 tablet_replica.cc:333] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.393474  6451 raft_consensus.cc:2243] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.394009  6451 raft_consensus.cc:2272] T 49f142d3ffe24e67b967c51897ba2e82 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.396400  6451 tablet_replica.cc:333] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.397011  6451 raft_consensus.cc:2243] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.397540  6451 raft_consensus.cc:2272] T 6d2aef671e0349d6ab0d618271ae26a8 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.400084  6451 tablet_replica.cc:333] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.400741  6451 raft_consensus.cc:2243] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.401691  6451 raft_consensus.cc:2272] T edef181ff5d249908318aaf871e8b47e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.404325  6451 tablet_replica.cc:333] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.404976  6451 raft_consensus.cc:2243] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.405493  6451 raft_consensus.cc:2272] T 9cd5aff4923949cd9a68bdae18218da4 P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.408046  6451 tablet_replica.cc:333] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.408735  6451 raft_consensus.cc:2243] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.409250  6451 raft_consensus.cc:2272] T bb2695d8605c4716960fd53bdc4eb899 P 30d17e55b097491f895c84a3443cb250 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.411587  6451 tablet_replica.cc:333] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.412209  6451 raft_consensus.cc:2243] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.412695  6451 raft_consensus.cc:2272] T 7da5cd04765b43bdaec5b9bae211aae0 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.415036  6451 tablet_replica.cc:333] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.415684  6451 raft_consensus.cc:2243] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.416237  6451 raft_consensus.cc:2272] T 85189191344a46deb54f096d7ac2c06d P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.418536  6451 tablet_replica.cc:333] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.419159  6451 raft_consensus.cc:2243] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.419762  6451 raft_consensus.cc:2272] T 57e8cc31758742738306cb81f8d38d38 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.422115  6451 tablet_replica.cc:333] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.422632  6451 raft_consensus.cc:2243] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:54.423043  6451 raft_consensus.cc:2272] T 79b1348052e745648f1cba4c11a7e863 P 30d17e55b097491f895c84a3443cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.425617  6451 tablet_replica.cc:333] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250: stopping tablet replica
I20260709 12:30:54.426209  6451 raft_consensus.cc:2243] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.427145  6451 raft_consensus.cc:2272] T 73f4f834976044f1be00f6e31249489e P 30d17e55b097491f895c84a3443cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.508697  6451 tablet_server.cc:196] TabletServer@127.6.76.196:0 shutdown complete.
I20260709 12:30:54.610741  6451 master.cc:562] Master@127.6.76.254:33695 shutting down...
I20260709 12:30:54.656064  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:54.656980  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:54.657503  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P d05bd585df814ae9904c4dc50a8943f2: stopping tablet replica
W20260709 12:30:55.680182  6451 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 17632)
W20260709 12:30:56.681145  6451 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 17632)
I20260709 12:30:57.133389  6451 master.cc:584] Master@127.6.76.254:33695 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (48385 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:30:57.281303  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:43155
I20260709 12:30:57.282745  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:57.290621 19125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.290818 19124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.291379 19127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:57.292362  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:57.293962  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:57.294217  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:57.294461  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600257294422 us; error 0 us; skew 500 ppm
I20260709 12:30:57.295112  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:57.298532  6451 webserver.cc:533] Webserver started at http://127.6.76.254:40331/ using document root <none> and password file <none>
I20260709 12:30:57.299171  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:57.299516  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:57.299876  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:57.301299  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "344fa86650ef4a1ba76a0ef1e8df7600"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.306830  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:30:57.312224 19132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.313289  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:30:57.313795  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "344fa86650ef4a1ba76a0ef1e8df7600"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.314160  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:57.332926  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:57.334492  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:57.384673  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:43155
I20260709 12:30:57.384896 19183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:43155 every 8 connection(s)
I20260709 12:30:57.389766 19184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:57.404031 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600: Bootstrap starting.
I20260709 12:30:57.409574 19184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:57.415289 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600: No bootstrap required, opened a new log
I20260709 12:30:57.418496 19184 raft_consensus.cc:359] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER }
I20260709 12:30:57.419085 19184 raft_consensus.cc:385] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:57.419339 19184 raft_consensus.cc:740] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344fa86650ef4a1ba76a0ef1e8df7600, State: Initialized, Role: FOLLOWER
I20260709 12:30:57.419951 19184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER }
I20260709 12:30:57.420481 19184 raft_consensus.cc:399] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:57.420713 19184 raft_consensus.cc:493] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:57.420964 19184 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:57.426667 19184 raft_consensus.cc:515] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER }
I20260709 12:30:57.427398 19184 leader_election.cc:304] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 344fa86650ef4a1ba76a0ef1e8df7600; no voters: 
I20260709 12:30:57.428840 19184 leader_election.cc:290] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:57.429499 19187 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:57.431404 19187 raft_consensus.cc:697] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 1 LEADER]: Becoming Leader. State: Replica: 344fa86650ef4a1ba76a0ef1e8df7600, State: Running, Role: LEADER
I20260709 12:30:57.432437 19184 sys_catalog.cc:565] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:57.432382 19187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER }
I20260709 12:30:57.441787 19189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 344fa86650ef4a1ba76a0ef1e8df7600. Latest consensus state: current_term: 1 leader_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER } }
I20260709 12:30:57.442468 19189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:57.444969 19197 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:57.444633 19188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344fa86650ef4a1ba76a0ef1e8df7600" member_type: VOTER } }
I20260709 12:30:57.445413 19188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:57.451140 19197 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:57.452363  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:57.463470 19197 catalog_manager.cc:1383] Generated new cluster ID: a0e30e5aafe64b45af6ee076d0bee5cb
I20260709 12:30:57.463914 19197 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:57.486557 19197 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:57.488468 19197 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:57.511850 19197 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600: Generated new TSK 0
I20260709 12:30:57.512616 19197 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:57.520552  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:57.527961 19205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.529585 19206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.535274 19208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:57.535563  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:57.537177  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:57.537439  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:57.537621  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600257537601 us; error 0 us; skew 500 ppm
I20260709 12:30:57.538285  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:57.541646  6451 webserver.cc:533] Webserver started at http://127.6.76.193:41329/ using document root <none> and password file <none>
I20260709 12:30:57.542284  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:57.542536  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:57.542846  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:57.544390  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "f41fa709b40c4b409a9b8cb61077dd36"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.550482  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:30:57.555413 19213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.556569  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:57.556945  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "f41fa709b40c4b409a9b8cb61077dd36"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.557262  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:57.571700  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:57.573385  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:57.575412  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:57.578733  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:57.578963  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.579206  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:57.579388  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 12:30:57.637024  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:46571
I20260709 12:30:57.637256 19275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:46571 every 8 connection(s)
I20260709 12:30:57.643507  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:57.652457 19280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.655872 19281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.662062 19283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:57.662113  6451 server_base.cc:1061] running on GCE node
I20260709 12:30:57.662973 19276 heartbeater.cc:344] Connected to a master server at 127.6.76.254:43155
I20260709 12:30:57.663707 19276 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:57.664280  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:57.664580  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:57.664729  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600257664717 us; error 0 us; skew 500 ppm
I20260709 12:30:57.664763 19276 heartbeater.cc:507] Master 127.6.76.254:43155 requested a full tablet report, sending...
I20260709 12:30:57.665428  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:57.667966 19149 ts_manager.cc:194] Registered new tserver with Master: f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571)
I20260709 12:30:57.669148  6451 webserver.cc:533] Webserver started at http://127.6.76.194:34987/ using document root <none> and password file <none>
I20260709 12:30:57.670032  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:57.670316  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:57.670655  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:57.671011 19149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52410
I20260709 12:30:57.672722  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.678828  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:30:57.683647 19288 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.684783  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:57.685117  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.685441  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:57.728027  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:57.729393  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:57.731200  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:57.734318  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:57.734601  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.734884  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:57.735080  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.786252  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:42191
I20260709 12:30:57.786389 19350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:42191 every 8 connection(s)
I20260709 12:30:57.792183  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:57.801160 19354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:57.802701 19355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:57.816094 19351 heartbeater.cc:344] Connected to a master server at 127.6.76.254:43155
I20260709 12:30:57.817366 19351 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:57.820289 19351 heartbeater.cc:507] Master 127.6.76.254:43155 requested a full tablet report, sending...
I20260709 12:30:57.823904  6451 server_base.cc:1061] running on GCE node
W20260709 12:30:57.824326 19357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:57.824363 19149 ts_manager.cc:194] Registered new tserver with Master: c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:57.825664  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:57.825986  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:57.826191  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600257826156 us; error 0 us; skew 500 ppm
I20260709 12:30:57.826618 19149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52414
I20260709 12:30:57.826910  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:57.830732  6451 webserver.cc:533] Webserver started at http://127.6.76.195:37259/ using document root <none> and password file <none>
I20260709 12:30:57.831549  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:57.831780  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:57.832175  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:57.833673  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "b69d54f4112a408faf42e325596135e2"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.839807  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 12:30:57.844651 19362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.845832  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 12:30:57.846192  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "b69d54f4112a408faf42e325596135e2"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:30:57.846513  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:57.866950  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:57.868446  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:57.870150  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:57.873136  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:57.873386  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.873769  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:57.873931  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:57.927469  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:36049
I20260709 12:30:57.927582 19424 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:36049 every 8 connection(s)
I20260709 12:30:57.946269 19425 heartbeater.cc:344] Connected to a master server at 127.6.76.254:43155
I20260709 12:30:57.946913 19425 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:57.948011 19425 heartbeater.cc:507] Master 127.6.76.254:43155 requested a full tablet report, sending...
I20260709 12:30:57.950982 19149 ts_manager.cc:194] Registered new tserver with Master: b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:57.951557  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020244293s
I20260709 12:30:57.952843 19149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52426
I20260709 12:30:57.986375 19149 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52442:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:57.990432 19149 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:58.098929 19241 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3f77d6fb6346a2a0fed5fe259e8279 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:58.100255 19238 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7a78d5fd54c249abb720de77eb788 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:58.100885 19241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3f77d6fb6346a2a0fed5fe259e8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.100682 19237 tablet_service.cc:1511] Processing CreateTablet for tablet 65cf56359aff4b6ab82243a7dc2b8494 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:58.101966 19238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7a78d5fd54c249abb720de77eb788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.103494 19240 tablet_service.cc:1511] Processing CreateTablet for tablet bff002b6ff9f404b9bf3b21e75a8537f (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:58.104220 19237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65cf56359aff4b6ab82243a7dc2b8494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.105306 19240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff002b6ff9f404b9bf3b21e75a8537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.106690 19316 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3f77d6fb6346a2a0fed5fe259e8279 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:58.108759 19316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3f77d6fb6346a2a0fed5fe259e8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.110551 19315 tablet_service.cc:1511] Processing CreateTablet for tablet bff002b6ff9f404b9bf3b21e75a8537f (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:58.112452 19315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff002b6ff9f404b9bf3b21e75a8537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.116881 19314 tablet_service.cc:1511] Processing CreateTablet for tablet 784ade7ed9fe4c95a81b2d533b01272d (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:58.118479 19236 tablet_service.cc:1511] Processing CreateTablet for tablet b55ff906fe38479787c242960e080e0e (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:58.120540 19236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55ff906fe38479787c242960e080e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.120230 19311 tablet_service.cc:1511] Processing CreateTablet for tablet b55ff906fe38479787c242960e080e0e (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:58.122061 19311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55ff906fe38479787c242960e080e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.124835 19313 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7a78d5fd54c249abb720de77eb788 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:58.126125 19235 tablet_service.cc:1511] Processing CreateTablet for tablet ae52cb849a654eb59ad9a30d5eb843ee (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:58.126791 19313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7a78d5fd54c249abb720de77eb788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.127880 19235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae52cb849a654eb59ad9a30d5eb843ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.128700 19312 tablet_service.cc:1511] Processing CreateTablet for tablet 65cf56359aff4b6ab82243a7dc2b8494 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:58.132696 19310 tablet_service.cc:1511] Processing CreateTablet for tablet ae52cb849a654eb59ad9a30d5eb843ee (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:58.130367 19312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65cf56359aff4b6ab82243a7dc2b8494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.135644 19234 tablet_service.cc:1511] Processing CreateTablet for tablet 82d14df91ba94e3fb1cc318a62c82db6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:58.136545 19309 tablet_service.cc:1511] Processing CreateTablet for tablet 82d14df91ba94e3fb1cc318a62c82db6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:58.137449 19234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d14df91ba94e3fb1cc318a62c82db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.138347 19310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae52cb849a654eb59ad9a30d5eb843ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.139667 19314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784ade7ed9fe4c95a81b2d533b01272d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.142262 19233 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff593059d734273ad123fd6c11736e8 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:58.144299 19233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff593059d734273ad123fd6c11736e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.108901 19239 tablet_service.cc:1511] Processing CreateTablet for tablet 784ade7ed9fe4c95a81b2d533b01272d (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:58.148339 19239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784ade7ed9fe4c95a81b2d533b01272d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.147626 19309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d14df91ba94e3fb1cc318a62c82db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.168344 19308 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff593059d734273ad123fd6c11736e8 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:58.171766 19308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff593059d734273ad123fd6c11736e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.192482 19440 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.206226 19386 tablet_service.cc:1511] Processing CreateTablet for tablet 65cf56359aff4b6ab82243a7dc2b8494 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:58.208603 19390 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3f77d6fb6346a2a0fed5fe259e8279 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:58.210371 19390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3f77d6fb6346a2a0fed5fe259e8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.212654 19389 tablet_service.cc:1511] Processing CreateTablet for tablet bff002b6ff9f404b9bf3b21e75a8537f (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:58.214416 19389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff002b6ff9f404b9bf3b21e75a8537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.217083 19388 tablet_service.cc:1511] Processing CreateTablet for tablet 784ade7ed9fe4c95a81b2d533b01272d (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:58.220804 19387 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7a78d5fd54c249abb720de77eb788 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:58.224586 19385 tablet_service.cc:1511] Processing CreateTablet for tablet b55ff906fe38479787c242960e080e0e (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:58.230314 19441 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.230017 19384 tablet_service.cc:1511] Processing CreateTablet for tablet ae52cb849a654eb59ad9a30d5eb843ee (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:58.219120 19388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784ade7ed9fe4c95a81b2d533b01272d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.234844 19386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65cf56359aff4b6ab82243a7dc2b8494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.238909 19387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7a78d5fd54c249abb720de77eb788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.240334 19384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae52cb849a654eb59ad9a30d5eb843ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.244906 19385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55ff906fe38479787c242960e080e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.247742 19383 tablet_service.cc:1511] Processing CreateTablet for tablet 82d14df91ba94e3fb1cc318a62c82db6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:58.248468 19235 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c0a2defbb4fe5b81c158dad78eda6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:58.250025 19383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d14df91ba94e3fb1cc318a62c82db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.250840 19235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c0a2defbb4fe5b81c158dad78eda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.255426 19382 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff593059d734273ad123fd6c11736e8 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:58.257426 19382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff593059d734273ad123fd6c11736e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.275532 19316 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c0a2defbb4fe5b81c158dad78eda6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:58.277153 19316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c0a2defbb4fe5b81c158dad78eda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.281878 19381 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c0a2defbb4fe5b81c158dad78eda6 (DEFAULT_TABLE table=test-workload [id=85364560f1a2463daed35c294c1b6d2b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:58.283970 19381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c0a2defbb4fe5b81c158dad78eda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.346638 19441 tablet_bootstrap.cc:654] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.364481 19441 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.365202 19441 ts_tablet_manager.cc:1403] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.135s	user 0.008s	sys 0.016s
I20260709 12:30:58.367570 19440 tablet_bootstrap.cc:654] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.369211 19441 raft_consensus.cc:359] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.370262 19441 raft_consensus.cc:385] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.370699 19441 raft_consensus.cc:740] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.371774 19441 consensus_queue.cc:260] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.380288 19441 ts_tablet_manager.cc:1434] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 12:30:58.381296 19351 heartbeater.cc:499] Master 127.6.76.254:43155 was elected leader, sending a full tablet report...
I20260709 12:30:58.404528 19441 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.413796 19441 tablet_bootstrap.cc:654] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.427621 19445 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.428280 19445 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.442878 19445 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
W20260709 12:30:58.445372 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:58.427054 19441 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.448292 19441 ts_tablet_manager.cc:1403] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.003s
I20260709 12:30:58.451591 19440 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.456110 19440 ts_tablet_manager.cc:1403] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.264s	user 0.014s	sys 0.004s
I20260709 12:30:58.457078 19442 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.453922 19441 raft_consensus.cc:359] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.458822 19441 raft_consensus.cc:385] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.459371 19441 raft_consensus.cc:740] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.461606 19441 consensus_queue.cc:260] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.471076 19442 tablet_bootstrap.cc:654] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.486732 19441 ts_tablet_manager.cc:1434] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.038s	user 0.011s	sys 0.000s
I20260709 12:30:58.488359 19441 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.496503 19441 tablet_bootstrap.cc:654] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.474674 19440 raft_consensus.cc:359] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.497170 19440 raft_consensus.cc:385] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.497570 19440 raft_consensus.cc:740] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.498093 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.498587 19440 consensus_queue.cc:260] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:30:58.500903 19292 leader_election.cc:343] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.501281 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.502947 19442 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
W20260709 12:30:58.503504 19290 leader_election.cc:343] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.504142 19290 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:58.505400 19445 raft_consensus.cc:2749] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:58.503656 19442 ts_tablet_manager.cc:1403] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.003s
I20260709 12:30:58.509716 19276 heartbeater.cc:499] Master 127.6.76.254:43155 was elected leader, sending a full tablet report...
I20260709 12:30:58.510816 19440 ts_tablet_manager.cc:1434] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.054s	user 0.011s	sys 0.000s
I20260709 12:30:58.510383 19442 raft_consensus.cc:359] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.511497 19442 raft_consensus.cc:385] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.511888 19442 raft_consensus.cc:740] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.512171 19440 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.513103 19442 consensus_queue.cc:260] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.516675 19441 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.517479 19441 ts_tablet_manager.cc:1403] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.002s
I20260709 12:30:58.521488 19441 raft_consensus.cc:359] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.522518 19441 raft_consensus.cc:385] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.522935 19441 raft_consensus.cc:740] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.524108 19441 consensus_queue.cc:260] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.524811 19440 tablet_bootstrap.cc:654] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.529944 19441 ts_tablet_manager.cc:1434] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:30:58.531327 19441 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.540022 19441 tablet_bootstrap.cc:654] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.541826 19425 heartbeater.cc:499] Master 127.6.76.254:43155 was elected leader, sending a full tablet report...
W20260709 12:30:58.541883 19352 tablet.cc:2406] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:58.553946 19445 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.554634 19445 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.561223 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
W20260709 12:30:58.563064 19290 leader_election.cc:343] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.564139 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.560004 19445 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
W20260709 12:30:58.565846 19292 leader_election.cc:343] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.567334 19292 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:58.568742 19442 ts_tablet_manager.cc:1434] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.023s
I20260709 12:30:58.568893 19445 raft_consensus.cc:2749] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:58.570369 19442 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.570571 19441 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.571466 19441 ts_tablet_manager.cc:1403] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.014s
I20260709 12:30:58.571802 19440 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.572512 19440 ts_tablet_manager.cc:1403] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.028s
I20260709 12:30:58.575711 19441 raft_consensus.cc:359] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.575906 19440 raft_consensus.cc:359] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.576853 19441 raft_consensus.cc:385] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.577019 19440 raft_consensus.cc:385] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.577327 19441 raft_consensus.cc:740] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.577522 19440 raft_consensus.cc:740] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.578464 19441 consensus_queue.cc:260] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.578788 19440 consensus_queue.cc:260] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.580628 19442 tablet_bootstrap.cc:654] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.581825 19440 ts_tablet_manager.cc:1434] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:30:58.583288 19440 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.584591 19441 ts_tablet_manager.cc:1434] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 12:30:58.585916 19441 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.592020 19440 tablet_bootstrap.cc:654] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.594115 19442 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:58.594805 19442 ts_tablet_manager.cc:1403] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.000s
I20260709 12:30:58.595378 19441 tablet_bootstrap.cc:654] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.598673 19442 raft_consensus.cc:359] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.599901 19442 raft_consensus.cc:385] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.600291 19442 raft_consensus.cc:740] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.601490 19442 consensus_queue.cc:260] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.604084 19442 ts_tablet_manager.cc:1434] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:30:58.605216 19442 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.607095 19440 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.608214 19440 ts_tablet_manager.cc:1403] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.007s
I20260709 12:30:58.613897 19442 tablet_bootstrap.cc:654] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.613502 19440 raft_consensus.cc:359] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.615693 19440 raft_consensus.cc:385] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.616315 19440 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.617588 19440 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.618319 19441 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.619004 19441 ts_tablet_manager.cc:1403] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.013s
I20260709 12:30:58.622583 19441 raft_consensus.cc:359] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.623458 19441 raft_consensus.cc:385] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.623837 19441 raft_consensus.cc:740] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.624307 19440 ts_tablet_manager.cc:1434] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:30:58.624874 19441 consensus_queue.cc:260] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.625761 19440 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.628229 19441 ts_tablet_manager.cc:1434] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:58.629838 19441 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.636050 19440 tablet_bootstrap.cc:654] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.638019 19441 tablet_bootstrap.cc:654] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.642968 19442 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:58.645437 19442 ts_tablet_manager.cc:1403] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.029s
I20260709 12:30:58.647012 19441 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.648054 19441 ts_tablet_manager.cc:1403] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 12:30:58.649443 19442 raft_consensus.cc:359] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.651985 19441 raft_consensus.cc:359] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.655998 19442 raft_consensus.cc:385] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.656167 19441 raft_consensus.cc:385] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.656486 19442 raft_consensus.cc:740] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.656684 19441 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.657622 19442 consensus_queue.cc:260] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.657799 19441 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.662750 19441 ts_tablet_manager.cc:1434] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:30:58.663182 19440 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.663873 19440 ts_tablet_manager.cc:1403] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.004s
I20260709 12:30:58.664276 19441 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.667708 19442 ts_tablet_manager.cc:1434] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:30:58.667395 19440 raft_consensus.cc:359] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.668538 19440 raft_consensus.cc:385] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.668988 19440 raft_consensus.cc:740] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.669510 19442 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.670185 19440 consensus_queue.cc:260] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.673624 19440 ts_tablet_manager.cc:1434] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:58.674547 19441 tablet_bootstrap.cc:654] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.677572 19440 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.678934 19442 tablet_bootstrap.cc:654] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.692931 19440 tablet_bootstrap.cc:654] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.700713 19441 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.702073 19441 ts_tablet_manager.cc:1403] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.038s	user 0.022s	sys 0.013s
I20260709 12:30:58.703569 19442 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:58.704329 19442 ts_tablet_manager.cc:1403] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.012s
I20260709 12:30:58.706278 19441 raft_consensus.cc:359] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.707371 19441 raft_consensus.cc:385] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.707736 19441 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.708240 19440 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.707929 19442 raft_consensus.cc:359] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.708842 19441 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.708886 19440 ts_tablet_manager.cc:1403] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.004s
I20260709 12:30:58.708904 19442 raft_consensus.cc:385] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.709868 19442 raft_consensus.cc:740] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.710858 19442 consensus_queue.cc:260] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.711741 19441 ts_tablet_manager.cc:1434] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:30:58.713404 19442 ts_tablet_manager.cc:1434] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:30:58.713157 19440 raft_consensus.cc:359] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.714047 19440 raft_consensus.cc:385] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.713617 19441 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.714412 19440 raft_consensus.cc:740] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.714735 19442 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.715539 19440 consensus_queue.cc:260] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.718743 19440 ts_tablet_manager.cc:1434] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:30:58.720238 19440 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.723155 19442 tablet_bootstrap.cc:654] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.724229 19441 tablet_bootstrap.cc:654] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.728988 19440 tablet_bootstrap.cc:654] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.731693 19445 raft_consensus.cc:493] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.732353 19445 raft_consensus.cc:515] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.736562 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.737099 19452 raft_consensus.cc:493] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.737382 19251 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 0.
I20260709 12:30:58.737671 19452 raft_consensus.cc:515] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.739719 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.740489 19400 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 0.
I20260709 12:30:58.742019 19290 leader_election.cc:304] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.735014 19445 leader_election.cc:290] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.743724 19445 raft_consensus.cc:2804] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.744144 19445 raft_consensus.cc:493] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.744971 19445 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.755873 19452 leader_election.cc:290] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:58.763253 19451 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.763988 19451 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.767107 19451 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.784803 19468 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.788440 19468 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.792222 19452 raft_consensus.cc:493] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.795442 19452 raft_consensus.cc:515] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.799844 19452 leader_election.cc:290] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:58.802193 19469 raft_consensus.cc:493] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.809454 19451 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.810483 19451 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.813242 19451 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.820544 19468 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.819901 19442 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:58.821336 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.823271 19442 ts_tablet_manager.cc:1403] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.109s	user 0.044s	sys 0.049s
W20260709 12:30:58.825189 19290 leader_election.cc:343] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.803000 19469 raft_consensus.cc:515] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.829861 19469 leader_election.cc:290] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.832641 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.833959 19251 raft_consensus.cc:2468] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 0.
I20260709 12:30:58.835647 19445 raft_consensus.cc:515] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.836001 19442 raft_consensus.cc:359] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.835951 19290 leader_election.cc:304] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.839650 19442 raft_consensus.cc:385] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.841836 19469 raft_consensus.cc:2804] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.842276 19469 raft_consensus.cc:493] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.842675 19469 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.842929 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:30:58.844847 19445 leader_election.cc:290] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.844089 19251 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.845554 19441 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:58.846217 19441 ts_tablet_manager.cc:1403] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.133s	user 0.008s	sys 0.017s
I20260709 12:30:58.841863 19442 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.851022 19250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.852445 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.853348 19249 raft_consensus.cc:2468] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 0.
I20260709 12:30:58.853739 19441 raft_consensus.cc:359] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.854586 19441 raft_consensus.cc:385] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.854943 19441 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.854791 19469 raft_consensus.cc:515] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.855281 19364 leader_election.cc:304] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.856462 19452 raft_consensus.cc:2804] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.856207 19441 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.856921 19452 raft_consensus.cc:493] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.857515 19469 leader_election.cc:290] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.857905 19452 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.858518 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:30:58.859375 19249 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.863999 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.864262 19325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.865041 19326 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:58.865271 19325 raft_consensus.cc:2468] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:58.869063 19217 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.870644 19451 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.871201 19451 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.871694 19451 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.872022 19249 raft_consensus.cc:2468] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:30:58.873786 19290 leader_election.cc:304] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.867949 19442 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.875064 19469 raft_consensus.cc:2804] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:58.875687 19469 raft_consensus.cc:697] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:30:58.875771 19217 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.876814 19469 consensus_queue.cc:237] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.881803 19440 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:58.876317 19251 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:30:58.882630 19440 ts_tablet_manager.cc:1403] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.163s	user 0.051s	sys 0.068s
I20260709 12:30:58.884017 19290 leader_election.cc:304] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.885258 19441 ts_tablet_manager.cc:1434] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.001s
I20260709 12:30:58.886713 19468 raft_consensus.cc:2804] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:58.887584 19440 raft_consensus.cc:359] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.887652 19468 raft_consensus.cc:697] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:30:58.889009 19440 raft_consensus.cc:385] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.889524 19440 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.890043 19451 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.890148 19468 consensus_queue.cc:237] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.890686 19440 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.898602 19452 raft_consensus.cc:515] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.901587 19452 leader_election.cc:290] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:58.902259 19452 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.904263 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:30:58.905537 19251 raft_consensus.cc:2393] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b69d54f4112a408faf42e325596135e2 in current term 1: Already voted for candidate c4396a52dbda4132bfe4d3ad8b404adb in this term.
I20260709 12:30:58.905537 19477 raft_consensus.cc:2804] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.907984 19477 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.908417 19477 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.909108 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.911526 19441 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:58.916576 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.917797 19400 raft_consensus.cc:2393] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4396a52dbda4132bfe4d3ad8b404adb in current term 1: Already voted for candidate b69d54f4112a408faf42e325596135e2 in this term.
I20260709 12:30:58.918473 19475 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.919131 19475 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.921181 19400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:58.922224 19475 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:58.922808 19400 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.932225 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:58.933209 19326 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.935067 19442 ts_tablet_manager.cc:1434] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.111s	user 0.031s	sys 0.035s
W20260709 12:30:58.935624 19292 leader_election.cc:343] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.936148 19292 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:58.936467 19442 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:58.932510 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:58.937891 19399 raft_consensus.cc:2393] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c4396a52dbda4132bfe4d3ad8b404adb in current term 1: Already voted for candidate b69d54f4112a408faf42e325596135e2 in this term.
I20260709 12:30:58.939459 19445 raft_consensus.cc:2749] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:58.902740 19452 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.944785 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.949409 19325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.934125 19451 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.949512 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:58.958591 19397 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.958137 19477 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.963989 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:58.964848 19324 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.965117 19251 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 0.
I20260709 12:30:58.965706 19326 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:58.971776 19440 ts_tablet_manager.cc:1434] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.021s
I20260709 12:30:58.973084 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:58.973860 19322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.974129 19440 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:58.974764 19322 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 0.
I20260709 12:30:58.948309 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.972944 19323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:58.975653 19364 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.947722 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:58.976665 19325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.976953 19476 raft_consensus.cc:2804] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.977474 19476 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.977851 19476 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.977936 19481 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:58.978549 19481 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:58.945483 19442 tablet_bootstrap.cc:654] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.954538 19452 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:58.984220 19441 tablet_bootstrap.cc:654] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.985365 19325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:58.987310 19325 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:58.989285 19217 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:58.990633 19479 raft_consensus.cc:2804] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:58.991362 19477 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:58.991403 19479 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:58.992619 19479 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:58.996352 19364 leader_election.cc:343] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:58.943045 19482 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.000672 19482 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.008560 19217 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.006820 19481 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.005123 19324 raft_consensus.cc:2468] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.013407 19477 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.013942 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.009408 19400 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:30:59.019639 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.016561 19477 raft_consensus.cc:697] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
W20260709 12:30:59.022085 19217 leader_election.cc:343] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.021373 19477 consensus_queue.cc:237] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.025902 19217 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.026499 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.027467 19324 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.031697 19451 raft_consensus.cc:2804] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.031952 19442 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:59.019712 19366 leader_election.cc:304] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.035584 19146 catalog_manager.cc:5697] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 0 to 1, leader changed from <none> to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 1 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.036940 19452 raft_consensus.cc:2749] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.038185 19146 catalog_manager.cc:5697] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 0 to 1, leader changed from <none> to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 1 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.039857 19366 leader_election.cc:304] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.021265 19482 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.041329 19452 raft_consensus.cc:2749] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.035549 19442 ts_tablet_manager.cc:1403] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.011s
W20260709 12:30:59.043704 19366 leader_election.cc:343] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.044365 19366 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.030184 19476 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.045384 19452 raft_consensus.cc:2749] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.046438 19442 raft_consensus.cc:359] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.047415 19442 raft_consensus.cc:385] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.047765 19442 raft_consensus.cc:740] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.048731 19442 consensus_queue.cc:260] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.047811 19217 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.052415 19440 tablet_bootstrap.cc:654] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:59.052639 19217 leader_election.cc:343] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.063689 19442 ts_tablet_manager.cc:1434] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:30:59.064402 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:30:59.053646 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.052484 19482 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.072926 19482 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.064656 19441 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:59.068996 19476 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:59.073559 19482 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.074188 19441 ts_tablet_manager.cc:1403] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.012s
I20260709 12:30:59.075454 19451 raft_consensus.cc:697] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.070043 19479 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.076820 19249 raft_consensus.cc:2393] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b69d54f4112a408faf42e325596135e2 in current term 1: Already voted for candidate f41fa709b40c4b409a9b8cb61077dd36 in this term.
I20260709 12:30:59.073302 19442 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:59.076730 19451 consensus_queue.cc:237] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.073094 19399 raft_consensus.cc:2393] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f41fa709b40c4b409a9b8cb61077dd36 in current term 1: Already voted for candidate b69d54f4112a408faf42e325596135e2 in this term.
I20260709 12:30:59.080703 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.078693 19441 raft_consensus.cc:359] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.082362 19441 raft_consensus.cc:385] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.082835 19441 raft_consensus.cc:740] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.068279 19397 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.089675 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.093627 19441 consensus_queue.cc:260] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.095160 19324 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.095341 19479 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.099115 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.100360 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.101462 19399 raft_consensus.cc:2393] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f41fa709b40c4b409a9b8cb61077dd36 in current term 1: Already voted for candidate b69d54f4112a408faf42e325596135e2 in this term.
I20260709 12:30:59.108973 19442 tablet_bootstrap.cc:654] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.110919 19364 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.111687 19441 ts_tablet_manager.cc:1434] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.004s
W20260709 12:30:59.099028 19217 leader_election.cc:343] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.112462 19476 raft_consensus.cc:2749] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.112960 19441 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: Bootstrap starting.
I20260709 12:30:59.114398 19482 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.118284 19476 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.119205 19440 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:59.118952 19476 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.119907 19440 ts_tablet_manager.cc:1403] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.146s	user 0.016s	sys 0.013s
I20260709 12:30:59.121639 19476 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:59.123839 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:59.124411 19441 tablet_bootstrap.cc:654] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.127553 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.128656 19326 raft_consensus.cc:2393] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b69d54f4112a408faf42e325596135e2 in current term 1: Already voted for candidate f41fa709b40c4b409a9b8cb61077dd36 in this term.
I20260709 12:30:59.131119 19366 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.132059 19482 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.132562 19476 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.133811 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.128070 19440 raft_consensus.cc:359] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.124882 19324 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.134856 19326 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.137645 19440 raft_consensus.cc:385] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.140825 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.141409 19442 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:59.144881 19440 raft_consensus.cc:740] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.149124 19442 ts_tablet_manager.cc:1403] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.076s	user 0.019s	sys 0.012s
I20260709 12:30:59.153492 19217 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: b69d54f4112a408faf42e325596135e2
I20260709 12:30:59.159373 19441 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: No bootstrap required, opened a new log
I20260709 12:30:59.160239 19441 ts_tablet_manager.cc:1403] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.019s
W20260709 12:30:59.162400 19217 leader_election.cc:343] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.163112 19442 raft_consensus.cc:359] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.164108 19442 raft_consensus.cc:385] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.163759 19441 raft_consensus.cc:359] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.164470 19442 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.164697 19441 raft_consensus.cc:385] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.166409 19326 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.167514 19451 raft_consensus.cc:2804] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.172066 19451 raft_consensus.cc:697] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.172232 19442 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.171618 19441 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.173125 19451 consensus_queue.cc:237] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.174397 19441 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.177125 19441 ts_tablet_manager.cc:1434] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:30:59.166968 19147 catalog_manager.cc:5697] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.179559 19442 ts_tablet_manager.cc:1434] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.005s
I20260709 12:30:59.179028 19147 catalog_manager.cc:5697] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.168882 19440 consensus_queue.cc:260] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.180941 19442 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:59.184234 19217 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: b69d54f4112a408faf42e325596135e2
I20260709 12:30:59.185273 19452 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.185962 19452 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.188683 19479 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.189392 19479 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.190382 19479 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.191040 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:59.182948 19440 ts_tablet_manager.cc:1434] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.063s	user 0.003s	sys 0.005s
I20260709 12:30:59.170894 19476 raft_consensus.cc:2749] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.200371 19440 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:59.204293 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.201604 19452 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:59.205708 19326 raft_consensus.cc:2393] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b69d54f4112a408faf42e325596135e2 in current term 1: Already voted for candidate f41fa709b40c4b409a9b8cb61077dd36 in this term.
I20260709 12:30:59.208438 19366 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.211324 19452 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.214811 19440 tablet_bootstrap.cc:654] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.221652 19442 tablet_bootstrap.cc:654] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.225062 19452 raft_consensus.cc:2749] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.229007 19149 catalog_manager.cc:5697] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.229652 19440 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:59.229678 19442 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:59.230471 19442 ts_tablet_manager.cc:1403] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.002s
I20260709 12:30:59.230471 19440 ts_tablet_manager.cc:1403] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.007s
I20260709 12:30:59.230911 19149 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.233826 19440 raft_consensus.cc:359] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.233981 19442 raft_consensus.cc:359] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.235124 19440 raft_consensus.cc:385] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.235275 19442 raft_consensus.cc:385] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.235603 19440 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.235694 19442 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.236557 19442 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.236557 19440 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.239353 19442 ts_tablet_manager.cc:1434] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:30:59.239652 19440 ts_tablet_manager.cc:1434] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:59.240629 19440 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:59.245589 19442 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:59.247684 19440 tablet_bootstrap.cc:654] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.254520 19442 tablet_bootstrap.cc:654] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.257867 19440 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:59.258946 19440 ts_tablet_manager.cc:1403] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.005s
I20260709 12:30:59.262166 19442 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:59.262878 19442 ts_tablet_manager.cc:1403] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 12:30:59.262427 19440 raft_consensus.cc:359] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.263284 19440 raft_consensus.cc:385] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.263619 19440 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.264457 19440 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.266211 19469 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.267000 19440 ts_tablet_manager.cc:1434] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:59.266634 19442 raft_consensus.cc:359] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.266956 19469 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.267627 19442 raft_consensus.cc:385] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.268117 19442 raft_consensus.cc:740] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.268683 19440 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:30:59.269115 19442 consensus_queue.cc:260] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.272097 19469 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.273769 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
I20260709 12:30:59.274652 19249 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 0.
I20260709 12:30:59.275146 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.276810 19440 tablet_bootstrap.cc:654] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.278043 19442 ts_tablet_manager.cc:1434] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:30:59.278404 19290 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.279433 19479 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.282855 19479 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.283564 19440 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: No bootstrap required, opened a new log
I20260709 12:30:59.284451 19440 ts_tablet_manager.cc:1403] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:30:59.285578 19479 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.279495 19442 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: Bootstrap starting.
I20260709 12:30:59.287673 19469 raft_consensus.cc:2804] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.288439 19469 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.288120 19440 raft_consensus.cc:359] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.288674 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.289031 19469 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.289119 19440 raft_consensus.cc:385] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.289726 19440 raft_consensus.cc:740] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.289784 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.290603 19326 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.290879 19440 consensus_queue.cc:260] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:30:59.280189 19292 leader_election.cc:343] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.293864 19440 ts_tablet_manager.cc:1434] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:30:59.294055 19442 tablet_bootstrap.cc:654] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.295583 19399 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.297003 19217 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.298576 19479 raft_consensus.cc:2804] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.299144 19479 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.299552 19479 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.300081 19468 consensus_queue.cc:1048] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.314718 19442 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: No bootstrap required, opened a new log
I20260709 12:30:59.315538 19442 ts_tablet_manager.cc:1403] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.002s
I20260709 12:30:59.320209 19469 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.323289 19469 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.324331 19249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:30:59.325191 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.325501 19479 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.326277 19442 raft_consensus.cc:359] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.328743 19479 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.329396 19442 raft_consensus.cc:385] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.329841 19442 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.325238 19249 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.330240 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.331125 19326 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.330732 19442 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.331874 19452 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.332412 19452 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.333349 19442 ts_tablet_manager.cc:1434] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:30:59.334532 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.335286 19399 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.336208 19452 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:30:59.338296 19251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36" is_pre_election: true
W20260709 12:30:59.327391 19292 leader_election.cc:343] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:59.340653 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.341601 19324 raft_consensus.cc:2393] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b69d54f4112a408faf42e325596135e2 in current term 1: Already voted for candidate c4396a52dbda4132bfe4d3ad8b404adb in this term.
I20260709 12:30:59.343611 19366 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:30:59.345343 19452 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.348028 19249 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:30:59.353063 19290 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: b69d54f4112a408faf42e325596135e2
I20260709 12:30:59.353535 19399 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.354502 19469 raft_consensus.cc:2804] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.355185 19217 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.356338 19479 raft_consensus.cc:2804] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.356995 19479 raft_consensus.cc:697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.357990 19479 consensus_queue.cc:237] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.359567 19452 raft_consensus.cc:2749] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:59.364256 19468 consensus_queue.cc:1048] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.355562 19469 raft_consensus.cc:697] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:30:59.368891 19469 consensus_queue.cc:237] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.375692 19149 catalog_manager.cc:5697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.378822 19326 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.383931 19468 consensus_queue.cc:1048] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.399641 19468 consensus_queue.cc:1048] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.408775 19147 catalog_manager.cc:5697] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 0 to 1, leader changed from <none> to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 1 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:59.449162 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:59.474365 19479 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.474965 19479 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.476899 19479 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.477938 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.478470 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.478811 19326 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.479426 19398 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.480373 19217 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.481200 19479 raft_consensus.cc:2804] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.481619 19479 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.482009 19479 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.484475 19451 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.485214 19451 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.487785 19451 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.488862 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:30:59.489269 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:30:59.489651 19326 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.489368 19479 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.490340 19398 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 0.
I20260709 12:30:59.491366 19217 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.492290 19451 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.492400 19479 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.492789 19451 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.493270 19451 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.493837 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.494566 19326 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.494589 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.495394 19398 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.501606 19326 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.502588 19451 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.503379 19217 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.504446 19479 raft_consensus.cc:2804] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.504992 19479 raft_consensus.cc:697] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.505440 19398 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.506023 19479 consensus_queue.cc:237] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.506605 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:30:59.507396 19451 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Requested vote from peers c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:30:59.507490 19326 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.507541 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "f41fa709b40c4b409a9b8cb61077dd36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:30:59.508795 19399 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.517963 19326 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.517666 19146 catalog_manager.cc:5697] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.518635 19399 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41fa709b40c4b409a9b8cb61077dd36 in term 1.
I20260709 12:30:59.519596 19217 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:30:59.520597 19479 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.521184 19479 raft_consensus.cc:697] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Becoming Leader. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Running, Role: LEADER
I20260709 12:30:59.522025 19479 consensus_queue.cc:237] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:30:59.535931 19146 catalog_manager.cc:5697] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f41fa709b40c4b409a9b8cb61077dd36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.580803 19451 consensus_queue.cc:1048] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.584764 19451 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.601495 19451 consensus_queue.cc:1048] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.605124 19479 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.651147 19479 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.673380 19482 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.673429 19479 consensus_queue.cc:1048] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.691486 19482 consensus_queue.cc:1048] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:59.806905 19511 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.821669 19469 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.825562  6451 ts_manager.cc:295] Set tserver state for f41fa709b40c4b409a9b8cb61077dd36 to MAINTENANCE_MODE
I20260709 12:30:59.827090  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
I20260709 12:30:59.871047  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:59.871991  6451 tablet_replica.cc:333] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.872745  6451 raft_consensus.cc:2243] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:59.873307  6451 raft_consensus.cc:2272] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.876214  6451 tablet_replica.cc:333] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.876930  6451 raft_consensus.cc:2243] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.877653  6451 pending_rounds.cc:70] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Trying to abort 1 pending ops.
I20260709 12:30:59.877861  6451 pending_rounds.cc:77] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626663012630528 op_type: NO_OP noop_request { }
I20260709 12:30:59.878177  6451 raft_consensus.cc:2889] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:59.878545  6451 raft_consensus.cc:2272] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.881233  6451 tablet_replica.cc:333] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.881978  6451 raft_consensus.cc:2243] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:59.882504  6451 raft_consensus.cc:2272] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.885515  6451 tablet_replica.cc:333] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.886379  6451 raft_consensus.cc:2243] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.887585  6451 raft_consensus.cc:2272] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.890599  6451 tablet_replica.cc:333] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.891360  6451 raft_consensus.cc:2243] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.892225  6451 pending_rounds.cc:70] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Trying to abort 1 pending ops.
I20260709 12:30:59.892498  6451 pending_rounds.cc:77] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626662948896768 op_type: NO_OP noop_request { }
I20260709 12:30:59.892799  6451 raft_consensus.cc:2889] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:59.893157  6451 raft_consensus.cc:2272] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.895834 19482 consensus_queue.cc:1048] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.896028  6451 tablet_replica.cc:333] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.897001  6451 raft_consensus.cc:2243] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.898109  6451 raft_consensus.cc:2272] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.901600  6451 tablet_replica.cc:333] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.902360  6451 raft_consensus.cc:2243] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.903399  6451 raft_consensus.cc:2272] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.906884  6451 tablet_replica.cc:333] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.907788  6451 raft_consensus.cc:2243] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.909001  6451 raft_consensus.cc:2272] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.912014 19451 consensus_queue.cc:1048] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:59.914019  6451 tablet_replica.cc:333] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.915028  6451 raft_consensus.cc:2243] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:59.916007  6451 raft_consensus.cc:2272] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:59.919991  6451 tablet_replica.cc:333] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: stopping tablet replica
I20260709 12:30:59.920737  6451 raft_consensus.cc:2243] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:59.921248  6451 raft_consensus.cc:2272] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:59.934544 19290 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 300 similar messages]
W20260709 12:30:59.938403 19290 consensus_peers.cc:597] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:59.958655  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
W20260709 12:31:00.258651 19290 consensus_peers.cc:597] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:00.262095 19290 consensus_peers.cc:597] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:00.450449 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:01.098073 19529 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.098757 19529 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.100850 19529 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.100842 19445 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:01.102110 19529 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.102334 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:31:01.102675 19529 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.105063 19529 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.104744 19445 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.106575 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:31:01.108429 19511 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
W20260709 12:31:01.108597 19364 leader_election.cc:336] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.109203 19364 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.109165 19511 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.110119 19469 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.110595 19529 raft_consensus.cc:2749] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:01.110913 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.111934 19398 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:31:01.112504 19511 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.113402 19445 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.113884 19292 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: 
I20260709 12:31:01.114984 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.110767 19469 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:31:01.115031 19290 leader_election.cc:336] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.115962 19398 raft_consensus.cc:2468] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
W20260709 12:31:01.116680 19364 leader_election.cc:336] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.117300 19364 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b69d54f4112a408faf42e325596135e2; no voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.117677 19469 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.118265 19529 raft_consensus.cc:2749] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:01.115041 19445 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:01.119103 19445 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:01.119535 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.118440 19292 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: 
I20260709 12:31:01.119565 19445 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.120584 19398 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
W20260709 12:31:01.120857 19290 leader_election.cc:336] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.121790 19469 raft_consensus.cc:2804] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:01.122303 19292 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: 
I20260709 12:31:01.122325 19469 raft_consensus.cc:493] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.123533 19511 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:01.123864 19469 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.124040 19511 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
W20260709 12:31:01.124496 19290 leader_election.cc:336] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.124820 19511 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.131394 19511 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.131327 19445 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.133188 19469 raft_consensus.cc:515] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.134313 19445 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.136198 19469 leader_election.cc:290] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.136904 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae52cb849a654eb59ad9a30d5eb843ee" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:01.137326 19511 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.136550 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:01.137784 19397 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.137013 19398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:01.138231 19399 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.138628 19398 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:31:01.141520 19290 leader_election.cc:336] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
W20260709 12:31:01.142385 19290 leader_election.cc:336] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
W20260709 12:31:01.143023 19290 leader_election.cc:336] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.147063 19398 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 2.
I20260709 12:31:01.147275 19397 raft_consensus.cc:2468] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 2.
I20260709 12:31:01.147534 19399 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 2.
I20260709 12:31:01.148761 19292 leader_election.cc:304] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.149602 19511 raft_consensus.cc:2804] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.150137 19511 raft_consensus.cc:697] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:01.150192 19292 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.151134 19469 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.151142 19511 consensus_queue.cc:237] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.151701 19292 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.151837 19469 raft_consensus.cc:697] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:01.152634 19445 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.152951 19469 consensus_queue.cc:237] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.158565 19445 raft_consensus.cc:697] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:01.158914 19533 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:01.159500 19533 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.159925 19445 consensus_queue.cc:237] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.162007 19533 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
W20260709 12:31:01.169548 19290 leader_election.cc:336] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.170568 19445 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.171381 19445 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.172705 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.173592 19397 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 1.
I20260709 12:31:01.174721 19445 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.176195 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.176239 19292 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.178100 19445 raft_consensus.cc:2804] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:01.178565 19445 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:01.179207 19529 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.179983 19445 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.180186 19529 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.183058 19529 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
W20260709 12:31:01.179307 19290 leader_election.cc:336] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.183593 19149 catalog_manager.cc:5697] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 2 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:31:01.187458 19364 leader_election.cc:336] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.187932 19292 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.188591 19511 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.188973 19532 raft_consensus.cc:2749] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:01.189291 19511 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.191694 19511 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.192058 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:31:01.191727 19445 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.192863 19326 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 1.
I20260709 12:31:01.194454 19445 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.195092 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
W20260709 12:31:01.196286 19290 leader_election.cc:336] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.196771 19529 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.197873 19366 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.198139 19529 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.200313 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2"
W20260709 12:31:01.200865 19290 leader_election.cc:336] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.201297 19397 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.200924 19529 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.204418 19529 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:31:01.207245 19364 leader_election.cc:336] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.204864 19529 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.208907 19529 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.211767 19292 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.214926 19445 raft_consensus.cc:2749] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:01.212790 19397 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 2.
I20260709 12:31:01.216987 19292 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.217056 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:31:01.218120 19326 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 1.
I20260709 12:31:01.218729 19445 raft_consensus.cc:2804] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.219414 19445 raft_consensus.cc:697] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:01.219744 19366 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.220404 19445 consensus_queue.cc:237] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.220806 19529 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.221195 19538 raft_consensus.cc:2804] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:01.222353 19538 raft_consensus.cc:493] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.222901 19538 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:31:01.229487 19364 leader_election.cc:336] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.230818 19529 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.233107 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:01.233915 19326 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.235867 19538 raft_consensus.cc:515] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.238479 19538 leader_election.cc:290] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.240396 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7a78d5fd54c249abb720de77eb788" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:01.241400 19324 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.240407 19149 catalog_manager.cc:5697] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 2 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:01.242537 19149 catalog_manager.cc:5697] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 2 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:01.243913 19326 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
W20260709 12:31:01.244187 19364 leader_election.cc:336] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.244557 19149 catalog_manager.cc:5697] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 2 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:01.245639 19366 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.246676 19538 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.247258 19538 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:01.248167 19538 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.250834 19324 raft_consensus.cc:2468] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:01.252385 19366 leader_election.cc:304] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.255438 19538 raft_consensus.cc:2804] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.256364 19538 raft_consensus.cc:697] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:01.257421 19538 consensus_queue.cc:237] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.260555 19146 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:01.276899 19146 catalog_manager.cc:5697] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:01.409735 19445 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.410306 19445 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.412335 19445 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:01.413838 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b69d54f4112a408faf42e325596135e2" is_pre_election: true
I20260709 12:31:01.416554 19538 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.417250 19538 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:31:01.417797 19290 leader_election.cc:336] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.418409 19290 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb; no voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.419143 19445 raft_consensus.cc:2749] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:01.419894 19538 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.421032 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" is_pre_election: true
I20260709 12:31:01.421864 19324 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 1.
I20260709 12:31:01.423069 19366 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: 
I20260709 12:31:01.423986 19538 raft_consensus.cc:2804] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:31:01.424301 19364 leader_election.cc:336] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.424547 19538 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f41fa709b40c4b409a9b8cb61077dd36)
I20260709 12:31:01.424944 19538 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:01.430972 19538 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:01.433145 19538 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:01.434860 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:01.435731 19324 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:31:01.438516 19364 leader_election.cc:336] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111)
I20260709 12:31:01.442945 19324 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:01.444464 19366 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: f41fa709b40c4b409a9b8cb61077dd36
I20260709 12:31:01.445271 19538 raft_consensus.cc:2804] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:01.445775 19538 raft_consensus.cc:697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:01.446664 19538 consensus_queue.cc:237] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:31:01.452075 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:01.456903 19146 catalog_manager.cc:5697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:01.593588 19397 raft_consensus.cc:1275] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:31:01.595408 19445 consensus_queue.cc:1048] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:01.601991 19399 raft_consensus.cc:1275] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:01.604343 19445 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:31:01.617413 19290 consensus_peers.cc:597] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:01.621043 19399 raft_consensus.cc:1275] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:01.623518 19445 consensus_queue.cc:1048] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:31:01.629607 19290 consensus_peers.cc:597] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:01.644266 19290 consensus_peers.cc:597] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:01.662459 19399 raft_consensus.cc:1275] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:31:01.664287 19445 consensus_queue.cc:1048] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:31:01.683637 19290 consensus_peers.cc:597] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:01.760573 19324 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:01.762677 19538 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:01.843540 19324 raft_consensus.cc:1275] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:01.845135 19553 consensus_queue.cc:1048] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:01.916416 19324 raft_consensus.cc:1275] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:01.918455 19553 consensus_queue.cc:1048] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:31:02.373270 19290 consensus_peers.cc:597] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:02.453429 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:31:02.720646 19290 consensus_peers.cc:597] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:02.773944 19364 consensus_peers.cc:597] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:02.777676 19364 consensus_peers.cc:597] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:02.778214 19364 consensus_peers.cc:597] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:02.781908 19290 consensus_peers.cc:597] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:03.454942 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:31:04.070363 19290 consensus_peers.cc:597] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:04.228057 19290 consensus_peers.cc:597] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:04.278081 19290 consensus_peers.cc:597] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:04.354662 19290 consensus_peers.cc:597] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:04.456236 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:31:04.726223 19364 consensus_peers.cc:597] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:04.796051 19290 consensus_peers.cc:597] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:31:04.973886 19290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111) [suppressed 92 similar messages]
W20260709 12:31:04.979676 19290 consensus_peers.cc:597] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:31:04.983688  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:04.991484 19561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:04.991991 19562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:04.996743  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:04.996587 19564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:04.998184  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:04.998440  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:04.998589  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600264998577 us; error 0 us; skew 500 ppm
I20260709 12:31:04.999138  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:05.002117  6451 webserver.cc:533] Webserver started at http://127.6.76.193:41329/ using document root <none> and password file <none>
I20260709 12:31:05.002720  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:05.002912  6451 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:31:05.007369  6451 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:31:05.012439 19569 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:05.013790  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.003s
I20260709 12:31:05.014256  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "f41fa709b40c4b409a9b8cb61077dd36"
format_stamp: "Formatted at 2026-07-09 12:30:57 on dist-test-slave-n447"
I20260709 12:31:05.014659  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:05.027107  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:05.028486  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:05.030130  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:05.034623 19576 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:31:05.057477 19364 consensus_peers.cc:597] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:31:05.088651 19364 consensus_peers.cc:597] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:31:05.109858  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:31:05.110244  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.077s	user 0.003s	sys 0.000s
I20260709 12:31:05.110515  6451 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:31:05.117130 19576 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.136519 19576 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.137720 19576 tablet_bootstrap.cc:492] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.138530 19576 ts_tablet_manager.cc:1403] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:31:05.141517 19576 raft_consensus.cc:359] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.142556 19576 raft_consensus.cc:740] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.143491 19576 consensus_queue.cc:260] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.146621 19576 ts_tablet_manager.cc:1434] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.004s
I20260709 12:31:05.147701 19576 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.165654 19576 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:31:05.166710 19576 tablet_bootstrap.cc:492] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.167570 19576 ts_tablet_manager.cc:1403] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 12:31:05.170555 19576 raft_consensus.cc:359] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.171612 19576 raft_consensus.cc:740] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.172397 19576 consensus_queue.cc:260] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.174149 19576 ts_tablet_manager.cc:1434] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:31:05.175068 19576 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.178447  6451 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:31:05.178889  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.068s	user 0.065s	sys 0.000s
I20260709 12:31:05.197741 19576 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.199090 19576 tablet_bootstrap.cc:492] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.200076 19576 ts_tablet_manager.cc:1403] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:31:05.203527 19576 raft_consensus.cc:359] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.204262 19576 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.205065 19576 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.207509 19576 ts_tablet_manager.cc:1434] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:31:05.208462 19576 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.222870 19576 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.224327 19576 tablet_bootstrap.cc:492] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.225090 19576 ts_tablet_manager.cc:1403] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:31:05.227626 19576 raft_consensus.cc:359] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.228237 19576 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.228978 19576 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.230736 19576 ts_tablet_manager.cc:1434] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:31:05.231717 19576 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.250671 19576 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.251909 19576 tablet_bootstrap.cc:492] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.252745 19576 ts_tablet_manager.cc:1403] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:31:05.256749  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:46571
I20260709 12:31:05.256413 19576 raft_consensus.cc:359] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.256994 19639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:46571 every 8 connection(s)
I20260709 12:31:05.257453 19576 raft_consensus.cc:740] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.258371 19576 consensus_queue.cc:260] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.260159 19576 ts_tablet_manager.cc:1434] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:31:05.261338 19576 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.263171  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:05.287178 19576 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:31:05.288337 19576 tablet_bootstrap.cc:492] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.289142 19576 ts_tablet_manager.cc:1403] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.008s
I20260709 12:31:05.289819 19640 heartbeater.cc:344] Connected to a master server at 127.6.76.254:43155
I20260709 12:31:05.290586 19640 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:05.291814 19640 heartbeater.cc:507] Master 127.6.76.254:43155 requested a full tablet report, sending...
I20260709 12:31:05.292738 19576 raft_consensus.cc:359] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.293792 19576 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.294549 19576 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.297139 19576 ts_tablet_manager.cc:1434] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:31:05.299425 19576 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.299364 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:05.300580 19324 raft_consensus.cc:606] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:05.304032 19149 ts_manager.cc:194] Re-registered known tserver with Master: f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571)
I20260709 12:31:05.304124 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:05.304790 19324 raft_consensus.cc:606] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:05.307853 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:05.308516 19324 raft_consensus.cc:606] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:05.309926  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:31:05.310580  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:05.311416  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:05.317920 19149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39340
I20260709 12:31:05.318639 19576 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.320112 19576 tablet_bootstrap.cc:492] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.321007 19576 ts_tablet_manager.cc:1403] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 12:31:05.322739 19640 heartbeater.cc:499] Master 127.6.76.254:43155 was elected leader, sending a full tablet report...
I20260709 12:31:05.323734 19576 raft_consensus.cc:359] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.324441 19576 raft_consensus.cc:740] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.325157 19576 consensus_queue.cc:260] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.327005 19576 ts_tablet_manager.cc:1434] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:31:05.327901 19576 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.345661 19576 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.346959 19576 tablet_bootstrap.cc:492] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.347924 19576 ts_tablet_manager.cc:1403] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:31:05.350566 19576 raft_consensus.cc:359] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.351190 19576 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.351979 19576 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.354167 19576 ts_tablet_manager.cc:1434] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 12:31:05.355106 19576 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.362311 19613 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.363282 19614 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.370949 19613 pending_rounds.cc:85] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Aborting all ops after (but not including) 0
I20260709 12:31:05.371479 19613 pending_rounds.cc:107] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:31:05.371819 19613 raft_consensus.cc:2889] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:31:05.373208 19576 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.374521 19576 tablet_bootstrap.cc:492] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.375640 19576 ts_tablet_manager.cc:1403] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.006s
I20260709 12:31:05.380208 19576 raft_consensus.cc:359] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.381417 19576 raft_consensus.cc:740] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.382339 19576 consensus_queue.cc:260] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.388087 19576 ts_tablet_manager.cc:1434] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 12:31:05.388943 19576 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap starting.
I20260709 12:31:05.404477 19576 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:31:05.405673 19576 tablet_bootstrap.cc:492] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Bootstrap complete.
I20260709 12:31:05.406534 19576 ts_tablet_manager.cc:1403] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 12:31:05.408975 19576 raft_consensus.cc:359] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.409881 19576 raft_consensus.cc:740] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41fa709b40c4b409a9b8cb61077dd36, State: Initialized, Role: FOLLOWER
I20260709 12:31:05.410787 19576 consensus_queue.cc:260] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.413146 19576 ts_tablet_manager.cc:1434] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
W20260709 12:31:05.457604 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:05.482566 19614 raft_consensus.cc:3060] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.518599 19557 raft_consensus.cc:993] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.519311 19657 raft_consensus.cc:1081] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.521412 19399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:05.522097 19399 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:05.522460 19399 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.531294 19399 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.533324 19399 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:05.535082 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:05.536127 19324 raft_consensus.cc:3055] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:05.536556 19324 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:05.537714 19324 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.539189 19324 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.546392 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:05.547324 19614 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.548951 19324 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:05.550647 19366 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, c4396a52dbda4132bfe4d3ad8b404adb; no voters: 
I20260709 12:31:05.551769 19583 raft_consensus.cc:2804] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:05.552276 19583 raft_consensus.cc:697] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:05.553268 19583 consensus_queue.cc:237] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.555253 19614 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:05.561937 19557 raft_consensus.cc:993] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.562505 19657 raft_consensus.cc:1081] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.562479 19149 catalog_manager.cc:5697] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:05.564733 19399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:05.565447 19399 raft_consensus.cc:493] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:05.565968 19399 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.576366 19399 raft_consensus.cc:515] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.579972 19399 leader_election.cc:290] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:05.580332 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:05.580684 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c0a2defbb4fe5b81c158dad78eda6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:05.581377 19614 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.581424 19324 raft_consensus.cc:3055] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:05.582475 19324 raft_consensus.cc:740] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:05.583649 19324 consensus_queue.cc:260] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.585764 19324 raft_consensus.cc:3060] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.589795 19614 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:05.591566 19364 leader_election.cc:304] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:05.592572 19583 raft_consensus.cc:2804] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:05.593125 19583 raft_consensus.cc:697] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:05.593454 19324 raft_consensus.cc:2468] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:05.594038 19583 consensus_queue.cc:237] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.600574 19614 raft_consensus.cc:3060] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.603152 19149 catalog_manager.cc:5697] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:05.625185 19614 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.660974 19614 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.668800 19614 pending_rounds.cc:85] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Aborting all ops after (but not including) 0
I20260709 12:31:05.669111 19614 pending_rounds.cc:107] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:31:05.669369 19614 raft_consensus.cc:2889] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:31:05.693810 19614 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:05.770903 19657 raft_consensus.cc:993] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.771579 19668 raft_consensus.cc:1081] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:05.773483 19399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:05.774168 19399 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:05.774550 19399 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:05.781392 19399 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.783689 19399 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:05.784920 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:05.784974 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:05.785696 19614 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:05.786187 19324 raft_consensus.cc:3055] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:05.786620 19324 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:05.787545 19324 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.789151 19324 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:05.795135 19614 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:05.796865 19364 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:05.797462 19324 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:05.798128 19558 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:05.798851 19558 raft_consensus.cc:697] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:05.799857 19558 consensus_queue.cc:237] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:05.808048 19149 catalog_manager.cc:5697] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 2 to 3, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:06.093811 19614 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.095959 19558 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:06.108639 19324 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.110591 19558 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:06.135031 19324 raft_consensus.cc:1275] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.136986 19558 consensus_queue.cc:1048] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:06.146438 19614 raft_consensus.cc:1275] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.148263 19583 consensus_queue.cc:1048] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:06.179477 19614 raft_consensus.cc:1275] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:06.181464 19673 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:06.198192 19324 raft_consensus.cc:1275] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:06.200169 19672 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:06.322985  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:06.332885 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:06.333636 19324 raft_consensus.cc:606] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:06.350406 19399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:06.351045 19399 raft_consensus.cc:606] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:06.354748 19399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:06.355585 19399 raft_consensus.cc:606] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:06.357240  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:31:06.358000  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:06.358843  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:06.373268 19679 raft_consensus.cc:993] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:06.373899 19672 raft_consensus.cc:1081] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:06.375891 19324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:06.376739 19324 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:06.377087 19324 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.385118 19324 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.387621 19324 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:06.388799 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:06.389072 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:06.389731 19614 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.389905 19399 raft_consensus.cc:3055] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:06.390416 19399 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:06.391320 19399 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.392396 19399 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.397346 19614 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:06.398288 19399 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:06.399019 19290 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:06.400128 19557 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:06.400748 19557 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:06.401604 19557 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.410169 19146 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 2 to 3, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 3 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:06.446578 19668 raft_consensus.cc:993] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:06.447139 19668 raft_consensus.cc:1081] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:06.449028 19399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:06.449823 19399 raft_consensus.cc:493] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:06.450172 19399 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:06.457410 19399 raft_consensus.cc:515] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
W20260709 12:31:06.458973 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:06.460031 19399 leader_election.cc:290] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:06.461357 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:06.461361 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65cf56359aff4b6ab82243a7dc2b8494" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:06.462411 19614 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:06.462445 19324 raft_consensus.cc:3055] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:06.463084 19324 raft_consensus.cc:740] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:06.464140 19324 consensus_queue.cc:260] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.465536 19324 raft_consensus.cc:3060] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:06.471462 19614 raft_consensus.cc:2468] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:06.473121 19364 leader_election.cc:304] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:06.473351 19324 raft_consensus.cc:2468] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 2.
I20260709 12:31:06.474128 19672 raft_consensus.cc:2804] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:06.474788 19672 raft_consensus.cc:697] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:06.475778 19672 consensus_queue.cc:237] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.484378 19146 catalog_manager.cc:5697] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 1 to 2, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:06.568691 19679 raft_consensus.cc:993] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:06.569380 19672 raft_consensus.cc:1081] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:06.571578 19324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:06.572324 19324 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:06.572788 19324 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.636052 19324 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.640213 19324 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:06.642611 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:06.642228 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:06.643891 19399 raft_consensus.cc:3055] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:06.644040 19614 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.645071 19399 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:06.646643 19399 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.649055 19399 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:06.656468 19614 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:06.658401 19399 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:06.659080 19290 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:06.660521 19668 raft_consensus.cc:2804] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:06.661207 19668 raft_consensus.cc:697] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:06.663045 19668 consensus_queue.cc:237] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:06.674384 19146 catalog_manager.cc:5697] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 2 to 3, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 3 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:06.827625 19614 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:06.829360 19668 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:06.830318 19397 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:06.832044 19668 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:06.967096 19324 raft_consensus.cc:1275] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.969501 19672 consensus_queue.cc:1048] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:06.983139 19614 raft_consensus.cc:1275] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:06.985929 19678 consensus_queue.cc:1048] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:07.141595 19397 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:07.141597 19614 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:07.143745 19668 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:07.144820 19557 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:07.366927  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:07.377709 19397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:07.378458 19397 raft_consensus.cc:606] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:07.382215 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:07.382741 19324 raft_consensus.cc:606] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:07.386097 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:07.386782 19324 raft_consensus.cc:606] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:07.388525  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:31:07.389218  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:07.390106  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:07.440053 19699 raft_consensus.cc:993] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:07.440676 19699 raft_consensus.cc:1081] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:07.442487 19397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:07.443100 19397 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:07.443516 19397 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:07.452152 19397 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.454310 19397 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:07.455366 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:07.455569 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:07.456248 19614 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:07.456396 19324 raft_consensus.cc:3055] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:07.457005 19324 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:07.458102 19324 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.459434 19324 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:31:07.460703 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:07.467074 19614 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:07.469015 19364 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:07.470180 19324 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:07.470199 19678 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:07.471487 19678 raft_consensus.cc:697] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:07.472934 19678 consensus_queue.cc:237] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.483000 19149 catalog_manager.cc:5697] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 2 to 3, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:07.660833 19704 raft_consensus.cc:993] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:07.661412 19672 raft_consensus.cc:1081] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:07.663892 19324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:07.664511 19324 raft_consensus.cc:493] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:07.664824 19324 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.671815 19324 raft_consensus.cc:515] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.673646 19324 leader_election.cc:290] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 4 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:07.675050 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:07.674955 19613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d14df91ba94e3fb1cc318a62c82db6" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:07.676200 19397 raft_consensus.cc:3055] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:31:07.676257 19613 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.676571 19397 raft_consensus.cc:740] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:07.677599 19397 consensus_queue.cc:260] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.678951 19397 raft_consensus.cc:3060] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.684880 19613 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 4.
I20260709 12:31:07.685907 19397 raft_consensus.cc:2468] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 4.
I20260709 12:31:07.686650 19290 leader_election.cc:304] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:07.687774 19699 raft_consensus.cc:2804] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:31:07.688323 19699 raft_consensus.cc:697] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:07.689319 19699 consensus_queue.cc:237] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.697317 19149 catalog_manager.cc:5697] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 3 to 4, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 4 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.860306 19668 raft_consensus.cc:993] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:07.860962 19699 raft_consensus.cc:1081] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:07.863106 19397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:07.863933 19397 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:07.864308 19397 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.871055 19397 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.873641 19397 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 4 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:07.874711 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:07.874500 19613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:07.875604 19324 raft_consensus.cc:3055] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:31:07.875604 19613 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.876194 19324 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:07.877362 19324 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.878978 19324 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:07.887501 19613 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 4.
I20260709 12:31:07.889007 19324 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 4.
I20260709 12:31:07.889315 19364 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:07.890625 19672 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:31:07.891266 19672 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:07.892318 19672 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:07.900959 19149 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 3 to 4, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 4 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:08.071763 19324 raft_consensus.cc:1275] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:31:08.073583 19672 consensus_queue.cc:1048] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:08.089496 19613 raft_consensus.cc:1275] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:31:08.093019 19678 consensus_queue.cc:1048] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.111093 19397 raft_consensus.cc:1275] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:08.113087 19699 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:08.126013 19614 raft_consensus.cc:1275] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:08.128895 19668 consensus_queue.cc:1048] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:08.398093  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:08.407511 19397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:08.408201 19397 raft_consensus.cc:606] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:08.412053 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:08.412731 19324 raft_consensus.cc:606] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:08.416064 19397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:08.416867 19397 raft_consensus.cc:606] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:08.418277  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:31:08.418905  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:08.419690  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:31:08.462006 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:08.482154 19614 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:08.484175 19672 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:08.487581 19324 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:08.489490 19678 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:08.503288 19678 raft_consensus.cc:993] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:08.503844 19678 raft_consensus.cc:1081] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:08.506351 19326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:08.508742 19326 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.509239 19326 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:31:08.517058 19715 raft_consensus.cc:993] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:08.517724 19668 raft_consensus.cc:1081] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:08.519718 19397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:08.520433 19397 raft_consensus.cc:493] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.520924 19397 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:08.521136 19326 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.527629 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:08.528000 19326 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 5 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:08.528446 19614 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:31:08.529194 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:08.529915 19399 raft_consensus.cc:3055] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:31:08.530231 19399 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:08.531484 19399 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.532914 19399 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:31:08.534885 19397 raft_consensus.cc:515] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.537576 19614 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 5.
I20260709 12:31:08.538820 19613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:08.538972 19290 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:08.539805 19613 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:08.540109 19668 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:31:08.540419 19326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff593059d734273ad123fd6c11736e8" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:08.540774 19668 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:08.541132 19326 raft_consensus.cc:3055] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:31:08.541386 19399 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 5.
I20260709 12:31:08.541457 19326 raft_consensus.cc:740] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:08.541738 19668 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.543473 19326 consensus_queue.cc:260] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.544914 19326 raft_consensus.cc:3060] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:08.547415 19397 leader_election.cc:290] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 4 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:08.549690 19613 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 4.
I20260709 12:31:08.551208 19364 leader_election.cc:304] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:08.552515 19709 raft_consensus.cc:2804] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:31:08.553131 19709 raft_consensus.cc:697] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:08.553416 19147 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 4 to 5, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 5 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.554276 19709 consensus_queue.cc:237] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.555145 19326 raft_consensus.cc:2468] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 4.
I20260709 12:31:08.566119 19149 catalog_manager.cc:5697] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 3 to 4, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 4 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:08.935026 19678 raft_consensus.cc:993] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:08.935636 19709 raft_consensus.cc:1081] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:08.937785 19326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:08.938359 19326 raft_consensus.cc:493] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.938650 19326 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:08.944988 19326 raft_consensus.cc:515] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.946910 19326 leader_election.cc:290] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:08.948047 19397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:08.948072 19613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3f77d6fb6346a2a0fed5fe259e8279" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:08.948947 19397 raft_consensus.cc:3055] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:08.949330 19613 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:08.949396 19397 raft_consensus.cc:740] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:08.950556 19397 consensus_queue.cc:260] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.951861 19397 raft_consensus.cc:3060] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:08.957201 19613 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:08.958418 19397 raft_consensus.cc:2468] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 3.
I20260709 12:31:08.958983 19290 leader_election.cc:304] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:08.959960 19668 raft_consensus.cc:2804] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:08.960486 19668 raft_consensus.cc:697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:08.961301 19668 consensus_queue.cc:237] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:08.970312 19149 catalog_manager.cc:5697] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 2 to 3, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 3 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:09.011973 19613 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:09.013863 19709 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:09.027635 19326 raft_consensus.cc:1275] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:31:09.029832 19709 consensus_queue.cc:1048] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:09.137950 19397 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 5 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:31:09.139767 19668 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:31:09.149526 19613 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 5 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:31:09.151540 19715 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:31:09.351842 19613 raft_consensus.cc:1275] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:09.353818 19715 consensus_queue.cc:1048] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:09.366581 19397 raft_consensus.cc:1275] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:09.368546 19699 consensus_queue.cc:1048] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:09.427070  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:09.436213 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:09.436784 19324 raft_consensus.cc:606] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:09.440001 19324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
mode: GRACEFUL
new_leader_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:40290
I20260709 12:31:09.440541 19324 raft_consensus.cc:606] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 LEADER]: Received request to transfer leadership to TS b69d54f4112a408faf42e325596135e2
I20260709 12:31:09.443991 19397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
mode: GRACEFUL
new_leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:54426
I20260709 12:31:09.444680 19397 raft_consensus.cc:606] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Received request to transfer leadership to TS c4396a52dbda4132bfe4d3ad8b404adb
I20260709 12:31:09.446324  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:31:09.446925  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:09.447628  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:31:09.463321 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:09.654942 19734 raft_consensus.cc:993] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: : Instructing follower c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:09.655651 19678 raft_consensus.cc:1081] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Signalling peer c4396a52dbda4132bfe4d3ad8b404adb to start an election
I20260709 12:31:09.657609 19324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f"
dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
 from {username='slave'} at 127.0.0.1:42546
I20260709 12:31:09.658383 19324 raft_consensus.cc:493] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:09.658828 19324 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:09.659885 19668 raft_consensus.cc:993] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:09.660503 19668 raft_consensus.cc:1081] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:09.662236 19397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b55ff906fe38479787c242960e080e0e"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:09.662814 19397 raft_consensus.cc:493] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:09.663141 19397 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:31:09.665428 19324 raft_consensus.cc:515] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.667384 19324 leader_election.cc:290] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 4 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), b69d54f4112a408faf42e325596135e2 (127.6.76.195:36049)
I20260709 12:31:09.668656 19399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b69d54f4112a408faf42e325596135e2"
I20260709 12:31:09.668321 19613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff002b6ff9f404b9bf3b21e75a8537f" candidate_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:09.669483 19399 raft_consensus.cc:3055] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:31:09.669548 19613 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:09.670009 19399 raft_consensus.cc:740] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:09.669705 19397 raft_consensus.cc:515] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.670944 19399 consensus_queue.cc:260] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.671916 19397 leader_election.cc:290] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 6 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:09.672399 19399 raft_consensus.cc:3060] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:09.672869 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:09.673362 19614 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:31:09.677634 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55ff906fe38479787c242960e080e0e" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:09.678505 19324 raft_consensus.cc:3055] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:31:09.678905 19324 raft_consensus.cc:740] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:09.678821 19613 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 4.
I20260709 12:31:09.680172 19324 consensus_queue.cc:260] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.680650 19290 leader_election.cc:304] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4396a52dbda4132bfe4d3ad8b404adb, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:09.681186 19399 raft_consensus.cc:2468] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4396a52dbda4132bfe4d3ad8b404adb in term 4.
I20260709 12:31:09.681603 19324 raft_consensus.cc:3060] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:31:09.681704 19668 raft_consensus.cc:2804] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:31:09.682330 19668 raft_consensus.cc:697] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 4 LEADER]: Becoming Leader. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:09.682621 19614 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 6.
I20260709 12:31:09.683279 19668 consensus_queue.cc:237] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.684080 19364 leader_election.cc:304] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:09.685014 19678 raft_consensus.cc:2804] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:31:09.685634 19678 raft_consensus.cc:697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 6 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:09.687141 19678 consensus_queue.cc:237] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.692162 19324 raft_consensus.cc:2468] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 6.
I20260709 12:31:09.695283 19147 catalog_manager.cc:5697] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 5 to 6, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 6 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:09.699952 19149 catalog_manager.cc:5697] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb reported cstate change: term changed from 3 to 4, leader changed from b69d54f4112a408faf42e325596135e2 (127.6.76.195) to c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194). New cstate: current_term: 4 leader_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:09.895890 19699 raft_consensus.cc:993] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: : Instructing follower b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:09.896561 19668 raft_consensus.cc:1081] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Signalling peer b69d54f4112a408faf42e325596135e2 to start an election
I20260709 12:31:09.898713 19399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d"
dest_uuid: "b69d54f4112a408faf42e325596135e2"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:31:09.899600 19399 raft_consensus.cc:493] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:09.899986 19399 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:09.906849 19399 raft_consensus.cc:515] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.908803 19399 leader_election.cc:290] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Requested vote from peers f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571), c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191)
I20260709 12:31:09.909647 19614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f41fa709b40c4b409a9b8cb61077dd36"
I20260709 12:31:09.910125 19324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784ade7ed9fe4c95a81b2d533b01272d" candidate_uuid: "b69d54f4112a408faf42e325596135e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "c4396a52dbda4132bfe4d3ad8b404adb"
I20260709 12:31:09.910694 19614 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:09.910858 19324 raft_consensus.cc:3055] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:09.911381 19324 raft_consensus.cc:740] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4396a52dbda4132bfe4d3ad8b404adb, State: Running, Role: LEADER
I20260709 12:31:09.912429 19324 consensus_queue.cc:260] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.913686 19324 raft_consensus.cc:3060] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:09.918301 19614 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:09.919816 19364 leader_election.cc:304] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b69d54f4112a408faf42e325596135e2, f41fa709b40c4b409a9b8cb61077dd36; no voters: 
I20260709 12:31:09.920130 19324 raft_consensus.cc:2468] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b69d54f4112a408faf42e325596135e2 in term 3.
I20260709 12:31:09.920791 19678 raft_consensus.cc:2804] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:09.921370 19678 raft_consensus.cc:697] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Becoming Leader. State: Replica: b69d54f4112a408faf42e325596135e2, State: Running, Role: LEADER
I20260709 12:31:09.922423 19678 consensus_queue.cc:237] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } }
I20260709 12:31:09.931922 19149 catalog_manager.cc:5697] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 reported cstate change: term changed from 2 to 3, leader changed from c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194) to b69d54f4112a408faf42e325596135e2 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "b69d54f4112a408faf42e325596135e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:10.107586 19399 raft_consensus.cc:1275] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:31:10.109706 19668 consensus_queue.cc:1048] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b69d54f4112a408faf42e325596135e2" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:10.122958 19614 raft_consensus.cc:1275] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Refusing update from remote peer c4396a52dbda4132bfe4d3ad8b404adb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:31:10.125188 19731 consensus_queue.cc:1048] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:10.232191 19324 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 6 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:31:10.234086 19678 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:31:10.246129 19613 raft_consensus.cc:1275] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 6 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:31:10.248962 19747 consensus_queue.cc:1048] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:31:10.366297 19324 raft_consensus.cc:1275] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:31:10.367712 19747 consensus_queue.cc:1048] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4396a52dbda4132bfe4d3ad8b404adb" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:10.383275 19613 raft_consensus.cc:1275] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Refusing update from remote peer b69d54f4112a408faf42e325596135e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:31:10.385356 19747 consensus_queue.cc:1048] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41fa709b40c4b409a9b8cb61077dd36" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:10.454733  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:31:10.464856 19196 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:31:10.494330  6451 tablet_server.cc:179] TabletServer@127.6.76.193:46571 shutting down...
W20260709 12:31:10.521116 19364 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:31:10.526592 19364 consensus_peers.cc:597] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.529824  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:10.530813  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.531836  6451 raft_consensus.cc:2243] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.532473  6451 raft_consensus.cc:2272] T 9b1c0a2defbb4fe5b81c158dad78eda6 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.535790  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.536688  6451 raft_consensus.cc:2243] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.537268  6451 raft_consensus.cc:2272] T 65cf56359aff4b6ab82243a7dc2b8494 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.540196  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.540964  6451 raft_consensus.cc:2243] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.541518  6451 raft_consensus.cc:2272] T 4f3f77d6fb6346a2a0fed5fe259e8279 P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.544699  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.545463  6451 raft_consensus.cc:2243] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.546092  6451 raft_consensus.cc:2272] T ae52cb849a654eb59ad9a30d5eb843ee P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.549158  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.549921  6451 raft_consensus.cc:2243] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.550493  6451 raft_consensus.cc:2272] T bff002b6ff9f404b9bf3b21e75a8537f P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.554308  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.555116  6451 raft_consensus.cc:2243] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.555835  6451 raft_consensus.cc:2272] T 7ff593059d734273ad123fd6c11736e8 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.559019  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.559724  6451 raft_consensus.cc:2243] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.560273  6451 raft_consensus.cc:2272] T 82d14df91ba94e3fb1cc318a62c82db6 P f41fa709b40c4b409a9b8cb61077dd36 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.562928  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.563611  6451 raft_consensus.cc:2243] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.564229  6451 raft_consensus.cc:2272] T b55ff906fe38479787c242960e080e0e P f41fa709b40c4b409a9b8cb61077dd36 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.568404  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.569180  6451 raft_consensus.cc:2243] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.569779  6451 raft_consensus.cc:2272] T 784ade7ed9fe4c95a81b2d533b01272d P f41fa709b40c4b409a9b8cb61077dd36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.574360  6451 tablet_replica.cc:333] stopping tablet replica
I20260709 12:31:10.575275  6451 raft_consensus.cc:2243] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.575963  6451 raft_consensus.cc:2272] T e2d7a78d5fd54c249abb720de77eb788 P f41fa709b40c4b409a9b8cb61077dd36 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:10.576215 19364 consensus_peers.cc:597] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:10.581269 19364 consensus_peers.cc:597] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:10.593132 19290 consensus_peers.cc:597] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.619913  6451 tablet_server.cc:196] TabletServer@127.6.76.193:46571 shutdown complete.
W20260709 12:31:10.620594 19364 consensus_peers.cc:597] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.658005  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
I20260709 12:31:10.687938  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:10.689030  6451 tablet_replica.cc:333] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.690395  6451 raft_consensus.cc:2243] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.691210  6451 raft_consensus.cc:2272] T 7ff593059d734273ad123fd6c11736e8 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.694169  6451 tablet_replica.cc:333] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.694924  6451 raft_consensus.cc:2243] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.696060  6451 raft_consensus.cc:2272] T bff002b6ff9f404b9bf3b21e75a8537f P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.699501  6451 tablet_replica.cc:333] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.700188  6451 raft_consensus.cc:2243] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.700711  6451 raft_consensus.cc:2272] T 784ade7ed9fe4c95a81b2d533b01272d P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.704393  6451 tablet_replica.cc:333] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.705077  6451 raft_consensus.cc:2243] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.705574  6451 raft_consensus.cc:2272] T b55ff906fe38479787c242960e080e0e P c4396a52dbda4132bfe4d3ad8b404adb [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.708735  6451 tablet_replica.cc:333] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.709431  6451 raft_consensus.cc:2243] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.710325  6451 raft_consensus.cc:2272] T 82d14df91ba94e3fb1cc318a62c82db6 P c4396a52dbda4132bfe4d3ad8b404adb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.712951  6451 tablet_replica.cc:333] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.713598  6451 raft_consensus.cc:2243] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.714118  6451 raft_consensus.cc:2272] T e2d7a78d5fd54c249abb720de77eb788 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.717937  6451 tablet_replica.cc:333] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.718832  6451 raft_consensus.cc:2243] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.719674  6451 raft_consensus.cc:2272] T 9b1c0a2defbb4fe5b81c158dad78eda6 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.723014  6451 tablet_replica.cc:333] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.723814  6451 raft_consensus.cc:2243] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.724874  6451 raft_consensus.cc:2272] T 4f3f77d6fb6346a2a0fed5fe259e8279 P c4396a52dbda4132bfe4d3ad8b404adb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.727593  6451 tablet_replica.cc:333] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.728250  6451 raft_consensus.cc:2243] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.728783  6451 raft_consensus.cc:2272] T 65cf56359aff4b6ab82243a7dc2b8494 P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.731483  6451 tablet_replica.cc:333] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb: stopping tablet replica
I20260709 12:31:10.732034  6451 raft_consensus.cc:2243] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.733040  6451 raft_consensus.cc:2272] T ae52cb849a654eb59ad9a30d5eb843ee P c4396a52dbda4132bfe4d3ad8b404adb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:10.767459 19364 consensus_peers.cc:597] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.770908  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
W20260709 12:31:10.781683 19366 consensus_peers.cc:597] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 -> Peer c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191): Couldn't send request to peer c4396a52dbda4132bfe4d3ad8b404adb. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:42191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:10.790771 19364 consensus_peers.cc:597] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 -> Peer f41fa709b40c4b409a9b8cb61077dd36 (127.6.76.193:46571): Couldn't send request to peer f41fa709b40c4b409a9b8cb61077dd36. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:46571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.811085  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
W20260709 12:31:10.841048 19366 consensus_peers.cc:597] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 -> Peer c4396a52dbda4132bfe4d3ad8b404adb (127.6.76.194:42191): Couldn't send request to peer c4396a52dbda4132bfe4d3ad8b404adb. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:42191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:10.853638  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:10.854838  6451 tablet_replica.cc:333] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.855819  6451 raft_consensus.cc:2243] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.856822  6451 raft_consensus.cc:2272] T 7ff593059d734273ad123fd6c11736e8 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.859779  6451 tablet_replica.cc:333] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.860589  6451 raft_consensus.cc:2243] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.861137  6451 raft_consensus.cc:2272] T bff002b6ff9f404b9bf3b21e75a8537f P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.865101  6451 tablet_replica.cc:333] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.866230  6451 raft_consensus.cc:2243] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.866938  6451 raft_consensus.cc:2272] T ae52cb849a654eb59ad9a30d5eb843ee P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.869957  6451 tablet_replica.cc:333] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.870644  6451 raft_consensus.cc:2243] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.871833  6451 raft_consensus.cc:2272] T b55ff906fe38479787c242960e080e0e P b69d54f4112a408faf42e325596135e2 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.875082  6451 tablet_replica.cc:333] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.875957  6451 raft_consensus.cc:2243] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.877279  6451 raft_consensus.cc:2272] T 784ade7ed9fe4c95a81b2d533b01272d P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.881156  6451 tablet_replica.cc:333] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.881965  6451 raft_consensus.cc:2243] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.882558  6451 raft_consensus.cc:2272] T 82d14df91ba94e3fb1cc318a62c82db6 P b69d54f4112a408faf42e325596135e2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.885254  6451 tablet_replica.cc:333] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.885949  6451 raft_consensus.cc:2243] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:10.886461  6451 raft_consensus.cc:2272] T 4f3f77d6fb6346a2a0fed5fe259e8279 P b69d54f4112a408faf42e325596135e2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.889328  6451 tablet_replica.cc:333] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.890030  6451 raft_consensus.cc:2243] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.891026  6451 raft_consensus.cc:2272] T 65cf56359aff4b6ab82243a7dc2b8494 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.893700  6451 tablet_replica.cc:333] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.894338  6451 raft_consensus.cc:2243] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.895475  6451 raft_consensus.cc:2272] T 9b1c0a2defbb4fe5b81c158dad78eda6 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.898746  6451 tablet_replica.cc:333] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2: stopping tablet replica
I20260709 12:31:10.899478  6451 raft_consensus.cc:2243] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.900767  6451 raft_consensus.cc:2272] T e2d7a78d5fd54c249abb720de77eb788 P b69d54f4112a408faf42e325596135e2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.938903  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:31:10.967797  6451 master.cc:562] Master@127.6.76.254:43155 shutting down...
I20260709 12:31:10.998610  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:10.999367  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:10.999773  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P 344fa86650ef4a1ba76a0ef1e8df7600: stopping tablet replica
I20260709 12:31:11.023355  6451 master.cc:584] Master@127.6.76.254:43155 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13797 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:31:11.078478  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:37311
I20260709 12:31:11.080067  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:11.086501 19757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.087514 19758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.089453 19760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:11.089802  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:11.091024  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:11.091267  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:11.091471  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600271091447 us; error 0 us; skew 500 ppm
I20260709 12:31:11.092125  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:11.095191  6451 webserver.cc:533] Webserver started at http://127.6.76.254:39643/ using document root <none> and password file <none>
I20260709 12:31:11.095862  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:11.096079  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:11.096385  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:11.097864  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "abe06caa6bc5424387681017c2d2c627"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.103811  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:31:11.108814 19765 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.110082  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 12:31:11.110567  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "abe06caa6bc5424387681017c2d2c627"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.110949  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:11.136484  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:11.138172  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:11.188298  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:37311
I20260709 12:31:11.188522 19816 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:37311 every 8 connection(s)
I20260709 12:31:11.193560 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.210451 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627: Bootstrap starting.
I20260709 12:31:11.217278 19817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.223412 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627: No bootstrap required, opened a new log
I20260709 12:31:11.226045 19817 raft_consensus.cc:359] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER }
I20260709 12:31:11.226521 19817 raft_consensus.cc:385] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:11.226842 19817 raft_consensus.cc:740] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe06caa6bc5424387681017c2d2c627, State: Initialized, Role: FOLLOWER
I20260709 12:31:11.227783 19817 consensus_queue.cc:260] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER }
I20260709 12:31:11.228344 19817 raft_consensus.cc:399] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:31:11.228607 19817 raft_consensus.cc:493] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:31:11.228906 19817 raft_consensus.cc:3060] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:11.234673 19817 raft_consensus.cc:515] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER }
I20260709 12:31:11.235381 19817 leader_election.cc:304] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abe06caa6bc5424387681017c2d2c627; no voters: 
I20260709 12:31:11.236837 19817 leader_election.cc:290] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:31:11.237389 19820 raft_consensus.cc:2804] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:11.239355 19820 raft_consensus.cc:697] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 1 LEADER]: Becoming Leader. State: Replica: abe06caa6bc5424387681017c2d2c627, State: Running, Role: LEADER
I20260709 12:31:11.240278 19820 consensus_queue.cc:237] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER }
I20260709 12:31:11.240554 19817 sys_catalog.cc:565] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:31:11.245685 19821 sys_catalog.cc:455] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abe06caa6bc5424387681017c2d2c627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER } }
I20260709 12:31:11.246680 19821 sys_catalog.cc:458] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:11.246834 19822 sys_catalog.cc:455] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abe06caa6bc5424387681017c2d2c627. Latest consensus state: current_term: 1 leader_uuid: "abe06caa6bc5424387681017c2d2c627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe06caa6bc5424387681017c2d2c627" member_type: VOTER } }
I20260709 12:31:11.247514 19822 sys_catalog.cc:458] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:11.250646 19828 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:31:11.260185 19828 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:31:11.261706  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:31:11.272426 19828 catalog_manager.cc:1383] Generated new cluster ID: 8328b3b3655e46458b6acc5bc4a9be35
I20260709 12:31:11.272848 19828 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:31:11.302683 19828 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:31:11.304325 19828 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:31:11.324667 19828 catalog_manager.cc:6070] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627: Generated new TSK 0
I20260709 12:31:11.325373 19828 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:31:11.394300  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:11.402254 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.403992 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.406087 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:11.406399  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:11.407625  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:11.407935  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:11.408119  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600271408096 us; error 0 us; skew 500 ppm
I20260709 12:31:11.408735  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:11.412065  6451 webserver.cc:533] Webserver started at http://127.6.76.193:45075/ using document root <none> and password file <none>
I20260709 12:31:11.412719  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:11.413028  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:11.413394  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:11.414753  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "cebd4e5e06a84cf28c1711e7c191f817"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.420704  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 12:31:11.425361 19846 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.426635  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:31:11.427038  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "cebd4e5e06a84cf28c1711e7c191f817"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.427477  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:11.440734  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:11.442374  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:11.444393  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:11.447165  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:11.447479  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:31:11.447777  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:11.447992  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.501166  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:33811
I20260709 12:31:11.501487 19908 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:33811 every 8 connection(s)
I20260709 12:31:11.507460  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:11.516601 19913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.518541 19914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:11.520630  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:11.522030 19916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:11.523365  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:11.523600  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:11.523837  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600271523797 us; error 0 us; skew 500 ppm
I20260709 12:31:11.524435 19909 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37311
I20260709 12:31:11.524490  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:11.524976 19909 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:11.525861 19909 heartbeater.cc:507] Master 127.6.76.254:37311 requested a full tablet report, sending...
I20260709 12:31:11.527905  6451 webserver.cc:533] Webserver started at http://127.6.76.194:44437/ using document root <none> and password file <none>
I20260709 12:31:11.528594  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:11.528625 19782 ts_manager.cc:194] Registered new tserver with Master: cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811)
I20260709 12:31:11.528853  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:11.529426  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:11.531374  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "3889f975b76645719118439712890c95"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.531657 19782 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51870
I20260709 12:31:11.537961  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:31:11.543418 19921 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.544556  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:11.544974  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "3889f975b76645719118439712890c95"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.545284  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:11.557839  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:11.559410  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:11.561240  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:11.564038  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:11.564363  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.564661  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:11.564870  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.618108  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:37585
I20260709 12:31:11.618268 19983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:37585 every 8 connection(s)
I20260709 12:31:11.624116  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:11.633033 19987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.634593 19988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:11.637110 19990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:11.638098  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:11.639605  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:11.639916  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:11.640087 19984 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37311
I20260709 12:31:11.640149  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600271640128 us; error 0 us; skew 500 ppm
I20260709 12:31:11.640702 19984 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:11.641196  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:11.642110 19984 heartbeater.cc:507] Master 127.6.76.254:37311 requested a full tablet report, sending...
I20260709 12:31:11.645043  6451 webserver.cc:533] Webserver started at http://127.6.76.195:35651/ using document root <none> and password file <none>
I20260709 12:31:11.645433 19782 ts_manager.cc:194] Registered new tserver with Master: 3889f975b76645719118439712890c95 (127.6.76.194:37585)
I20260709 12:31:11.645954  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:11.646277  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:11.646629  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:11.647603 19782 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51886
I20260709 12:31:11.648339  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.654295  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:31:11.659880 19995 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.661363  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:31:11.661832  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
format_stamp: "Formatted at 2026-07-09 12:31:11 on dist-test-slave-n447"
I20260709 12:31:11.662145  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:11.677435  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:11.679342  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:11.681095  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:11.684056  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:11.684381  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.684676  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:11.684883  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:11.742733  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:44167
I20260709 12:31:11.742861 20057 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:44167 every 8 connection(s)
I20260709 12:31:11.761165 20058 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37311
I20260709 12:31:11.761961 20058 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:11.762893 20058 heartbeater.cc:507] Master 127.6.76.254:37311 requested a full tablet report, sending...
I20260709 12:31:11.765590 19782 ts_manager.cc:194] Registered new tserver with Master: c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:11.765868  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019358432s
I20260709 12:31:11.767607 19782 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51898
I20260709 12:31:11.799360 19782 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51910:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:11.801853 19782 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:11.859495 19874 tablet_service.cc:1511] Processing CreateTablet for tablet 11ace9b43a6d4b27b10e3984ddeb66c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:11.861634 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ace9b43a6d4b27b10e3984ddeb66c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.862555 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 80c0a8f928914529a8c6813f4994c752 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:11.862205 20023 tablet_service.cc:1511] Processing CreateTablet for tablet 11ace9b43a6d4b27b10e3984ddeb66c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:11.859855 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 80c0a8f928914529a8c6813f4994c752 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:11.864365 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c0a8f928914529a8c6813f4994c752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.864449 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c0a8f928914529a8c6813f4994c752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.865206 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ace9b43a6d4b27b10e3984ddeb66c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.890347 20073 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: Bootstrap starting.
I20260709 12:31:11.893136 19949 tablet_service.cc:1511] Processing CreateTablet for tablet 11ace9b43a6d4b27b10e3984ddeb66c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:11.895402 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ace9b43a6d4b27b10e3984ddeb66c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.901026 19948 tablet_service.cc:1511] Processing CreateTablet for tablet 80c0a8f928914529a8c6813f4994c752 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=67481fbb943740048ddf75377c9fce50]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:11.902694 20073 tablet_bootstrap.cc:654] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.903146 19948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c0a8f928914529a8c6813f4994c752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:11.920802 20074 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: Bootstrap starting.
I20260709 12:31:11.924041 20073 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: No bootstrap required, opened a new log
I20260709 12:31:11.924980 20073 ts_tablet_manager.cc:1403] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.011s
I20260709 12:31:11.935304 20074 tablet_bootstrap.cc:654] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.936933 20073 raft_consensus.cc:359] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:11.938450 20073 raft_consensus.cc:385] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:11.938925 20073 raft_consensus.cc:740] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:11.940320 20076 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: Bootstrap starting.
I20260709 12:31:11.940030 20073 consensus_queue.cc:260] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:11.943409 20058 heartbeater.cc:499] Master 127.6.76.254:37311 was elected leader, sending a full tablet report...
I20260709 12:31:11.943871 20073 ts_tablet_manager.cc:1434] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.002s
I20260709 12:31:11.944602 20074 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: No bootstrap required, opened a new log
I20260709 12:31:11.945322 20074 ts_tablet_manager.cc:1403] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260709 12:31:11.945780 20073 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: Bootstrap starting.
I20260709 12:31:11.949009 20074 raft_consensus.cc:359] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:11.949944 20074 raft_consensus.cc:385] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:11.949543 20076 tablet_bootstrap.cc:654] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.950516 20074 raft_consensus.cc:740] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Initialized, Role: FOLLOWER
I20260709 12:31:11.951704 20074 consensus_queue.cc:260] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:11.957388 20073 tablet_bootstrap.cc:654] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.964067 19909 heartbeater.cc:499] Master 127.6.76.254:37311 was elected leader, sending a full tablet report...
I20260709 12:31:11.964556 20074 ts_tablet_manager.cc:1434] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:31:11.965677 20074 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: Bootstrap starting.
I20260709 12:31:11.974817 20073 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: No bootstrap required, opened a new log
I20260709 12:31:11.974851 20074 tablet_bootstrap.cc:654] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:11.975915 20073 ts_tablet_manager.cc:1403] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.003s
I20260709 12:31:11.980954 20073 raft_consensus.cc:359] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:11.982488 20073 raft_consensus.cc:385] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:11.983103 20073 raft_consensus.cc:740] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:11.984782 20073 consensus_queue.cc:260] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:11.991806 20073 ts_tablet_manager.cc:1434] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:31:11.998875 20076 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: No bootstrap required, opened a new log
I20260709 12:31:11.999181 20074 tablet_bootstrap.cc:492] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: No bootstrap required, opened a new log
I20260709 12:31:11.999614 20076 ts_tablet_manager.cc:1403] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.039s
I20260709 12:31:11.999967 20074 ts_tablet_manager.cc:1403] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.015s
I20260709 12:31:12.003136 20074 raft_consensus.cc:359] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.003971 20074 raft_consensus.cc:385] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.003499 20076 raft_consensus.cc:359] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.004309 20074 raft_consensus.cc:740] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.004520 20076 raft_consensus.cc:385] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.004978 20076 raft_consensus.cc:740] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.005337 20074 consensus_queue.cc:260] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.008261 20074 ts_tablet_manager.cc:1434] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:31:12.005841 20076 consensus_queue.cc:260] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.013293 19984 heartbeater.cc:499] Master 127.6.76.254:37311 was elected leader, sending a full tablet report...
I20260709 12:31:12.013692 20076 ts_tablet_manager.cc:1434] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 12:31:12.014953 20076 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: Bootstrap starting.
I20260709 12:31:12.022728 20076 tablet_bootstrap.cc:654] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.028802 20076 tablet_bootstrap.cc:492] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: No bootstrap required, opened a new log
I20260709 12:31:12.029465 20076 ts_tablet_manager.cc:1403] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:31:12.032102 20076 raft_consensus.cc:359] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.032719 20076 raft_consensus.cc:385] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.032977 20076 raft_consensus.cc:740] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.033710 20076 consensus_queue.cc:260] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.035957 20076 ts_tablet_manager.cc:1434] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:31:12.072744 20083 raft_consensus.cc:493] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:12.073342 20083 raft_consensus.cc:515] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.076164 20083 leader_election.cc:290] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.087651 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817" is_pre_election: true
I20260709 12:31:12.088618 19884 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 0.
I20260709 12:31:12.089627 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" is_pre_election: true
I20260709 12:31:12.090034 19924 leader_election.cc:304] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.090579 20033 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 0.
I20260709 12:31:12.091176 20083 raft_consensus.cc:2804] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:12.091727 20083 raft_consensus.cc:493] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:12.092110 20083 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.099555 20083 raft_consensus.cc:515] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.101773 20083 leader_election.cc:290] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 election: Requested vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.103157 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:12.102962 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
I20260709 12:31:12.104120 20033 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.104202 19884 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.107017 20078 raft_consensus.cc:493] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:12.107853 20078 raft_consensus.cc:515] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.111356 20078 leader_election.cc:290] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), 3889f975b76645719118439712890c95 (127.6.76.194:37585)
I20260709 12:31:12.115696 19884 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 1.
I20260709 12:31:12.115960 20033 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 1.
I20260709 12:31:12.118023 19924 leader_election.cc:304] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, c542a3c906144ac3ae2ba81d9ac0b3d7; no voters: 
I20260709 12:31:12.119064 20083 raft_consensus.cc:2804] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:12.120685 20083 raft_consensus.cc:697] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 LEADER]: Becoming Leader. State: Replica: 3889f975b76645719118439712890c95, State: Running, Role: LEADER
I20260709 12:31:12.122314 20083 consensus_queue.cc:237] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.127331 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817" is_pre_election: true
I20260709 12:31:12.128382 19884 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 0.
I20260709 12:31:12.130353 19998 leader_election.cc:304] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c542a3c906144ac3ae2ba81d9ac0b3d7, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.131521 20078 raft_consensus.cc:2804] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:12.131965 20078 raft_consensus.cc:493] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:12.132367 20078 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.132682 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3889f975b76645719118439712890c95" is_pre_election: true
I20260709 12:31:12.133539 19959 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 0.
I20260709 12:31:12.136204 19782 catalog_manager.cc:5697] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3889f975b76645719118439712890c95 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "3889f975b76645719118439712890c95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.139556 20078 raft_consensus.cc:515] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.141661 20078 leader_election.cc:290] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 1 election: Requested vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), 3889f975b76645719118439712890c95 (127.6.76.194:37585)
I20260709 12:31:12.142939 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3889f975b76645719118439712890c95"
I20260709 12:31:12.142860 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
I20260709 12:31:12.144106 19884 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.144734 19959 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.153929 19884 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 1.
I20260709 12:31:12.154909 19959 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 1.
I20260709 12:31:12.156040 19998 leader_election.cc:304] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c542a3c906144ac3ae2ba81d9ac0b3d7, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.157562 20078 raft_consensus.cc:2804] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:12.160110 20078 raft_consensus.cc:697] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 LEADER]: Becoming Leader. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Running, Role: LEADER
I20260709 12:31:12.161053 20078 consensus_queue.cc:237] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.169159 19782 catalog_manager.cc:5697] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.248984 19782 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51916:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:12.251636 19782 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:12.283401 19874 tablet_service.cc:1511] Processing CreateTablet for tablet 17fe282e44b84863884faaf2d5d399ef (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:12.284926 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fe282e44b84863884faaf2d5d399ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.284735 19948 tablet_service.cc:1511] Processing CreateTablet for tablet 17fe282e44b84863884faaf2d5d399ef (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:12.286293 19948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fe282e44b84863884faaf2d5d399ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.285931 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 17fe282e44b84863884faaf2d5d399ef (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:31:12.287541 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fe282e44b84863884faaf2d5d399ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.289352 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2c22b580464153a6464f548c184f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:12.291119 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2c22b580464153a6464f548c184f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.294456 20023 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2c22b580464153a6464f548c184f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:12.293326 19949 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2c22b580464153a6464f548c184f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d25c1edba65c4638b427d298c138beaf]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:31:12.296079 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2c22b580464153a6464f548c184f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.296079 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2c22b580464153a6464f548c184f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:12.309904 20074 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: Bootstrap starting.
I20260709 12:31:12.309904 20073 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Bootstrap starting.
I20260709 12:31:12.318913 20073 tablet_bootstrap.cc:654] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.320102 20076 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: Bootstrap starting.
I20260709 12:31:12.318913 20074 tablet_bootstrap.cc:654] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.330961 20076 tablet_bootstrap.cc:654] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.338894 20073 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: No bootstrap required, opened a new log
I20260709 12:31:12.339850 20073 ts_tablet_manager.cc:1403] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.013s
I20260709 12:31:12.345849 20074 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: No bootstrap required, opened a new log
I20260709 12:31:12.346693 20074 ts_tablet_manager.cc:1403] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.010s
I20260709 12:31:12.346693 20076 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: No bootstrap required, opened a new log
I20260709 12:31:12.346524 20073 raft_consensus.cc:359] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.347745 20076 ts_tablet_manager.cc:1403] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 12:31:12.347739 20073 raft_consensus.cc:385] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.348480 20073 raft_consensus.cc:740] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.349687 20073 consensus_queue.cc:260] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.350849 20074 raft_consensus.cc:359] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.351790 20076 raft_consensus.cc:359] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.352694 20076 raft_consensus.cc:385] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.353163 20076 raft_consensus.cc:740] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.353298 20074 raft_consensus.cc:385] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.353735 20074 raft_consensus.cc:740] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.354398 20076 consensus_queue.cc:260] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.354808 20074 consensus_queue.cc:260] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.356240 20073 ts_tablet_manager.cc:1434] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:31:12.357682 20073 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: Bootstrap starting.
I20260709 12:31:12.358112 20076 ts_tablet_manager.cc:1434] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:31:12.359890 20076 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: Bootstrap starting.
I20260709 12:31:12.366809 20073 tablet_bootstrap.cc:654] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.369412 20076 tablet_bootstrap.cc:654] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.371654 20074 ts_tablet_manager.cc:1434] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 12:31:12.379959 20074 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: Bootstrap starting.
I20260709 12:31:12.396762 20074 tablet_bootstrap.cc:654] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:12.398087 20076 tablet_bootstrap.cc:492] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: No bootstrap required, opened a new log
I20260709 12:31:12.398924 20076 ts_tablet_manager.cc:1403] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: Time spent bootstrapping tablet: real 0.039s	user 0.026s	sys 0.011s
I20260709 12:31:12.399922 20073 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: No bootstrap required, opened a new log
I20260709 12:31:12.400820 20073 ts_tablet_manager.cc:1403] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.012s
I20260709 12:31:12.402716 20076 raft_consensus.cc:359] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.403855 20076 raft_consensus.cc:385] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.404276 20076 raft_consensus.cc:740] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.405301 20076 consensus_queue.cc:260] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.405270 20073 raft_consensus.cc:359] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.406318 20073 raft_consensus.cc:385] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.407382 20074 tablet_bootstrap.cc:492] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: No bootstrap required, opened a new log
I20260709 12:31:12.408016 20076 ts_tablet_manager.cc:1434] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:31:12.408548 20074 ts_tablet_manager.cc:1403] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.008s
I20260709 12:31:12.407429 20073 raft_consensus.cc:740] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.410612 20073 consensus_queue.cc:260] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.412214 20074 raft_consensus.cc:359] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.413448 20073 ts_tablet_manager.cc:1434] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 12:31:12.413455 20074 raft_consensus.cc:385] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:12.414759 20074 raft_consensus.cc:740] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Initialized, Role: FOLLOWER
I20260709 12:31:12.415719 20074 consensus_queue.cc:260] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.418385 20074 ts_tablet_manager.cc:1434] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:31:12.498558 20079 raft_consensus.cc:493] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:12.499157 20079 raft_consensus.cc:515] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.502086 20079 leader_election.cc:290] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3889f975b76645719118439712890c95 (127.6.76.194:37585), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.511366 20083 consensus_queue.cc:1048] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:12.515761 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2c22b580464153a6464f548c184f74" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3889f975b76645719118439712890c95" is_pre_election: true
I20260709 12:31:12.516692 19959 raft_consensus.cc:2468] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 0.
I20260709 12:31:12.518311 19849 leader_election.cc:304] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.519562 20079 raft_consensus.cc:2804] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:12.520074 20079 raft_consensus.cc:493] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:12.520052 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2c22b580464153a6464f548c184f74" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" is_pre_election: true
I20260709 12:31:12.520862 20079 raft_consensus.cc:3060] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.521181 20032 raft_consensus.cc:2468] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 0.
I20260709 12:31:12.531821 20079 raft_consensus.cc:515] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.534636 20079 leader_election.cc:290] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 1 election: Requested vote from peers 3889f975b76645719118439712890c95 (127.6.76.194:37585), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.535306 20083 consensus_queue.cc:1048] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:12.536717 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2c22b580464153a6464f548c184f74" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3889f975b76645719118439712890c95"
I20260709 12:31:12.537532 19959 raft_consensus.cc:3060] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.537636 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2c22b580464153a6464f548c184f74" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:12.538554 20032 raft_consensus.cc:3060] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.550176 20032 raft_consensus.cc:2468] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 1.
I20260709 12:31:12.550675 19959 raft_consensus.cc:2468] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 1.
I20260709 12:31:12.551918 19849 leader_election.cc:304] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c542a3c906144ac3ae2ba81d9ac0b3d7, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.552935 20079 raft_consensus.cc:2804] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:12.555366 20079 raft_consensus.cc:697] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 LEADER]: Becoming Leader. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Running, Role: LEADER
I20260709 12:31:12.556245 20079 consensus_queue.cc:237] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.560956 20083 raft_consensus.cc:493] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:12.561636 20083 raft_consensus.cc:515] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.564175 20083 leader_election.cc:290] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.565095 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817" is_pre_election: true
I20260709 12:31:12.565516 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" is_pre_election: true
I20260709 12:31:12.566063 19884 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 0.
I20260709 12:31:12.566277 20032 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 0.
I20260709 12:31:12.567561 19924 leader_election.cc:304] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.566587 19778 catalog_manager.cc:5697] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 reported cstate change: term changed from 0 to 1, leader changed from <none> to cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "cebd4e5e06a84cf28c1711e7c191f817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:12.568476 20083 raft_consensus.cc:2804] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:12.568962 20083 raft_consensus.cc:493] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:12.569394 20083 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.577512 20083 raft_consensus.cc:515] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.579666 20083 leader_election.cc:290] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 election: Requested vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.580531 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
I20260709 12:31:12.580873 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:12.581305 19884 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.581481 20032 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:12.590517 20032 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 1.
I20260709 12:31:12.590517 19884 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 1.
I20260709 12:31:12.592355 19924 leader_election.cc:304] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.593317 20083 raft_consensus.cc:2804] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:12.593894 20083 raft_consensus.cc:697] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 LEADER]: Becoming Leader. State: Replica: 3889f975b76645719118439712890c95, State: Running, Role: LEADER
I20260709 12:31:12.594892 20083 consensus_queue.cc:237] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.603960 19778 catalog_manager.cc:5697] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3889f975b76645719118439712890c95 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "3889f975b76645719118439712890c95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:12.674278 19959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6"
dest_uuid: "3889f975b76645719118439712890c95"
mode: GRACEFUL
new_leader_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
 from {username='slave'} at 127.0.0.1:40644
I20260709 12:31:12.675010 19959 raft_consensus.cc:606] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 LEADER]: Received request to transfer leadership to TS cebd4e5e06a84cf28c1711e7c191f817
I20260709 12:31:12.692210 20032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752"
dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
mode: GRACEFUL
new_leader_uuid: "3889f975b76645719118439712890c95"
 from {username='slave'} at 127.0.0.1:48376
I20260709 12:31:12.693001 20032 raft_consensus.cc:606] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 LEADER]: Received request to transfer leadership to TS 3889f975b76645719118439712890c95
I20260709 12:31:12.694600 20078 consensus_queue.cc:1048] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:12.701422 19959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef"
dest_uuid: "3889f975b76645719118439712890c95"
mode: GRACEFUL
new_leader_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
 from {username='slave'} at 127.0.0.1:40644
I20260709 12:31:12.702152 19959 raft_consensus.cc:606] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 LEADER]: Received request to transfer leadership to TS cebd4e5e06a84cf28c1711e7c191f817
I20260709 12:31:12.712872 20078 consensus_queue.cc:1048] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:12.723932 20114 raft_consensus.cc:993] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: : Instructing follower 3889f975b76645719118439712890c95 to start an election
I20260709 12:31:12.724602 20078 raft_consensus.cc:1081] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 LEADER]: Signalling peer 3889f975b76645719118439712890c95 to start an election
I20260709 12:31:12.727164 19959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752"
dest_uuid: "3889f975b76645719118439712890c95"
 from {username='slave'} at 127.0.0.1:40640
I20260709 12:31:12.727959 19959 raft_consensus.cc:493] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:12.728376 19959 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.736400 19959 raft_consensus.cc:515] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.739387 19959 leader_election.cc:290] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 2 election: Requested vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:12.741746 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:12.741886 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80c0a8f928914529a8c6813f4994c752" candidate_uuid: "3889f975b76645719118439712890c95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
I20260709 12:31:12.742724 20032 raft_consensus.cc:3055] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:12.742902 19884 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.743212 20032 raft_consensus.cc:740] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Running, Role: LEADER
I20260709 12:31:12.744336 20032 consensus_queue.cc:260] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.745848 20032 raft_consensus.cc:3060] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.752166 19884 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 2.
I20260709 12:31:12.753773 19924 leader_election.cc:304] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.754758 20083 raft_consensus.cc:2804] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:12.755424 20083 raft_consensus.cc:697] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 2 LEADER]: Becoming Leader. State: Replica: 3889f975b76645719118439712890c95, State: Running, Role: LEADER
I20260709 12:31:12.755455 20032 raft_consensus.cc:2468] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3889f975b76645719118439712890c95 in term 2.
I20260709 12:31:12.756595 20083 consensus_queue.cc:237] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:12.765125 19778 catalog_manager.cc:5697] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 reported cstate change: term changed from 1 to 2, leader changed from c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195) to 3889f975b76645719118439712890c95 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "3889f975b76645719118439712890c95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:12.928454 20087 raft_consensus.cc:993] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: : Instructing follower cebd4e5e06a84cf28c1711e7c191f817 to start an election
I20260709 12:31:12.929116 20083 raft_consensus.cc:1081] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 LEADER]: Signalling peer cebd4e5e06a84cf28c1711e7c191f817 to start an election
I20260709 12:31:12.931171 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6"
dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
 from {username='slave'} at 127.0.0.1:55122
I20260709 12:31:12.931891 19884 raft_consensus.cc:493] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:12.932425 19884 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.941162 19884 raft_consensus.cc:515] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.943742 19884 leader_election.cc:290] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 2 election: Requested vote from peers c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167), 3889f975b76645719118439712890c95 (127.6.76.194:37585)
I20260709 12:31:12.945032 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:12.945487 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ace9b43a6d4b27b10e3984ddeb66c6" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3889f975b76645719118439712890c95"
I20260709 12:31:12.946005 20032 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.946363 19959 raft_consensus.cc:3055] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:12.946870 19959 raft_consensus.cc:740] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Running, Role: LEADER
I20260709 12:31:12.947853 19959 consensus_queue.cc:260] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.948972 19959 raft_consensus.cc:3060] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.954007 20032 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 2.
I20260709 12:31:12.955780 19849 leader_election.cc:304] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c542a3c906144ac3ae2ba81d9ac0b3d7, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:12.956077 19959 raft_consensus.cc:2468] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 2.
I20260709 12:31:12.957013 20079 raft_consensus.cc:2804] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:12.957684 20079 raft_consensus.cc:697] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Becoming Leader. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Running, Role: LEADER
I20260709 12:31:12.958609 20079 consensus_queue.cc:237] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } }
I20260709 12:31:12.966763 19778 catalog_manager.cc:5697] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 reported cstate change: term changed from 1 to 2, leader changed from 3889f975b76645719118439712890c95 (127.6.76.194) to cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "cebd4e5e06a84cf28c1711e7c191f817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:13.021673 20079 consensus_queue.cc:1048] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:13.035396 20109 consensus_queue.cc:1048] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:13.138985 20083 consensus_queue.cc:1048] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:13.153235 20116 consensus_queue.cc:1048] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:13.163282 20087 raft_consensus.cc:993] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: : Instructing follower cebd4e5e06a84cf28c1711e7c191f817 to start an election
I20260709 12:31:13.163937 20116 raft_consensus.cc:1081] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 LEADER]: Signalling peer cebd4e5e06a84cf28c1711e7c191f817 to start an election
I20260709 12:31:13.165783 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef"
dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
 from {username='slave'} at 127.0.0.1:55122
I20260709 12:31:13.166396 19884 raft_consensus.cc:493] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:13.166746 19884 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:13.175868 19883 raft_consensus.cc:1275] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Refusing update from remote peer 3889f975b76645719118439712890c95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.175737 19884 raft_consensus.cc:515] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.178001 20116 consensus_queue.cc:1048] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:13.179190 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3889f975b76645719118439712890c95"
I20260709 12:31:13.180070 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "cebd4e5e06a84cf28c1711e7c191f817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
I20260709 12:31:13.180266 19884 leader_election.cc:290] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 2 election: Requested vote from peers 3889f975b76645719118439712890c95 (127.6.76.194:37585), c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195:44167)
I20260709 12:31:13.180783 20032 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:13.180088 19959 raft_consensus.cc:3055] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:13.182631 19959 raft_consensus.cc:740] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3889f975b76645719118439712890c95, State: Running, Role: LEADER
I20260709 12:31:13.183617 19959 consensus_queue.cc:260] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.185174 19959 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:13.189532 20033 raft_consensus.cc:1275] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Refusing update from remote peer 3889f975b76645719118439712890c95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.192436 20032 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 2.
I20260709 12:31:13.196416 19959 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebd4e5e06a84cf28c1711e7c191f817 in term 2.
I20260709 12:31:13.196590 19849 leader_election.cc:304] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c542a3c906144ac3ae2ba81d9ac0b3d7, cebd4e5e06a84cf28c1711e7c191f817; no voters: 
I20260709 12:31:13.197779 20109 raft_consensus.cc:2804] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:13.198316 20109 raft_consensus.cc:697] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Becoming Leader. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Running, Role: LEADER
I20260709 12:31:13.199112 20109 consensus_queue.cc:237] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.201565 20083 consensus_queue.cc:1048] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:13.211982 19780 catalog_manager.cc:5697] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 reported cstate change: term changed from 1 to 2, leader changed from 3889f975b76645719118439712890c95 (127.6.76.194) to cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "cebd4e5e06a84cf28c1711e7c191f817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:13.258217  6451 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:31:13.261255  6451 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:31:13.261897  6451 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:31:13.280687 19883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef"
dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
mode: GRACEFUL
new_leader_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
 from {username='slave'} at 127.0.0.1:55132
I20260709 12:31:13.281337 19883 raft_consensus.cc:606] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Received request to transfer leadership to TS c542a3c906144ac3ae2ba81d9ac0b3d7
I20260709 12:31:13.282804  6451 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:31:13.380965 19959 raft_consensus.cc:1275] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Refusing update from remote peer cebd4e5e06a84cf28c1711e7c191f817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.383077 20109 consensus_queue.cc:1048] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:13.397171 20032 raft_consensus.cc:1275] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Refusing update from remote peer cebd4e5e06a84cf28c1711e7c191f817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.399051 20079 consensus_queue.cc:1048] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:13.623068 19959 raft_consensus.cc:1275] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Refusing update from remote peer cebd4e5e06a84cf28c1711e7c191f817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.625278 20079 consensus_queue.cc:1048] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:13.639511 20032 raft_consensus.cc:1275] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Refusing update from remote peer cebd4e5e06a84cf28c1711e7c191f817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:13.642710 20109 consensus_queue.cc:1048] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:13.649852 20079 raft_consensus.cc:993] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: : Instructing follower c542a3c906144ac3ae2ba81d9ac0b3d7 to start an election
I20260709 12:31:13.650475 20109 raft_consensus.cc:1081] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Signalling peer c542a3c906144ac3ae2ba81d9ac0b3d7 to start an election
I20260709 12:31:13.652454 20032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef"
dest_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7"
 from {username='slave'} at 127.0.0.1:48374
I20260709 12:31:13.653400 20032 raft_consensus.cc:493] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:13.654012 20032 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:13.663136 20032 raft_consensus.cc:515] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.665673 20032 leader_election.cc:290] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 3 election: Requested vote from peers cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193:33811), 3889f975b76645719118439712890c95 (127.6.76.194:37585)
I20260709 12:31:13.667008 19883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cebd4e5e06a84cf28c1711e7c191f817"
I20260709 12:31:13.667075 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fe282e44b84863884faaf2d5d399ef" candidate_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3889f975b76645719118439712890c95"
I20260709 12:31:13.667987 19883 raft_consensus.cc:3055] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:13.668056 19959 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:13.668512 19883 raft_consensus.cc:740] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cebd4e5e06a84cf28c1711e7c191f817, State: Running, Role: LEADER
I20260709 12:31:13.669802 19883 consensus_queue.cc:260] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.671530 19883 raft_consensus.cc:3060] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:13.676360 19959 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 3.
I20260709 12:31:13.677816 19998 leader_election.cc:304] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3889f975b76645719118439712890c95, c542a3c906144ac3ae2ba81d9ac0b3d7; no voters: 
I20260709 12:31:13.678661 20078 raft_consensus.cc:2804] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:13.679901 20078 raft_consensus.cc:697] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 LEADER]: Becoming Leader. State: Replica: c542a3c906144ac3ae2ba81d9ac0b3d7, State: Running, Role: LEADER
I20260709 12:31:13.679946 19883 raft_consensus.cc:2468] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c542a3c906144ac3ae2ba81d9ac0b3d7 in term 3.
I20260709 12:31:13.680914 20078 consensus_queue.cc:237] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } }
I20260709 12:31:13.688777 19780 catalog_manager.cc:5697] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 reported cstate change: term changed from 2 to 3, leader changed from cebd4e5e06a84cf28c1711e7c191f817 (127.6.76.193) to c542a3c906144ac3ae2ba81d9ac0b3d7 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cebd4e5e06a84cf28c1711e7c191f817" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3889f975b76645719118439712890c95" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c542a3c906144ac3ae2ba81d9ac0b3d7" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 44167 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:13.943590  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
I20260709 12:31:13.984010  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:13.984908  6451 tablet_replica.cc:333] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817: stopping tablet replica
I20260709 12:31:13.985699  6451 raft_consensus.cc:2243] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:13.986708  6451 raft_consensus.cc:2272] T 9d2c22b580464153a6464f548c184f74 P cebd4e5e06a84cf28c1711e7c191f817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:13.989503  6451 tablet_replica.cc:333] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817: stopping tablet replica
I20260709 12:31:13.990206  6451 raft_consensus.cc:2243] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:13.990737  6451 raft_consensus.cc:2272] T 17fe282e44b84863884faaf2d5d399ef P cebd4e5e06a84cf28c1711e7c191f817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:13.993748  6451 tablet_replica.cc:333] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817: stopping tablet replica
I20260709 12:31:13.994431  6451 raft_consensus.cc:2243] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:13.994951  6451 raft_consensus.cc:2272] T 80c0a8f928914529a8c6813f4994c752 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:13.997926  6451 tablet_replica.cc:333] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817: stopping tablet replica
I20260709 12:31:13.998575  6451 raft_consensus.cc:2243] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:13.999678  6451 raft_consensus.cc:2272] T 11ace9b43a6d4b27b10e3984ddeb66c6 P cebd4e5e06a84cf28c1711e7c191f817 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.030601  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
I20260709 12:31:14.049439  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
I20260709 12:31:14.084417  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:14.085449  6451 tablet_replica.cc:333] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95: stopping tablet replica
I20260709 12:31:14.086162  6451 raft_consensus.cc:2243] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.086697  6451 raft_consensus.cc:2272] T 9d2c22b580464153a6464f548c184f74 P 3889f975b76645719118439712890c95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.089002  6451 tablet_replica.cc:333] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95: stopping tablet replica
I20260709 12:31:14.089632  6451 raft_consensus.cc:2243] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.090152  6451 raft_consensus.cc:2272] T 11ace9b43a6d4b27b10e3984ddeb66c6 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.093902  6451 tablet_replica.cc:333] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95: stopping tablet replica
I20260709 12:31:14.094659  6451 raft_consensus.cc:2243] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.095320  6451 raft_consensus.cc:2272] T 17fe282e44b84863884faaf2d5d399ef P 3889f975b76645719118439712890c95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.098191  6451 tablet_replica.cc:333] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95: stopping tablet replica
I20260709 12:31:14.098810  6451 raft_consensus.cc:2243] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:14.100018  6451 raft_consensus.cc:2272] T 80c0a8f928914529a8c6813f4994c752 P 3889f975b76645719118439712890c95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.127408  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
I20260709 12:31:14.146042  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
I20260709 12:31:14.172266  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:14.173126  6451 tablet_replica.cc:333] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7: stopping tablet replica
I20260709 12:31:14.173815  6451 raft_consensus.cc:2243] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.174403  6451 raft_consensus.cc:2272] T 9d2c22b580464153a6464f548c184f74 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.177089  6451 tablet_replica.cc:333] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7: stopping tablet replica
I20260709 12:31:14.177760  6451 raft_consensus.cc:2243] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.178284  6451 raft_consensus.cc:2272] T 11ace9b43a6d4b27b10e3984ddeb66c6 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.181891  6451 tablet_replica.cc:333] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: stopping tablet replica
I20260709 12:31:14.182583  6451 raft_consensus.cc:2243] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:14.183437  6451 pending_rounds.cc:70] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Trying to abort 1 pending ops.
I20260709 12:31:14.183682  6451 pending_rounds.cc:77] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626721005989888 op_type: NO_OP noop_request { }
I20260709 12:31:14.184058  6451 raft_consensus.cc:2889] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:31:14.184448  6451 raft_consensus.cc:2272] T 17fe282e44b84863884faaf2d5d399ef P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.187702  6451 tablet_replica.cc:333] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7: stopping tablet replica
I20260709 12:31:14.188313  6451 raft_consensus.cc:2243] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:14.188829  6451 raft_consensus.cc:2272] T 80c0a8f928914529a8c6813f4994c752 P c542a3c906144ac3ae2ba81d9ac0b3d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.216719  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:31:14.231907  6451 master.cc:562] Master@127.6.76.254:37311 shutting down...
I20260709 12:31:14.260690  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:14.261518  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:14.261916  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P abe06caa6bc5424387681017c2d2c627: stopping tablet replica
I20260709 12:31:14.286042  6451 master.cc:584] Master@127.6.76.254:37311 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3253 ms)
[----------] 6 tests from LeaderRebalancerTest (455117 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:31:14.332557  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:39479
I20260709 12:31:14.334771  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:14.342468 20127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.343106 20128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.345706 20130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.349172  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:14.350214  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:14.350454  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:14.350651  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600274350630 us; error 0 us; skew 500 ppm
I20260709 12:31:14.351351  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:14.354945  6451 webserver.cc:533] Webserver started at http://127.6.76.254:35473/ using document root <none> and password file <none>
I20260709 12:31:14.356580  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:14.356928  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:14.357335  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:14.359081  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "bf513f8ef2274dc89f5767178052c758"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.364485  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:31:14.368897 20135 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.370100  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:31:14.370453  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "bf513f8ef2274dc89f5767178052c758"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.370760  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:14.395169  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:14.396659  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:14.443670  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:39479
I20260709 12:31:14.443809 20186 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:39479 every 8 connection(s)
I20260709 12:31:14.448173 20187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:14.462309 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: Bootstrap starting.
I20260709 12:31:14.467391 20187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:14.472738 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: No bootstrap required, opened a new log
I20260709 12:31:14.475620 20187 raft_consensus.cc:359] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER }
I20260709 12:31:14.476058 20187 raft_consensus.cc:385] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:14.476362 20187 raft_consensus.cc:740] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf513f8ef2274dc89f5767178052c758, State: Initialized, Role: FOLLOWER
I20260709 12:31:14.477574 20187 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER }
I20260709 12:31:14.478125 20187 raft_consensus.cc:399] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:31:14.478343 20187 raft_consensus.cc:493] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:31:14.478680 20187 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:14.485354 20187 raft_consensus.cc:515] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER }
I20260709 12:31:14.486106 20187 leader_election.cc:304] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf513f8ef2274dc89f5767178052c758; no voters: 
I20260709 12:31:14.487622 20187 leader_election.cc:290] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:31:14.488133 20190 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:14.489847 20190 raft_consensus.cc:697] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 1 LEADER]: Becoming Leader. State: Replica: bf513f8ef2274dc89f5767178052c758, State: Running, Role: LEADER
I20260709 12:31:14.490662 20190 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER }
I20260709 12:31:14.491154 20187 sys_catalog.cc:565] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:31:14.494019 20191 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf513f8ef2274dc89f5767178052c758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER } }
I20260709 12:31:14.495005 20191 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:14.494035 20192 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf513f8ef2274dc89f5767178052c758. Latest consensus state: current_term: 1 leader_uuid: "bf513f8ef2274dc89f5767178052c758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf513f8ef2274dc89f5767178052c758" member_type: VOTER } }
I20260709 12:31:14.496604 20192 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:14.501798 20195 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:31:14.507969 20195 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:31:14.510468  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:31:14.520028 20195 catalog_manager.cc:1383] Generated new cluster ID: afa4e34d203344de90de0ba0594041fd
I20260709 12:31:14.520376 20195 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:31:14.538697 20195 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:31:14.540226 20195 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:31:14.562045 20195 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: Generated new TSK 0
I20260709 12:31:14.562718 20195 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:31:14.578469  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:14.585878 20208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.586691 20209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.590160  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:14.590233 20211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.592183  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:14.592562  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:14.592810  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600274592788 us; error 0 us; skew 500 ppm
I20260709 12:31:14.593747  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:14.597070  6451 webserver.cc:533] Webserver started at http://127.6.76.193:39009/ using document root <none> and password file <none>
I20260709 12:31:14.597579  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:14.597750  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:14.597986  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:14.599145  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "c8380535ade4453897c237ed66a87884"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.604328  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:31:14.608740 20216 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.609942  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:31:14.610344  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "c8380535ade4453897c237ed66a87884"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.610632  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:14.635620  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:14.636896  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:14.638607  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:14.641276  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:14.641501  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.641697  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:14.641844  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.694446  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:40555
I20260709 12:31:14.694634 20278 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:40555 every 8 connection(s)
I20260709 12:31:14.700623  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:14.708839 20283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.710649 20284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.712489 20286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.714314  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:14.716284 20279 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39479
I20260709 12:31:14.716462  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:14.716894  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:14.716893 20279 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:14.717180  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600274717165 us; error 0 us; skew 500 ppm
I20260709 12:31:14.717911  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:14.717978 20279 heartbeater.cc:507] Master 127.6.76.254:39479 requested a full tablet report, sending...
I20260709 12:31:14.721576 20152 ts_manager.cc:194] Registered new tserver with Master: c8380535ade4453897c237ed66a87884 (127.6.76.193:40555)
I20260709 12:31:14.721774  6451 webserver.cc:533] Webserver started at http://127.6.76.194:43881/ using document root <none> and password file <none>
I20260709 12:31:14.722541  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:14.722813  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:14.723161  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:14.724334 20152 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58540
I20260709 12:31:14.724818  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "2372546982404aca8dd9280c3cef67d7"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.730679  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:31:14.735474 20291 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.736763  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:14.737234  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "2372546982404aca8dd9280c3cef67d7"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.737665  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:14.750937  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:14.752745  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:14.754539  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:14.757434  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:14.757642  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.757943  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:14.758111  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.808413  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:39449
I20260709 12:31:14.808552 20353 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:39449 every 8 connection(s)
I20260709 12:31:14.814523  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:14.823828 20357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:14.824391 20358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.830214  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:14.831693 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:14.832059 20354 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39479
I20260709 12:31:14.832648 20354 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:14.833218  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:31:14.833629 20354 heartbeater.cc:507] Master 127.6.76.254:39479 requested a full tablet report, sending...
W20260709 12:31:14.833633  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:14.834374  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600274834348 us; error 0 us; skew 500 ppm
I20260709 12:31:14.835592  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:14.836830 20152 ts_manager.cc:194] Registered new tserver with Master: 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:14.838966  6451 webserver.cc:533] Webserver started at http://127.6.76.195:40135/ using document root <none> and password file <none>
I20260709 12:31:14.839087 20152 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58548
I20260709 12:31:14.840015  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:14.840353  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:14.840730  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:14.842070  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "0d452c6b77c24d3383c18bfecf46b553"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.848109  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:31:14.852964 20365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.853969  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 12:31:14.854404  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "0d452c6b77c24d3383c18bfecf46b553"
format_stamp: "Formatted at 2026-07-09 12:31:14 on dist-test-slave-n447"
I20260709 12:31:14.854719  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:14.868956  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:14.870404  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:14.872080  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:14.874770  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:14.875001  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.875408  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:14.875587  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:14.928241  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:34907
I20260709 12:31:14.928359 20427 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:34907 every 8 connection(s)
I20260709 12:31:14.945920 20428 heartbeater.cc:344] Connected to a master server at 127.6.76.254:39479
I20260709 12:31:14.946623 20428 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:14.947539 20428 heartbeater.cc:507] Master 127.6.76.254:39479 requested a full tablet report, sending...
I20260709 12:31:14.950635 20152 ts_manager.cc:194] Registered new tserver with Master: 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:14.950812  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018970317s
I20260709 12:31:14.952431 20152 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58556
I20260709 12:31:14.985159 20152 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58566:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:14.988046 20152 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:15.074486 20244 tablet_service.cc:1511] Processing CreateTablet for tablet 025ffab388944f0d9a9a06066cf9c047 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:15.076056 20244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ffab388944f0d9a9a06066cf9c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.080634 20243 tablet_service.cc:1511] Processing CreateTablet for tablet 469c697d0b89410a8601a811d219a886 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:15.082628 20243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469c697d0b89410a8601a811d219a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.085004 20241 tablet_service.cc:1511] Processing CreateTablet for tablet d7dd179b8b364dc6a77fba8d32af59da (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:15.085285 20393 tablet_service.cc:1511] Processing CreateTablet for tablet 025ffab388944f0d9a9a06066cf9c047 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:15.086968 20241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7dd179b8b364dc6a77fba8d32af59da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.088667 20392 tablet_service.cc:1511] Processing CreateTablet for tablet 469c697d0b89410a8601a811d219a886 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:15.091351 20391 tablet_service.cc:1511] Processing CreateTablet for tablet 06577a91b394473e9e25580cb5321af8 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:15.085004 20242 tablet_service.cc:1511] Processing CreateTablet for tablet 06577a91b394473e9e25580cb5321af8 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:15.093827 20242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06577a91b394473e9e25580cb5321af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.095995 20390 tablet_service.cc:1511] Processing CreateTablet for tablet d7dd179b8b364dc6a77fba8d32af59da (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:15.096030 20240 tablet_service.cc:1511] Processing CreateTablet for tablet 73b4aa4959e742839c0bfc9bc1473d1d (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:15.087030 20393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ffab388944f0d9a9a06066cf9c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.098824 20240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b4aa4959e742839c0bfc9bc1473d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.103796 20390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7dd179b8b364dc6a77fba8d32af59da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.104674 20391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06577a91b394473e9e25580cb5321af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.105522 20392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469c697d0b89410a8601a811d219a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.110997 20389 tablet_service.cc:1511] Processing CreateTablet for tablet 73b4aa4959e742839c0bfc9bc1473d1d (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:15.113566 20389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b4aa4959e742839c0bfc9bc1473d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.126958 20239 tablet_service.cc:1511] Processing CreateTablet for tablet b56bd5b1a67a4cb4985b3ac616570040 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:15.129202 20239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56bd5b1a67a4cb4985b3ac616570040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.129458 20319 tablet_service.cc:1511] Processing CreateTablet for tablet 025ffab388944f0d9a9a06066cf9c047 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:15.130020 20318 tablet_service.cc:1511] Processing CreateTablet for tablet 469c697d0b89410a8601a811d219a886 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:15.132146 20319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ffab388944f0d9a9a06066cf9c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.133092 20318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469c697d0b89410a8601a811d219a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.142195 20317 tablet_service.cc:1511] Processing CreateTablet for tablet 06577a91b394473e9e25580cb5321af8 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:15.144349 20317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06577a91b394473e9e25580cb5321af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.149197 20388 tablet_service.cc:1511] Processing CreateTablet for tablet b56bd5b1a67a4cb4985b3ac616570040 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:15.148685 20316 tablet_service.cc:1511] Processing CreateTablet for tablet d7dd179b8b364dc6a77fba8d32af59da (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:15.151291 20238 tablet_service.cc:1511] Processing CreateTablet for tablet ea30683a652f4c079ea1c7f9c35f6ead (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:15.153331 20314 tablet_service.cc:1511] Processing CreateTablet for tablet b56bd5b1a67a4cb4985b3ac616570040 (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:15.154261 20238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea30683a652f4c079ea1c7f9c35f6ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.151356 20315 tablet_service.cc:1511] Processing CreateTablet for tablet 73b4aa4959e742839c0bfc9bc1473d1d (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:15.156747 20313 tablet_service.cc:1511] Processing CreateTablet for tablet ea30683a652f4c079ea1c7f9c35f6ead (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:15.158200 20387 tablet_service.cc:1511] Processing CreateTablet for tablet ea30683a652f4c079ea1c7f9c35f6ead (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:15.152326 20316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7dd179b8b364dc6a77fba8d32af59da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.151207 20388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56bd5b1a67a4cb4985b3ac616570040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.160063 20315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b4aa4959e742839c0bfc9bc1473d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.163697 20313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea30683a652f4c079ea1c7f9c35f6ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.163790 20387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea30683a652f4c079ea1c7f9c35f6ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.167142 20314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56bd5b1a67a4cb4985b3ac616570040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.167840 20237 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa1bf79364874984b9d4160e6c4ef (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:15.169471 20237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa1bf79364874984b9d4160e6c4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.196938 20312 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa1bf79364874984b9d4160e6c4ef (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:15.199059 20312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa1bf79364874984b9d4160e6c4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.200949 20386 tablet_service.cc:1511] Processing CreateTablet for tablet eb6fa1bf79364874984b9d4160e6c4ef (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:15.203496 20386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6fa1bf79364874984b9d4160e6c4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.239660 20236 tablet_service.cc:1511] Processing CreateTablet for tablet a6036cf4951d448cac3b1e812244d02b (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:15.241288 20236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6036cf4951d448cac3b1e812244d02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.242416 20444 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.261770 20444 tablet_bootstrap.cc:654] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.282792 20443 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.283111 20445 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.290783 20445 tablet_bootstrap.cc:654] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.306357 20387 tablet_service.cc:1511] Processing CreateTablet for tablet a6036cf4951d448cac3b1e812244d02b (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:15.317948 20443 tablet_bootstrap.cc:654] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.318671 20387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6036cf4951d448cac3b1e812244d02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.324256 20313 tablet_service.cc:1511] Processing CreateTablet for tablet a6036cf4951d448cac3b1e812244d02b (DEFAULT_TABLE table=test-workload [id=2898a36eeb7f4073b4d9191b336edd06]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:15.334805 20313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6036cf4951d448cac3b1e812244d02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:15.383792 20444 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.384776 20444 ts_tablet_manager.cc:1403] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.143s	user 0.025s	sys 0.006s
I20260709 12:31:15.386873 20445 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.387630 20445 ts_tablet_manager.cc:1403] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.105s	user 0.018s	sys 0.050s
I20260709 12:31:15.390950 20445 raft_consensus.cc:359] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.391657 20445 raft_consensus.cc:385] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.392004 20445 raft_consensus.cc:740] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.393761 20445 consensus_queue.cc:260] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.395864 20443 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:15.396539 20354 heartbeater.cc:499] Master 127.6.76.254:39479 was elected leader, sending a full tablet report...
I20260709 12:31:15.396641 20443 ts_tablet_manager.cc:1403] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.114s	user 0.045s	sys 0.012s
I20260709 12:31:15.397177 20445 ts_tablet_manager.cc:1434] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:31:15.398489 20445 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.401094 20443 raft_consensus.cc:359] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.402024 20443 raft_consensus.cc:385] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.402326 20443 raft_consensus.cc:740] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.393383 20444 raft_consensus.cc:359] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.403360 20443 consensus_queue.cc:260] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.403652 20444 raft_consensus.cc:385] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.405257 20444 raft_consensus.cc:740] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.405757 20445 tablet_bootstrap.cc:654] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.406731 20444 consensus_queue.cc:260] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.422001 20444 ts_tablet_manager.cc:1434] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.037s	user 0.013s	sys 0.001s
I20260709 12:31:15.423125 20428 heartbeater.cc:499] Master 127.6.76.254:39479 was elected leader, sending a full tablet report...
I20260709 12:31:15.423987 20444 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.439469 20449 raft_consensus.cc:493] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.439985 20444 tablet_bootstrap.cc:654] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.440558 20449 raft_consensus.cc:515] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.460572 20444 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.461540 20444 ts_tablet_manager.cc:1403] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.004s
I20260709 12:31:15.466030 20444 raft_consensus.cc:359] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.467166 20444 raft_consensus.cc:385] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.467876 20444 raft_consensus.cc:740] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.469740 20444 consensus_queue.cc:260] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.471905 20449 leader_election.cc:290] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.490000 20279 heartbeater.cc:499] Master 127.6.76.254:39479 was elected leader, sending a full tablet report...
I20260709 12:31:15.493534 20445 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.494740 20445 ts_tablet_manager.cc:1403] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.097s	user 0.043s	sys 0.050s
I20260709 12:31:15.498971 20445 raft_consensus.cc:359] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.500063 20445 raft_consensus.cc:385] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.500509 20445 raft_consensus.cc:740] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.500797 20443 ts_tablet_manager.cc:1434] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.103s	user 0.044s	sys 0.043s
I20260709 12:31:15.501717 20445 consensus_queue.cc:260] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.504817 20445 ts_tablet_manager.cc:1434] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
W20260709 12:31:15.505204 20200 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:15.506413 20445 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.506958 20443 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.514120 20445 tablet_bootstrap.cc:654] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.519167 20443 tablet_bootstrap.cc:654] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.527659 20445 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.528394 20445 ts_tablet_manager.cc:1403] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.009s
I20260709 12:31:15.532224 20445 raft_consensus.cc:359] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.533453 20445 raft_consensus.cc:385] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.533895 20445 raft_consensus.cc:740] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.533897 20443 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:15.535508 20443 ts_tablet_manager.cc:1403] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.013s
I20260709 12:31:15.535562 20445 consensus_queue.cc:260] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.536734 20452 raft_consensus.cc:493] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.539083 20445 ts_tablet_manager.cc:1434] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:31:15.539863 20452 raft_consensus.cc:515] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.541627 20445 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.549170 20452 leader_election.cc:290] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:15.540096 20443 raft_consensus.cc:359] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.553061 20443 raft_consensus.cc:385] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.553367 20443 raft_consensus.cc:740] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.556533 20443 consensus_queue.cc:260] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.561801 20443 ts_tablet_manager.cc:1434] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 12:31:15.563292 20443 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.565486 20445 tablet_bootstrap.cc:654] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.565687 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:15.566653 20254 raft_consensus.cc:2468] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:15.568816 20294 leader_election.cc:304] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.570173 20449 raft_consensus.cc:2804] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.570619 20449 raft_consensus.cc:493] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.570959 20449 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.559661 20444 ts_tablet_manager.cc:1434] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.098s	user 0.055s	sys 0.028s
I20260709 12:31:15.574550 20444 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.579051 20443 tablet_bootstrap.cc:654] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.579752 20445 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.580854 20445 ts_tablet_manager.cc:1403] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.008s
I20260709 12:31:15.582662 20449 raft_consensus.cc:515] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.585127 20445 raft_consensus.cc:359] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.586385 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:15.586480 20445 raft_consensus.cc:385] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.587146 20254 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.587327 20445 raft_consensus.cc:740] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.588948 20445 consensus_queue.cc:260] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.597332 20254 raft_consensus.cc:2468] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:15.612562 20294 leader_election.cc:304] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.601183 20445 ts_tablet_manager.cc:1434] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:31:15.616173 20445 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.601663 20444 tablet_bootstrap.cc:654] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.608484 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:15.617753 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:15.618623 20329 raft_consensus.cc:2468] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:15.620174 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:15.620879 20369 leader_election.cc:304] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
W20260709 12:31:15.622051 20292 leader_election.cc:343] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.623953 20445 tablet_bootstrap.cc:654] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.623710 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
W20260709 12:31:15.624185 20292 leader_election.cc:343] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.610798 20449 leader_election.cc:290] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.625854 20449 raft_consensus.cc:2804] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:15.627699 20456 raft_consensus.cc:2804] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.628535 20456 raft_consensus.cc:493] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.629115 20449 raft_consensus.cc:697] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:15.641425 20449 consensus_queue.cc:237] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.629292 20456 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.637320 20445 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
W20260709 12:31:15.627650 20368 leader_election.cc:343] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.649941 20445 ts_tablet_manager.cc:1403] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.000s
I20260709 12:31:15.650732 20443 tablet_bootstrap.cc:492] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:15.650982 20444 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.651490 20443 ts_tablet_manager.cc:1403] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.089s	user 0.023s	sys 0.041s
I20260709 12:31:15.651839 20444 ts_tablet_manager.cc:1403] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.021s
I20260709 12:31:15.654186 20445 raft_consensus.cc:359] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.656392 20443 raft_consensus.cc:359] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.659677 20445 raft_consensus.cc:385] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.659889 20443 raft_consensus.cc:385] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.660223 20445 raft_consensus.cc:740] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.656636 20444 raft_consensus.cc:359] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.660917 20444 raft_consensus.cc:385] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.661278 20444 raft_consensus.cc:740] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.660367 20443 raft_consensus.cc:740] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.661460 20445 consensus_queue.cc:260] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.662344 20443 consensus_queue.cc:260] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.664239 20445 ts_tablet_manager.cc:1434] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:31:15.665344 20443 ts_tablet_manager.cc:1434] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:31:15.665513 20445 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.665985 20456 raft_consensus.cc:515] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.666535 20444 consensus_queue.cc:260] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.669710 20444 ts_tablet_manager.cc:1434] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:31:15.670408 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:15.671048 20444 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.671245 20254 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.672314 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:15.673105 20329 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.680281 20443 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.683514 20445 tablet_bootstrap.cc:654] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.665601 20152 catalog_manager.cc:5697] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:15.687825 20443 tablet_bootstrap.cc:654] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.691419 20444 tablet_bootstrap.cc:654] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.692386 20254 raft_consensus.cc:2468] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:15.695154 20456 leader_election.cc:290] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:15.705971 20329 raft_consensus.cc:2468] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:15.708909 20369 leader_election.cc:304] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:15.711104 20456 raft_consensus.cc:2804] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:15.713390 20456 raft_consensus.cc:697] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:15.715598 20444 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.715299 20456 consensus_queue.cc:237] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.716632 20444 ts_tablet_manager.cc:1403] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.008s
I20260709 12:31:15.722921 20450 raft_consensus.cc:493] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.723625 20450 raft_consensus.cc:515] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.724124 20443 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:15.724820 20443 ts_tablet_manager.cc:1403] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.016s
I20260709 12:31:15.725107 20445 tablet_bootstrap.cc:492] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.725755 20445 ts_tablet_manager.cc:1403] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.024s
I20260709 12:31:15.733016 20450 leader_election.cc:290] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.736012 20444 raft_consensus.cc:359] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.737169 20444 raft_consensus.cc:385] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.737560 20444 raft_consensus.cc:740] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.738610 20444 consensus_queue.cc:260] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.730008 20443 raft_consensus.cc:359] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.730346 20445 raft_consensus.cc:359] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.740010 20443 raft_consensus.cc:385] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.740120 20445 raft_consensus.cc:385] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.740820 20445 raft_consensus.cc:740] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.742219 20443 raft_consensus.cc:740] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.741793 20445 consensus_queue.cc:260] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.743342 20443 consensus_queue.cc:260] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.745040 20445 ts_tablet_manager.cc:1434] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 12:31:15.746313 20445 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.746361 20443 ts_tablet_manager.cc:1434] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:31:15.749140 20444 ts_tablet_manager.cc:1434] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:31:15.750598 20444 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.755599 20445 tablet_bootstrap.cc:654] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.759478 20443 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.769632 20443 tablet_bootstrap.cc:654] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.775283 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:15.776165 20329 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 0.
I20260709 12:31:15.776937 20444 tablet_bootstrap.cc:654] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.777281 20152 catalog_manager.cc:5697] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:15.778100 20220 leader_election.cc:304] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.779577 20450 raft_consensus.cc:2804] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.780057 20450 raft_consensus.cc:493] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.780429 20445 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.780475 20450 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.782968 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:15.783700 20445 ts_tablet_manager.cc:1403] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.012s
I20260709 12:31:15.783849 20403 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 0.
I20260709 12:31:15.787380 20445 raft_consensus.cc:359] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.788884 20445 raft_consensus.cc:385] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.789296 20445 raft_consensus.cc:740] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.790511 20445 consensus_queue.cc:260] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.792595 20443 tablet_bootstrap.cc:492] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:15.793431 20443 ts_tablet_manager.cc:1403] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.003s
I20260709 12:31:15.794657 20445 ts_tablet_manager.cc:1434] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:31:15.796650 20445 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.797657 20443 raft_consensus.cc:359] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.798805 20443 raft_consensus.cc:385] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.799412 20443 raft_consensus.cc:740] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.800613 20443 consensus_queue.cc:260] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.801128 20456 raft_consensus.cc:493] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.801782 20456 raft_consensus.cc:515] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.806257 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:15.807407 20254 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:15.809468 20445 tablet_bootstrap.cc:654] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.810640 20368 leader_election.cc:304] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.812388 20452 raft_consensus.cc:2804] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.812987 20452 raft_consensus.cc:493] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.813486 20452 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.829243 20449 raft_consensus.cc:493] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.830245 20449 raft_consensus.cc:515] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.833523 20456 leader_election.cc:290] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:15.833971 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:15.834491 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:15.835150 20329 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
W20260709 12:31:15.836308 20294 leader_election.cc:343] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.841032 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
W20260709 12:31:15.844797 20292 leader_election.cc:343] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.845630 20292 leader_election.cc:304] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7; no voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884
I20260709 12:31:15.847199 20464 raft_consensus.cc:2749] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:15.848634 20449 leader_election.cc:290] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.853581 20450 raft_consensus.cc:515] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.850649 20452 raft_consensus.cc:515] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.858830 20450 leader_election.cc:290] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.859905 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:15.860718 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:15.861841 20403 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.862061 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:15.862946 20254 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.860718 20329 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.873036 20444 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.874014 20444 ts_tablet_manager.cc:1403] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.124s	user 0.038s	sys 0.023s
I20260709 12:31:15.874594 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:15.875535 20328 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.875964 20254 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:15.877525 20368 leader_election.cc:304] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.877974 20403 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 1.
I20260709 12:31:15.878957 20456 raft_consensus.cc:2804] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:15.880367 20217 leader_election.cc:304] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.886361 20452 leader_election.cc:290] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:15.886907 20444 raft_consensus.cc:359] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.888525 20444 raft_consensus.cc:385] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.889035 20444 raft_consensus.cc:740] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.889535 20450 raft_consensus.cc:2804] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:15.890050 20329 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 1.
I20260709 12:31:15.890352 20450 raft_consensus.cc:697] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:15.890568 20444 consensus_queue.cc:260] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.891458 20450 consensus_queue.cc:237] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.892935 20456 raft_consensus.cc:697] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:15.887157 20328 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:15.894932 20444 ts_tablet_manager.cc:1434] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 12:31:15.894976 20456 consensus_queue.cc:237] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.897058 20444 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:15.903589 20443 ts_tablet_manager.cc:1434] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.110s	user 0.032s	sys 0.052s
I20260709 12:31:15.904495 20445 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:15.905328 20443 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:15.906687 20444 tablet_bootstrap.cc:654] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.905678 20445 ts_tablet_manager.cc:1403] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.109s	user 0.020s	sys 0.037s
I20260709 12:31:15.911125 20445 raft_consensus.cc:359] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.920099 20445 raft_consensus.cc:385] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.920850 20445 raft_consensus.cc:740] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.915833 20449 raft_consensus.cc:493] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.922398 20449 raft_consensus.cc:515] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.922374 20445 consensus_queue.cc:260] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.925891 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa1bf79364874984b9d4160e6c4ef" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:15.926821 20254 raft_consensus.cc:2468] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:15.928418 20445 ts_tablet_manager.cc:1434] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 12:31:15.931073 20443 tablet_bootstrap.cc:654] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.931401 20445 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:15.933197 20444 tablet_bootstrap.cc:492] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:15.933874 20444 ts_tablet_manager.cc:1403] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.015s
I20260709 12:31:15.939383 20464 raft_consensus.cc:493] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.939915 20456 raft_consensus.cc:493] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:15.940094 20464 raft_consensus.cc:515] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.940532 20456 raft_consensus.cc:515] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.944741 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:15.944927 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:15.946290 20328 raft_consensus.cc:2468] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
W20260709 12:31:15.947829 20368 leader_election.cc:343] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:15.927824 20150 catalog_manager.cc:5697] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:15.929023 20294 leader_election.cc:304] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:15.920526 20152 catalog_manager.cc:5697] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:15.958904 20449 leader_election.cc:290] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.965056 20449 raft_consensus.cc:2804] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.965632 20449 raft_consensus.cc:493] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.966091 20449 raft_consensus.cc:3060] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.989473 20369 leader_election.cc:304] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: c8380535ade4453897c237ed66a87884
I20260709 12:31:15.990309 20444 raft_consensus.cc:359] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:15.990816 20482 raft_consensus.cc:2804] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:15.991710 20482 raft_consensus.cc:493] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:15.992165 20482 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:15.991729 20444 raft_consensus.cc:385] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:15.991222 20445 tablet_bootstrap.cc:654] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:15.994230 20444 raft_consensus.cc:740] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:15.997715 20456 leader_election.cc:290] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
W20260709 12:31:15.996908 20368 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:40555, user_credentials={real_user=slave}} blocked reactor thread for 49198.9us
I20260709 12:31:15.998064 20464 leader_election.cc:290] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:15.999677 20456 raft_consensus.cc:493] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:16.000298 20456 raft_consensus.cc:515] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.000439 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa1bf79364874984b9d4160e6c4ef" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:16.002591 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:16.003816 20444 consensus_queue.cc:260] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.004508 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:16.005455 20403 raft_consensus.cc:2468] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:16.005555 20456 leader_election.cc:290] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.009929 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:16.005483 20328 raft_consensus.cc:2468] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
W20260709 12:31:16.010677 20292 leader_election.cc:343] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:31:16.014536 20294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.76.193:40555, user_credentials={real_user=slave}} blocked reactor thread for 85633.8us
I20260709 12:31:16.007115 20444 ts_tablet_manager.cc:1434] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.073s	user 0.018s	sys 0.011s
I20260709 12:31:16.018170 20444 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:16.018664 20369 leader_election.cc:304] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:16.020105 20443 tablet_bootstrap.cc:492] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:16.022105 20443 ts_tablet_manager.cc:1403] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.117s	user 0.019s	sys 0.006s
I20260709 12:31:16.023989 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:16.024287 20456 raft_consensus.cc:2804] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:16.024899 20456 raft_consensus.cc:493] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:16.025318 20456 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:16.026373 20294 leader_election.cc:343] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:16.026672 20444 tablet_bootstrap.cc:654] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.026997 20294 leader_election.cc:304] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7; no voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884
I20260709 12:31:16.027602 20482 raft_consensus.cc:515] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.031736 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.032595 20328 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.026077 20443 raft_consensus.cc:359] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.036978 20456 raft_consensus.cc:515] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.037503 20443 raft_consensus.cc:385] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.038131 20443 raft_consensus.cc:740] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.002787 20449 raft_consensus.cc:515] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.036274 20487 raft_consensus.cc:2749] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:16.039340 20443 consensus_queue.cc:260] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.045612 20443 ts_tablet_manager.cc:1434] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:31:16.046931 20443 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:16.048818 20328 raft_consensus.cc:2468] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.049717 20445 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:16.050398 20445 ts_tablet_manager.cc:1403] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.001s
W20260709 12:31:16.051306 20368 leader_election.cc:343] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:16.052668 20369 leader_election.cc:304] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:16.054021 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.054255 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.054926 20254 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:16.059676 20368 leader_election.cc:343] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:16.061084 20449 leader_election.cc:290] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.061375 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa1bf79364874984b9d4160e6c4ef" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.061815 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6fa1bf79364874984b9d4160e6c4ef" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.062235 20403 raft_consensus.cc:3060] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.062528 20253 raft_consensus.cc:3060] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.039691 20482 leader_election.cc:290] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.064255 20452 raft_consensus.cc:2804] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:16.067782 20452 raft_consensus.cc:697] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.068840 20452 consensus_queue.cc:237] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.070966 20443 tablet_bootstrap.cc:654] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.074918 20456 leader_election.cc:290] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.077911 20445 raft_consensus.cc:359] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.078666 20445 raft_consensus.cc:385] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.078549 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.079320 20445 raft_consensus.cc:740] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.079689 20328 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.080299 20445 consensus_queue.cc:260] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.082230 20445 ts_tablet_manager.cc:1434] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
W20260709 12:31:16.088810 20292 consensus_peers.cc:597] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 -> Peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Couldn't send request to peer 0d452c6b77c24d3383c18bfecf46b553. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:16.096136 20253 raft_consensus.cc:2468] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:16.099568 20444 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:16.106377 20294 leader_election.cc:304] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.108312 20449 raft_consensus.cc:2804] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:16.109110 20449 raft_consensus.cc:697] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:16.109186 20328 raft_consensus.cc:2468] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.110472 20449 consensus_queue.cc:237] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.111459 20369 leader_election.cc:304] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: c8380535ade4453897c237ed66a87884
I20260709 12:31:16.112628 20456 raft_consensus.cc:2804] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:16.111565 20254 raft_consensus.cc:2468] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.113274 20456 raft_consensus.cc:697] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.114818 20456 consensus_queue.cc:237] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.100497 20444 ts_tablet_manager.cc:1403] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.016s
I20260709 12:31:16.121091 20444 raft_consensus.cc:359] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.122061 20444 raft_consensus.cc:385] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.122401 20444 raft_consensus.cc:740] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.124864 20444 consensus_queue.cc:260] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.127414 20443 tablet_bootstrap.cc:492] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:16.128288 20443 ts_tablet_manager.cc:1403] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.082s	user 0.024s	sys 0.010s
I20260709 12:31:16.127334 20150 catalog_manager.cc:5697] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:16.131295 20444 ts_tablet_manager.cc:1434] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:31:16.132654 20444 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:16.135991 20443 raft_consensus.cc:359] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.136976 20443 raft_consensus.cc:385] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.137334 20443 raft_consensus.cc:740] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.138373 20443 consensus_queue.cc:260] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.129308 20403 raft_consensus.cc:2468] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:16.141713 20444 tablet_bootstrap.cc:654] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.144761 20443 ts_tablet_manager.cc:1434] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:31:16.146340 20443 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:16.156903 20443 tablet_bootstrap.cc:654] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.161091 20152 catalog_manager.cc:5697] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:16.163169 20452 consensus_queue.cc:1048] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.167456 20444 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:16.167837 20443 tablet_bootstrap.cc:492] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:16.168161 20444 ts_tablet_manager.cc:1403] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.018s
I20260709 12:31:16.168488 20443 ts_tablet_manager.cc:1403] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.006s
I20260709 12:31:16.172298 20443 raft_consensus.cc:359] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.173305 20443 raft_consensus.cc:385] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.172899 20444 raft_consensus.cc:359] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.174278 20444 raft_consensus.cc:385] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.174764 20444 raft_consensus.cc:740] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.176064 20444 consensus_queue.cc:260] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.173765 20443 raft_consensus.cc:740] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.180051 20443 consensus_queue.cc:260] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.182665 20444 ts_tablet_manager.cc:1434] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 12:31:16.184195 20444 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:16.188086 20443 ts_tablet_manager.cc:1434] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:31:16.189409 20443 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:16.192358 20444 tablet_bootstrap.cc:654] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.197470 20443 tablet_bootstrap.cc:654] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:16.201447 20482 consensus_queue.cc:1048] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.180540 20151 catalog_manager.cc:5697] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:16.243709 20450 raft_consensus.cc:493] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:16.244690 20450 raft_consensus.cc:515] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.256610 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:16.258193 20328 raft_consensus.cc:2393] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8380535ade4453897c237ed66a87884 in current term 1: Already voted for candidate 0d452c6b77c24d3383c18bfecf46b553 in this term.
I20260709 12:31:16.259981 20456 raft_consensus.cc:493] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:16.261152 20456 raft_consensus.cc:515] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.264103 20450 leader_election.cc:290] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.264300 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:16.266270 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
W20260709 12:31:16.268563 20368 leader_election.cc:343] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:16.268941 20217 leader_election.cc:304] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8380535ade4453897c237ed66a87884; no voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:16.271050 20480 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.271454 20328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:16.272755 20328 raft_consensus.cc:2468] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:16.286375 20449 consensus_queue.cc:1048] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.293176 20369 leader_election.cc:304] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: c8380535ade4453897c237ed66a87884
I20260709 12:31:16.295027 20482 raft_consensus.cc:2804] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:16.295737 20482 raft_consensus.cc:493] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:16.296320 20482 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.299142 20480 raft_consensus.cc:2749] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:16.307375 20456 leader_election.cc:290] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.329249 20452 consensus_queue.cc:1048] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.332363 20452 raft_consensus.cc:493] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:16.332885 20482 raft_consensus.cc:515] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.332934 20443 tablet_bootstrap.cc:492] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:16.334463 20443 ts_tablet_manager.cc:1403] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.145s	user 0.073s	sys 0.042s
I20260709 12:31:16.333006 20452 raft_consensus.cc:515] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.337224 20452 leader_election.cc:290] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.338589 20452 consensus_queue.cc:1048] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.338152 20443 raft_consensus.cc:359] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.339753 20443 raft_consensus.cc:385] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.340284 20443 raft_consensus.cc:740] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.341459 20443 consensus_queue.cc:260] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.343092 20444 tablet_bootstrap.cc:492] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:16.343750 20444 ts_tablet_manager.cc:1403] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.160s	user 0.046s	sys 0.094s
I20260709 12:31:16.344555 20443 ts_tablet_manager.cc:1434] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:31:16.347289 20444 raft_consensus.cc:359] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.348261 20444 raft_consensus.cc:385] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:16.348585 20444 raft_consensus.cc:740] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:16.349521 20444 consensus_queue.cc:260] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.341305 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6036cf4951d448cac3b1e812244d02b" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:16.352689 20327 raft_consensus.cc:2468] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:16.339051 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6036cf4951d448cac3b1e812244d02b" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:16.354759 20254 raft_consensus.cc:2468] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:16.355679 20444 ts_tablet_manager.cc:1434] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:31:16.356554 20368 leader_election.cc:304] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.357565 20452 raft_consensus.cc:2804] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:16.358071 20452 raft_consensus.cc:493] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:16.358458 20452 raft_consensus.cc:3060] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.365396 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.366250 20254 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.369388 20452 raft_consensus.cc:515] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.373235 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6036cf4951d448cac3b1e812244d02b" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.373538 20452 leader_election.cc:290] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.374056 20253 raft_consensus.cc:3060] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.374881 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6036cf4951d448cac3b1e812244d02b" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.376166 20327 raft_consensus.cc:3060] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.380383 20482 leader_election.cc:290] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:16.382630 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.394085 20329 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.384763 20254 raft_consensus.cc:2468] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.402801 20368 leader_election.cc:304] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.387665 20327 raft_consensus.cc:2468] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.389153 20253 raft_consensus.cc:2468] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.404492 20329 raft_consensus.cc:2468] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:16.405136 20369 leader_election.cc:304] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:16.407354 20482 raft_consensus.cc:2804] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:16.411018 20452 raft_consensus.cc:2804] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:16.422492 20452 raft_consensus.cc:697] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.423448 20482 raft_consensus.cc:697] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.424016 20452 consensus_queue.cc:237] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.424649 20482 consensus_queue.cc:237] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.440210 20151 catalog_manager.cc:5697] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:16.458278 20151 catalog_manager.cc:5697] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:16.464177 20403 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:16.494160 20450 consensus_queue.cc:1048] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:16.509742 20450 consensus_queue.cc:1048] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.548426 20482 consensus_queue.cc:1048] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.553124 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469c697d0b89410a8601a811d219a886"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:16.553922 20403 raft_consensus.cc:606] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:16.562609 20482 consensus_queue.cc:1048] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.563966 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06577a91b394473e9e25580cb5321af8"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:16.564860 20403 raft_consensus.cc:606] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:16.574913 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:16.575699 20403 raft_consensus.cc:606] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:16.576328 20482 raft_consensus.cc:993] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.576941 20452 raft_consensus.cc:1081] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.578923 20254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "469c697d0b89410a8601a811d219a886"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50084
I20260709 12:31:16.579722 20254 raft_consensus.cc:493] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:16.580183 20254 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.590036 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:16.590440 20254 raft_consensus.cc:515] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.591212 20403 raft_consensus.cc:606] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:16.594636 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.594970 20254 leader_election.cc:290] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.595443 20329 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.596402 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469c697d0b89410a8601a811d219a886" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.597263 20402 raft_consensus.cc:3055] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:16.597640 20402 raft_consensus.cc:740] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.598615 20402 consensus_queue.cc:260] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.600476 20402 raft_consensus.cc:3060] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.604799 20329 raft_consensus.cc:2468] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.606462 20482 raft_consensus.cc:993] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.606750 20220 leader_election.cc:304] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.607211 20482 raft_consensus.cc:1081] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.607996 20513 raft_consensus.cc:2804] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:16.607959 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:16.608863 20513 raft_consensus.cc:697] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:16.609056 20403 raft_consensus.cc:606] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:16.609933 20513 consensus_queue.cc:237] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.610001 20254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06577a91b394473e9e25580cb5321af8"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50084
I20260709 12:31:16.611176 20254 raft_consensus.cc:493] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:16.611685 20254 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.618551 20402 raft_consensus.cc:2468] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.621070 20152 catalog_manager.cc:5697] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:16.624019 20254 raft_consensus.cc:515] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.626276 20254 leader_election.cc:290] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.627346 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.627491 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06577a91b394473e9e25580cb5321af8" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.628253 20329 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.628371 20402 raft_consensus.cc:3055] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:16.628862 20402 raft_consensus.cc:740] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.629798 20402 consensus_queue.cc:260] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.631276 20402 raft_consensus.cc:3060] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.636619 20329 raft_consensus.cc:2468] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.637305 20464 consensus_queue.cc:1048] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.638226 20220 leader_election.cc:304] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.639302 20513 raft_consensus.cc:2804] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:16.639919 20513 raft_consensus.cc:697] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:16.640841 20513 consensus_queue.cc:237] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.641553 20402 raft_consensus.cc:2468] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.658418 20449 consensus_queue.cc:1048] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.663208 20482 consensus_queue.cc:1048] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.652062 20152 catalog_manager.cc:5697] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:16.729295 20482 consensus_queue.cc:1048] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.747730 20482 raft_consensus.cc:993] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.748692 20506 raft_consensus.cc:1081] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:16.752379 20254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50084
I20260709 12:31:16.753558 20254 raft_consensus.cc:493] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:16.754172 20254 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.772696 20254 raft_consensus.cc:515] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.775902 20254 leader_election.cc:290] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.777472 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:16.778231 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7dd179b8b364dc6a77fba8d32af59da" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.778604 20329 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.778975 20403 raft_consensus.cc:3055] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:16.779356 20403 raft_consensus.cc:740] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.780874 20403 consensus_queue.cc:260] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.782814 20403 raft_consensus.cc:3060] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.792492 20329 raft_consensus.cc:2468] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.794247 20220 leader_election.cc:304] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.795387 20513 raft_consensus.cc:2804] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:16.795747 20403 raft_consensus.cc:2468] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:16.796060 20513 raft_consensus.cc:697] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:16.797508 20513 consensus_queue.cc:237] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.808130 20482 raft_consensus.cc:993] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:16.807186 20151 catalog_manager.cc:5697] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:16.808972 20482 raft_consensus.cc:1081] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:16.811296 20329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57502
I20260709 12:31:16.811935 20329 raft_consensus.cc:493] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:16.812353 20329 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.822031 20329 raft_consensus.cc:515] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.824278 20482 consensus_queue.cc:1048] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.824627 20329 leader_election.cc:290] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.826109 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.826988 20254 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.826344 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea30683a652f4c079ea1c7f9c35f6ead" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.828243 20403 raft_consensus.cc:3055] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:16.828671 20403 raft_consensus.cc:740] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.830353 20403 consensus_queue.cc:260] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.832819 20403 raft_consensus.cc:3060] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.841768 20254 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:16.844576 20294 leader_election.cc:304] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.846364 20464 raft_consensus.cc:2804] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:16.847115 20464 raft_consensus.cc:697] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:16.848156 20452 consensus_queue.cc:1048] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.848429 20464 consensus_queue.cc:237] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.865572 20152 catalog_manager.cc:5697] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:16.866106 20403 raft_consensus.cc:2468] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:16.933713 20506 consensus_queue.cc:1048] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:16.949153 20506 consensus_queue.cc:1048] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:16.959884 20452 raft_consensus.cc:993] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:16.960541 20506 raft_consensus.cc:1081] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:16.962869 20327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57502
I20260709 12:31:16.963578 20327 raft_consensus.cc:493] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:16.963976 20327 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.973407 20327 raft_consensus.cc:515] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.975764 20327 leader_election.cc:290] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:16.977192 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:16.977288 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b56bd5b1a67a4cb4985b3ac616570040" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:16.978138 20254 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.978791 20403 raft_consensus.cc:3055] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:16.979152 20403 raft_consensus.cc:740] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:16.980248 20403 consensus_queue.cc:260] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:16.981536 20403 raft_consensus.cc:3060] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:16.988193 20254 raft_consensus.cc:2468] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:16.989838 20294 leader_election.cc:304] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:16.990887 20464 raft_consensus.cc:2804] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:16.991446 20403 raft_consensus.cc:2468] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:16.991701 20464 raft_consensus.cc:697] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:16.993296 20464 consensus_queue.cc:237] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:17.001721 20152 catalog_manager.cc:5697] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:17.045060 20327 raft_consensus.cc:1275] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.046670 20513 consensus_queue.cc:1048] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:17.060050 20403 raft_consensus.cc:1275] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.062476 20513 consensus_queue.cc:1048] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.078766 20403 raft_consensus.cc:1275] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.080945 20513 consensus_queue.cc:1048] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.093940 20327 raft_consensus.cc:1275] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.095893 20450 consensus_queue.cc:1048] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.303915 20254 raft_consensus.cc:1275] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.305795 20464 consensus_queue.cc:1048] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:17.316671 20403 raft_consensus.cc:1275] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.319197 20464 consensus_queue.cc:1048] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.355104 20327 raft_consensus.cc:1275] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.357405 20450 consensus_queue.cc:1048] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:17.367837 20403 raft_consensus.cc:1275] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.370733 20513 consensus_queue.cc:1048] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.380782 20254 raft_consensus.cc:1275] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.382851 20464 consensus_queue.cc:1048] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:17.395931 20403 raft_consensus.cc:1275] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:17.398070 20449 consensus_queue.cc:1048] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:19.664048 20152 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58578:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:19.666575 20152 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:19.734396 20236 tablet_service.cc:1511] Processing CreateTablet for tablet f8162708ac5a4022aa155d36674a3cd0 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:19.735435 20313 tablet_service.cc:1511] Processing CreateTablet for tablet f8162708ac5a4022aa155d36674a3cd0 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:19.736865 20236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8162708ac5a4022aa155d36674a3cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.737432 20313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8162708ac5a4022aa155d36674a3cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.738093 20389 tablet_service.cc:1511] Processing CreateTablet for tablet f8162708ac5a4022aa155d36674a3cd0 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:19.740175 20389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8162708ac5a4022aa155d36674a3cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.742030 20239 tablet_service.cc:1511] Processing CreateTablet for tablet 629a31e0e0ad48ef9df8ec525def7cb8 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:19.744047 20239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629a31e0e0ad48ef9df8ec525def7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.747550 20388 tablet_service.cc:1511] Processing CreateTablet for tablet 629a31e0e0ad48ef9df8ec525def7cb8 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:19.747550 20314 tablet_service.cc:1511] Processing CreateTablet for tablet 629a31e0e0ad48ef9df8ec525def7cb8 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:19.749562 20388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629a31e0e0ad48ef9df8ec525def7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.749562 20314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629a31e0e0ad48ef9df8ec525def7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.754944 20243 tablet_service.cc:1511] Processing CreateTablet for tablet b4f9396dffa84a6697298e9e444d9914 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:19.757026 20243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f9396dffa84a6697298e9e444d9914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.756906 20316 tablet_service.cc:1511] Processing CreateTablet for tablet b4f9396dffa84a6697298e9e444d9914 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:19.758791 20316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f9396dffa84a6697298e9e444d9914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.760847 20387 tablet_service.cc:1511] Processing CreateTablet for tablet b4f9396dffa84a6697298e9e444d9914 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:19.763547 20387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f9396dffa84a6697298e9e444d9914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.773797 20241 tablet_service.cc:1511] Processing CreateTablet for tablet 9d05a7545c5c46b5b96177a5c1e42b66 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:19.775696 20241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d05a7545c5c46b5b96177a5c1e42b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.782176 20318 tablet_service.cc:1511] Processing CreateTablet for tablet 9d05a7545c5c46b5b96177a5c1e42b66 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:19.784412 20318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d05a7545c5c46b5b96177a5c1e42b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.791270 20386 tablet_service.cc:1511] Processing CreateTablet for tablet 9d05a7545c5c46b5b96177a5c1e42b66 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:19.793231 20386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d05a7545c5c46b5b96177a5c1e42b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.795984 20392 tablet_service.cc:1511] Processing CreateTablet for tablet 61a2bad011874fb6afc9ddc4baaa6c05 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:19.796494 20319 tablet_service.cc:1511] Processing CreateTablet for tablet 61a2bad011874fb6afc9ddc4baaa6c05 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:19.797879 20392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a2bad011874fb6afc9ddc4baaa6c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.798813 20319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a2bad011874fb6afc9ddc4baaa6c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.792742 20238 tablet_service.cc:1511] Processing CreateTablet for tablet 61a2bad011874fb6afc9ddc4baaa6c05 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:19.803047 20238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a2bad011874fb6afc9ddc4baaa6c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.809262 20312 tablet_service.cc:1511] Processing CreateTablet for tablet 165f2afbd9dc4b5ebc36110a8692c340 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:19.811168 20312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165f2afbd9dc4b5ebc36110a8692c340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.812309 20237 tablet_service.cc:1511] Processing CreateTablet for tablet 165f2afbd9dc4b5ebc36110a8692c340 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:19.814002 20237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165f2afbd9dc4b5ebc36110a8692c340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.841120 20390 tablet_service.cc:1511] Processing CreateTablet for tablet 165f2afbd9dc4b5ebc36110a8692c340 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:19.842963 20390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165f2afbd9dc4b5ebc36110a8692c340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.876355 20240 tablet_service.cc:1511] Processing CreateTablet for tablet 20f2b8675e7044f2b8901f87d7900977 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:19.878480 20240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f2b8675e7044f2b8901f87d7900977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.885214 20317 tablet_service.cc:1511] Processing CreateTablet for tablet 20f2b8675e7044f2b8901f87d7900977 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:19.887027 20317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f2b8675e7044f2b8901f87d7900977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.892208 20242 tablet_service.cc:1511] Processing CreateTablet for tablet b59a66e5956f41998bee5e4d84f0713b (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:19.894053 20242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59a66e5956f41998bee5e4d84f0713b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.897853 20315 tablet_service.cc:1511] Processing CreateTablet for tablet b59a66e5956f41998bee5e4d84f0713b (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:19.899780 20315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59a66e5956f41998bee5e4d84f0713b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:31:19.926761 20294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:40555, user_credentials={real_user=slave}} blocked reactor thread for 100920us
I20260709 12:31:19.892328 20544 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:19.936861 20392 tablet_service.cc:1511] Processing CreateTablet for tablet 20f2b8675e7044f2b8901f87d7900977 (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:19.943521 20388 tablet_service.cc:1511] Processing CreateTablet for tablet b59a66e5956f41998bee5e4d84f0713b (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:19.951808 20545 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:19.964874 20392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f2b8675e7044f2b8901f87d7900977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.971902 20388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59a66e5956f41998bee5e4d84f0713b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.981905 20243 tablet_service.cc:1511] Processing CreateTablet for tablet c1e6593a015f4abb8917944bf6431cde (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:19.983793 20243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e6593a015f4abb8917944bf6431cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:19.963069 20544 tablet_bootstrap.cc:654] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.010820 20390 tablet_service.cc:1511] Processing CreateTablet for tablet c1e6593a015f4abb8917944bf6431cde (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:20.013477 20390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e6593a015f4abb8917944bf6431cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:20.018081 20545 tablet_bootstrap.cc:654] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.020768 20547 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.015344 20315 tablet_service.cc:1511] Processing CreateTablet for tablet c1e6593a015f4abb8917944bf6431cde (DEFAULT_TABLE table=soft_deleted_table [id=4d5e01143baf4411937065ac14661254]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:20.023388 20544 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.024222 20544 ts_tablet_manager.cc:1403] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.132s	user 0.015s	sys 0.015s
I20260709 12:31:20.024741 20315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e6593a015f4abb8917944bf6431cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:20.035813 20544 raft_consensus.cc:359] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.036813 20544 raft_consensus.cc:385] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.037276 20544 raft_consensus.cc:740] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.038305 20544 consensus_queue.cc:260] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.039126 20547 tablet_bootstrap.cc:654] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.040508 20545 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.041215 20545 ts_tablet_manager.cc:1403] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.090s	user 0.012s	sys 0.004s
I20260709 12:31:20.042258 20544 ts_tablet_manager.cc:1434] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:31:20.043864 20544 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.045092 20545 raft_consensus.cc:359] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.046103 20545 raft_consensus.cc:385] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.046461 20545 raft_consensus.cc:740] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.047403 20545 consensus_queue.cc:260] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.050972 20545 ts_tablet_manager.cc:1434] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:31:20.052323 20544 tablet_bootstrap.cc:654] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.052423 20545 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.061411 20545 tablet_bootstrap.cc:654] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.084645 20545 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.084831 20544 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.085014 20547 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.085407 20545 ts_tablet_manager.cc:1403] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.008s
I20260709 12:31:20.085837 20547 ts_tablet_manager.cc:1403] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.002s
I20260709 12:31:20.085991 20544 ts_tablet_manager.cc:1403] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.016s
I20260709 12:31:20.089469 20545 raft_consensus.cc:359] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.090371 20545 raft_consensus.cc:385] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.090225 20544 raft_consensus.cc:359] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.090111 20547 raft_consensus.cc:359] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.091241 20545 raft_consensus.cc:740] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.091467 20544 raft_consensus.cc:385] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.091554 20547 raft_consensus.cc:385] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.092155 20547 raft_consensus.cc:740] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.092278 20544 raft_consensus.cc:740] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.092330 20545 consensus_queue.cc:260] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.093422 20547 consensus_queue.cc:260] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.093389 20544 consensus_queue.cc:260] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.096889 20547 ts_tablet_manager.cc:1434] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.008s
I20260709 12:31:20.097348 20544 ts_tablet_manager.cc:1434] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:31:20.098336 20547 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.098554 20544 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.104645 20545 ts_tablet_manager.cc:1434] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:31:20.106789 20547 tablet_bootstrap.cc:654] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.106789 20544 tablet_bootstrap.cc:654] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.120754 20545 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.127606 20547 tablet_bootstrap.cc:492] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.128363 20547 ts_tablet_manager.cc:1403] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.011s
I20260709 12:31:20.129288 20544 tablet_bootstrap.cc:492] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.129774 20545 tablet_bootstrap.cc:654] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.130084 20544 ts_tablet_manager.cc:1403] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.007s
I20260709 12:31:20.132031 20547 raft_consensus.cc:359] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.132946 20547 raft_consensus.cc:385] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.133284 20547 raft_consensus.cc:740] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.134148 20544 raft_consensus.cc:359] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.134178 20547 consensus_queue.cc:260] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.135188 20544 raft_consensus.cc:385] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.135578 20544 raft_consensus.cc:740] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.135687 20545 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.136397 20545 ts_tablet_manager.cc:1403] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:31:20.136585 20544 consensus_queue.cc:260] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.137329 20547 ts_tablet_manager.cc:1434] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:31:20.138731 20547 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.139520 20544 ts_tablet_manager.cc:1434] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.006s
I20260709 12:31:20.140925 20544 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.140753 20545 raft_consensus.cc:359] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.141687 20545 raft_consensus.cc:385] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.142063 20545 raft_consensus.cc:740] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.143128 20545 consensus_queue.cc:260] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.147343 20547 tablet_bootstrap.cc:654] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.149376 20545 ts_tablet_manager.cc:1434] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:31:20.150187 20544 tablet_bootstrap.cc:654] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.150493 20487 raft_consensus.cc:493] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.150741 20545 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.151064 20487 raft_consensus.cc:515] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.163363 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.164551 20254 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.175432 20487 leader_election.cc:290] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.175745 20294 leader_election.cc:304] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.177199 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.178134 20402 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.181825 20543 raft_consensus.cc:2804] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.182462 20543 raft_consensus.cc:493] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.182854 20543 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.186859 20545 tablet_bootstrap.cc:654] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.190804 20543 raft_consensus.cc:515] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.200322 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.197540 20543 leader_election.cc:290] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.197573 20547 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.204702 20254 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.205390 20547 ts_tablet_manager.cc:1403] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.007s
I20260709 12:31:20.206897 20545 tablet_bootstrap.cc:492] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.201045 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:20.207573 20545 ts_tablet_manager.cc:1403] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.007s
I20260709 12:31:20.207949 20402 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.209249 20547 raft_consensus.cc:359] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.210108 20547 raft_consensus.cc:385] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.210422 20547 raft_consensus.cc:740] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.211361 20543 raft_consensus.cc:493] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.211591 20547 consensus_queue.cc:260] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.211897 20543 raft_consensus.cc:515] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.214462 20547 ts_tablet_manager.cc:1434] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:31:20.215734 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.216683 20253 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.217664 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.218313 20294 leader_election.cc:304] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.218523 20403 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.219575 20557 raft_consensus.cc:2804] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.220023 20557 raft_consensus.cc:493] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.220388 20557 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.215775 20543 leader_election.cc:290] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.221895 20544 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.222602 20544 ts_tablet_manager.cc:1403] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.018s
I20260709 12:31:20.226840 20547 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.229844 20402 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.215845 20545 raft_consensus.cc:359] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.233383 20292 leader_election.cc:304] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:20.235051 20545 raft_consensus.cc:385] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.240494 20545 raft_consensus.cc:740] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.241837 20557 raft_consensus.cc:515] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.239714 20544 raft_consensus.cc:359] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.243079 20545 consensus_queue.cc:260] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.244942 20487 raft_consensus.cc:2804] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.245582 20487 raft_consensus.cc:697] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:20.246518 20487 consensus_queue.cc:237] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.247093 20545 ts_tablet_manager.cc:1434] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.039s	user 0.011s	sys 0.000s
I20260709 12:31:20.247974 20544 raft_consensus.cc:385] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.221727 20254 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.248216 20513 raft_consensus.cc:493] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.248481 20544 raft_consensus.cc:740] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.249173 20545 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.248900 20513 raft_consensus.cc:515] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.252627 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.253525 20254 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.253345 20544 consensus_queue.cc:260] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.254726 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8162708ac5a4022aa155d36674a3cd0" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.255726 20327 raft_consensus.cc:2468] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 0.
I20260709 12:31:20.256722 20544 ts_tablet_manager.cc:1434] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 12:31:20.258033 20544 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.258404 20545 tablet_bootstrap.cc:654] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.259984 20513 leader_election.cc:290] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.260252 20557 leader_election.cc:290] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.262928 20220 leader_election.cc:304] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.264055 20547 tablet_bootstrap.cc:654] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.267561 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:20.268368 20402 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.270183 20542 raft_consensus.cc:2804] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.274325 20544 tablet_bootstrap.cc:654] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.281277 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8162708ac5a4022aa155d36674a3cd0" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.282439 20403 raft_consensus.cc:2468] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 0.
I20260709 12:31:20.270876 20542 raft_consensus.cc:493] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.285336 20542 raft_consensus.cc:3060] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.287508 20544 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.288371 20544 ts_tablet_manager.cc:1403] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.013s
I20260709 12:31:20.292889 20402 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.298687 20254 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.301098 20294 leader_election.cc:304] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.302146 20559 raft_consensus.cc:2804] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.302734 20559 raft_consensus.cc:697] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:20.303723 20559 consensus_queue.cc:237] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.301793 20544 raft_consensus.cc:359] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.315915 20544 raft_consensus.cc:385] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.316316 20544 raft_consensus.cc:740] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.317256 20544 consensus_queue.cc:260] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.323693 20547 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.324954 20547 ts_tablet_manager.cc:1403] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.001s
I20260709 12:31:20.328486 20542 raft_consensus.cc:515] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.328774 20547 raft_consensus.cc:359] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.329692 20547 raft_consensus.cc:385] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.330055 20547 raft_consensus.cc:740] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.330937 20547 consensus_queue.cc:260] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.332562 20545 tablet_bootstrap.cc:492] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.333276 20545 ts_tablet_manager.cc:1403] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.084s	user 0.037s	sys 0.035s
I20260709 12:31:20.333971 20547 ts_tablet_manager.cc:1434] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:31:20.334249 20150 catalog_manager.cc:5697] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:20.335546 20547 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.335745 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8162708ac5a4022aa155d36674a3cd0" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:20.336508 20327 raft_consensus.cc:3060] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.336932 20545 raft_consensus.cc:359] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.337839 20545 raft_consensus.cc:385] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.338191 20545 raft_consensus.cc:740] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.339565 20542 leader_election.cc:290] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.340541 20545 consensus_queue.cc:260] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.343477 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8162708ac5a4022aa155d36674a3cd0" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:20.343691 20530 raft_consensus.cc:493] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.345784 20545 ts_tablet_manager.cc:1434] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:31:20.346339 20530 raft_consensus.cc:515] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.347103 20545 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.352531 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.353369 20254 raft_consensus.cc:2468] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.355302 20368 leader_election.cc:304] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.356091 20547 tablet_bootstrap.cc:654] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.357450 20545 tablet_bootstrap.cc:654] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.345834 20402 raft_consensus.cc:3060] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.371090 20545 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.371966 20545 ts_tablet_manager.cc:1403] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.009s
I20260709 12:31:20.373371 20530 leader_election.cc:290] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.376401 20568 raft_consensus.cc:493] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.376998 20568 raft_consensus.cc:515] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.380313 20568 leader_election.cc:290] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.380510 20567 raft_consensus.cc:2804] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.381052 20567 raft_consensus.cc:493] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.376901 20545 raft_consensus.cc:359] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.381474 20567 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.381815 20545 raft_consensus.cc:385] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.382160 20545 raft_consensus.cc:740] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.383075 20545 consensus_queue.cc:260] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.385789 20545 ts_tablet_manager.cc:1434] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:31:20.387102 20545 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.392787 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.393909 20254 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.395002 20545 tablet_bootstrap.cc:654] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.396287 20368 leader_election.cc:304] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.398524 20568 raft_consensus.cc:2804] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.398973 20568 raft_consensus.cc:493] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.399394 20568 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.404383 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.405813 20542 raft_consensus.cc:493] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.406512 20542 raft_consensus.cc:515] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.409698 20568 raft_consensus.cc:515] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.410698 20542 leader_election.cc:290] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.410878 20327 raft_consensus.cc:2468] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 1.
I20260709 12:31:20.411139 20401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.412597 20568 leader_election.cc:290] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.412675 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.414624 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.415735 20254 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:20.413488 20217 leader_election.cc:343] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:31:20.416719 20220 leader_election.cc:343] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:20.417451 20220 leader_election.cc:304] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8380535ade4453897c237ed66a87884; no voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:20.419131 20513 raft_consensus.cc:2749] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:20.423552 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.424822 20329 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.426360 20220 leader_election.cc:304] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.427964 20542 raft_consensus.cc:2804] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.428828 20542 raft_consensus.cc:697] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:20.430339 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:20.431739 20329 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.436156 20402 raft_consensus.cc:2468] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 1.
I20260709 12:31:20.441887 20547 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.442824 20547 ts_tablet_manager.cc:1403] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.108s	user 0.029s	sys 0.017s
I20260709 12:31:20.447860 20547 raft_consensus.cc:359] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.448990 20547 raft_consensus.cc:385] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.449386 20547 raft_consensus.cc:740] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.441740 20254 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.451417 20368 leader_election.cc:304] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.452725 20568 raft_consensus.cc:2804] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.453365 20568 raft_consensus.cc:697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:20.454341 20568 consensus_queue.cc:237] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.462177 20547 consensus_queue.cc:260] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.430195 20542 consensus_queue.cc:237] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.466329 20547 ts_tablet_manager.cc:1434] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 12:31:20.468151 20547 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
W20260709 12:31:20.476034 20369 leader_election.cc:343] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:20.479868 20547 tablet_bootstrap.cc:654] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.481271 20152 catalog_manager.cc:5697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:20.494117 20544 ts_tablet_manager.cc:1434] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.205s	user 0.057s	sys 0.050s
I20260709 12:31:20.494771 20547 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.495481 20547 ts_tablet_manager.cc:1403] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.003s
I20260709 12:31:20.495889 20544 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.499835 20547 raft_consensus.cc:359] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.500855 20547 raft_consensus.cc:385] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.501251 20547 raft_consensus.cc:740] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.502389 20547 consensus_queue.cc:260] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.505571 20544 tablet_bootstrap.cc:654] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.505693 20547 ts_tablet_manager.cc:1434] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:31:20.511320 20547 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.519641 20545 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.520475 20545 ts_tablet_manager.cc:1403] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.134s	user 0.041s	sys 0.046s
I20260709 12:31:20.523440 20329 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.524660 20545 raft_consensus.cc:359] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.525924 20545 raft_consensus.cc:385] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.526248 20545 raft_consensus.cc:740] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.527338 20547 tablet_bootstrap.cc:654] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.527251 20545 consensus_queue.cc:260] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.528601 20151 catalog_manager.cc:5697] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:20.537900 20545 ts_tablet_manager.cc:1434] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:31:20.539356 20545 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.543025 20567 raft_consensus.cc:515] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.547282 20544 tablet_bootstrap.cc:492] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.547852 20545 tablet_bootstrap.cc:654] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.547984 20544 ts_tablet_manager.cc:1403] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.008s
I20260709 12:31:20.550381 20542 raft_consensus.cc:493] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.551024 20542 raft_consensus.cc:515] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.555570 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.555747 20542 leader_election.cc:290] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.556367 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.557083 20254 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:20.557592 20217 leader_election.cc:343] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:20.569458 20254 raft_consensus.cc:2468] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.571623 20368 leader_election.cc:304] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.573400 20566 raft_consensus.cc:2804] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.574224 20566 raft_consensus.cc:697] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:20.575549 20566 consensus_queue.cc:237] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
W20260709 12:31:20.580468 20200 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:20.584105 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.585405 20329 raft_consensus.cc:2468] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 0.
I20260709 12:31:20.587116 20220 leader_election.cc:304] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:20.588544 20542 raft_consensus.cc:2804] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.589002 20542 raft_consensus.cc:493] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.589363 20542 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.580313 20150 catalog_manager.cc:5697] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:20.551875 20544 raft_consensus.cc:359] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.593415 20544 raft_consensus.cc:385] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.593470 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
W20260709 12:31:20.603470 20369 leader_election.cc:343] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:20.597909 20547 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.609814 20547 ts_tablet_manager.cc:1403] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.099s	user 0.037s	sys 0.032s
I20260709 12:31:20.611073 20544 raft_consensus.cc:740] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.600600 20542 raft_consensus.cc:515] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.613272 20544 consensus_queue.cc:260] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.615089 20545 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.615811 20545 ts_tablet_manager.cc:1403] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.077s	user 0.027s	sys 0.027s
I20260709 12:31:20.615607 20547 raft_consensus.cc:359] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.616617 20547 raft_consensus.cc:385] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.616874 20544 ts_tablet_manager.cc:1434] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.000s
I20260709 12:31:20.616991 20547 raft_consensus.cc:740] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.620573 20544 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.619707 20545 raft_consensus.cc:359] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.621783 20545 raft_consensus.cc:385] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.622077 20545 raft_consensus.cc:740] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.601414 20567 leader_election.cc:290] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.623143 20545 consensus_queue.cc:260] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.625931 20545 ts_tablet_manager.cc:1434] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:31:20.625916 20542 leader_election.cc:290] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.626446 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:20.620829 20547 consensus_queue.cc:260] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.627535 20329 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.629817 20544 tablet_bootstrap.cc:654] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.630671 20545 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: Bootstrap starting.
I20260709 12:31:20.633188 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
W20260709 12:31:20.636009 20217 leader_election.cc:343] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:20.629698 20547 ts_tablet_manager.cc:1434] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:31:20.639127 20547 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.638643 20149 catalog_manager.cc:5697] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:20.644794 20329 raft_consensus.cc:2468] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 1.
I20260709 12:31:20.645394 20545 tablet_bootstrap.cc:654] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.647740 20547 tablet_bootstrap.cc:654] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.648996 20544 tablet_bootstrap.cc:492] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.649600 20544 ts_tablet_manager.cc:1403] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 12:31:20.650388 20220 leader_election.cc:304] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:20.652169 20542 raft_consensus.cc:2804] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.652748 20542 raft_consensus.cc:697] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:20.653209 20544 raft_consensus.cc:359] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.654106 20544 raft_consensus.cc:385] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.654452 20544 raft_consensus.cc:740] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.653648 20542 consensus_queue.cc:237] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.655571 20544 consensus_queue.cc:260] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.659282 20544 ts_tablet_manager.cc:1434] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:31:20.660641 20544 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.675822 20544 tablet_bootstrap.cc:654] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.682669 20545 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: No bootstrap required, opened a new log
I20260709 12:31:20.683303 20545 ts_tablet_manager.cc:1403] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: Time spent bootstrapping tablet: real 0.053s	user 0.007s	sys 0.012s
I20260709 12:31:20.683117 20544 tablet_bootstrap.cc:492] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.684127 20544 ts_tablet_manager.cc:1403] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:31:20.686492 20545 raft_consensus.cc:359] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.687412 20545 raft_consensus.cc:385] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.687748 20545 raft_consensus.cc:740] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.687107 20544 raft_consensus.cc:359] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.688248 20544 raft_consensus.cc:385] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.688622 20544 raft_consensus.cc:740] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.688683 20545 consensus_queue.cc:260] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.689553 20544 consensus_queue.cc:260] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.691781 20545 ts_tablet_manager.cc:1434] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:31:20.692550 20544 ts_tablet_manager.cc:1434] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:31:20.694024 20544 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Bootstrap starting.
I20260709 12:31:20.702538 20544 tablet_bootstrap.cc:654] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.702819 20151 catalog_manager.cc:5697] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 1 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:20.705546 20547 tablet_bootstrap.cc:492] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.706187 20547 ts_tablet_manager.cc:1403] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.067s	user 0.035s	sys 0.030s
I20260709 12:31:20.710595 20547 raft_consensus.cc:359] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.711766 20547 raft_consensus.cc:385] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.712169 20547 raft_consensus.cc:740] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.713388 20547 consensus_queue.cc:260] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.716886 20547 ts_tablet_manager.cc:1434] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.006s
I20260709 12:31:20.718426 20547 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: Bootstrap starting.
I20260709 12:31:20.725761 20544 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: No bootstrap required, opened a new log
I20260709 12:31:20.726626 20544 ts_tablet_manager.cc:1403] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.011s
I20260709 12:31:20.731436 20544 raft_consensus.cc:359] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.732342 20544 raft_consensus.cc:385] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.732717 20544 raft_consensus.cc:740] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.732813 20543 raft_consensus.cc:493] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.733549 20543 raft_consensus.cc:515] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.733819 20544 consensus_queue.cc:260] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.736287 20543 leader_election.cc:290] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.737852 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59a66e5956f41998bee5e4d84f0713b" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.738076 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59a66e5956f41998bee5e4d84f0713b" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:20.738695 20254 raft_consensus.cc:2468] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.738940 20402 raft_consensus.cc:2468] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 0.
I20260709 12:31:20.741070 20292 leader_election.cc:304] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:20.742065 20544 ts_tablet_manager.cc:1434] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:31:20.743746 20543 raft_consensus.cc:2804] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.744299 20543 raft_consensus.cc:493] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.744704 20543 raft_consensus.cc:3060] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.744830 20547 tablet_bootstrap.cc:654] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:20.745868 20487 consensus_queue.cc:1048] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:20.761868 20543 raft_consensus.cc:515] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.764657 20543 leader_election.cc:290] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:20.766657 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59a66e5956f41998bee5e4d84f0713b" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.767431 20254 raft_consensus.cc:3060] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.769533 20547 tablet_bootstrap.cc:492] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: No bootstrap required, opened a new log
I20260709 12:31:20.770161 20547 ts_tablet_manager.cc:1403] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.011s
I20260709 12:31:20.772486 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59a66e5956f41998bee5e4d84f0713b" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:20.773201 20487 consensus_queue.cc:1048] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:20.772521 20567 raft_consensus.cc:493] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.775633 20567 raft_consensus.cc:515] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.776914 20254 raft_consensus.cc:2468] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.778110 20567 leader_election.cc:290] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.780138 20294 leader_election.cc:304] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.773240 20402 raft_consensus.cc:3060] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.782335 20487 raft_consensus.cc:2804] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.782356 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.783828 20329 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.785939 20369 leader_election.cc:304] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:20.787331 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.788187 20253 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.789500 20567 raft_consensus.cc:2804] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.782897 20487 raft_consensus.cc:697] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:20.790077 20567 raft_consensus.cc:493] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.790601 20567 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.791085 20487 consensus_queue.cc:237] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.800760 20547 raft_consensus.cc:359] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.801740 20547 raft_consensus.cc:385] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:20.802037 20547 raft_consensus.cc:740] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Initialized, Role: FOLLOWER
I20260709 12:31:20.803045 20547 consensus_queue.cc:260] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.805531 20547 ts_tablet_manager.cc:1434] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:31:20.808494 20567 raft_consensus.cc:515] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.809938 20151 catalog_manager.cc:5697] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:20.812183 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.813041 20254 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.813835 20402 raft_consensus.cc:2468] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 1.
I20260709 12:31:20.817386 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:20.818286 20329 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.823433 20567 leader_election.cc:290] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.827740 20567 raft_consensus.cc:493] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.828555 20567 raft_consensus.cc:515] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.831113 20567 leader_election.cc:290] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.832661 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e6593a015f4abb8917944bf6431cde" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:20.834141 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e6593a015f4abb8917944bf6431cde" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:20.835680 20253 raft_consensus.cc:2468] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.835991 20327 raft_consensus.cc:2468] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 0.
I20260709 12:31:20.834514 20329 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.837437 20368 leader_election.cc:304] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.840054 20567 raft_consensus.cc:2804] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:20.840590 20567 raft_consensus.cc:493] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:20.840979 20567 raft_consensus.cc:3060] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.841666 20369 leader_election.cc:304] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:20.842782 20566 raft_consensus.cc:2804] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.843372 20566 raft_consensus.cc:697] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:20.843545 20254 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.844305 20566 consensus_queue.cc:237] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.849355 20567 raft_consensus.cc:515] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.852011 20567 leader_election.cc:290] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:20.852994 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e6593a015f4abb8917944bf6431cde" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:20.853484 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e6593a015f4abb8917944bf6431cde" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:20.853839 20254 raft_consensus.cc:3060] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.854208 20329 raft_consensus.cc:3060] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:20.855724 20148 catalog_manager.cc:5697] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:20.862282 20329 raft_consensus.cc:2468] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.862282 20254 raft_consensus.cc:2468] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 1.
I20260709 12:31:20.864511 20368 leader_election.cc:304] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:20.866387 20567 raft_consensus.cc:2804] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:20.867020 20567 raft_consensus.cc:697] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:20.868055 20567 consensus_queue.cc:237] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.877887 20148 catalog_manager.cc:5697] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:20.918007 20487 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:20.923939 20487 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:20.961244 20542 consensus_queue.cc:1048] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:20.976604 20542 consensus_queue.cc:1048] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:20.996795 20567 raft_consensus.cc:493] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:20.997498 20567 raft_consensus.cc:515] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:20.999833 20567 leader_election.cc:290] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:21.001359 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2372546982404aca8dd9280c3cef67d7" is_pre_election: true
I20260709 12:31:21.001443 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:21.002429 20329 raft_consensus.cc:2393] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d452c6b77c24d3383c18bfecf46b553 in current term 1: Already voted for candidate c8380535ade4453897c237ed66a87884 in this term.
I20260709 12:31:21.004122 20369 leader_election.cc:304] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553; no voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884
I20260709 12:31:21.005842 20567 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:21.009996 20566 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.015187 20567 raft_consensus.cc:2749] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:21.016582 20566 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.023597 20568 consensus_queue.cc:1048] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.026648 20329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:41862
I20260709 12:31:21.028936 20329 raft_consensus.cc:606] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:21.039839 20487 raft_consensus.cc:493] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:21.040544 20487 raft_consensus.cc:515] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.045670 20252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8380535ade4453897c237ed66a87884" is_pre_election: true
I20260709 12:31:21.055047 20487 leader_election.cc:290] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.061702 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d452c6b77c24d3383c18bfecf46b553" is_pre_election: true
I20260709 12:31:21.063668 20327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:41862
I20260709 12:31:21.064481 20327 raft_consensus.cc:606] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:21.064973 20329 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:21.066682 20292 leader_election.cc:304] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7; no voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884
I20260709 12:31:21.073577 20513 consensus_queue.cc:1048] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.076958 20542 consensus_queue.cc:1048] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.079620 20567 consensus_queue.cc:1048] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.081423 20543 raft_consensus.cc:2749] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:21.093868 20401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:21.094996 20401 raft_consensus.cc:606] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS c8380535ade4453897c237ed66a87884
I20260709 12:31:21.121938 20401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:21.122686 20401 raft_consensus.cc:606] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:21.139494 20403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
mode: GRACEFUL
new_leader_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57990
I20260709 12:31:21.140245 20403 raft_consensus.cc:606] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Received request to transfer leadership to TS 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:21.168797 20254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977"
dest_uuid: "c8380535ade4453897c237ed66a87884"
mode: GRACEFUL
new_leader_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:52598
I20260709 12:31:21.169411 20254 raft_consensus.cc:606] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Received request to transfer leadership to TS 2372546982404aca8dd9280c3cef67d7
I20260709 12:31:21.266176 20567 consensus_queue.cc:1048] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.280480 20567 consensus_queue.cc:1048] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.306146 20559 raft_consensus.cc:993] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.306720 20487 raft_consensus.cc:1081] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.308676 20254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50074
I20260709 12:31:21.309398 20254 raft_consensus.cc:493] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.309794 20254 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.315654 20254 raft_consensus.cc:515] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.317353 20254 leader_election.cc:290] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.318770 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:21.318802 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629a31e0e0ad48ef9df8ec525def7cb8" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.320155 20402 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.320160 20327 raft_consensus.cc:3055] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.320935 20327 raft_consensus.cc:740] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:21.322007 20327 consensus_queue.cc:260] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.323597 20327 raft_consensus.cc:3060] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.329962 20402 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.331897 20217 leader_election.cc:304] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:21.332700 20327 raft_consensus.cc:2468] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.332958 20600 raft_consensus.cc:2804] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.333582 20600 raft_consensus.cc:697] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:21.334616 20600 consensus_queue.cc:237] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.342630 20152 catalog_manager.cc:5697] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 2372546982404aca8dd9280c3cef67d7 (127.6.76.194) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.351701 20568 consensus_queue.cc:1048] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.365095 20566 consensus_queue.cc:1048] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.375929 20566 raft_consensus.cc:993] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.376510 20566 raft_consensus.cc:1081] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.378866 20327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57502
I20260709 12:31:21.379624 20327 raft_consensus.cc:493] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.380046 20327 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.386708 20327 raft_consensus.cc:515] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.388809 20327 leader_election.cc:290] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.390115 20402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.391312 20402 raft_consensus.cc:3055] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.391706 20402 raft_consensus.cc:740] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:21.391748 20254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165f2afbd9dc4b5ebc36110a8692c340" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:21.392547 20254 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.392889 20402 consensus_queue.cc:260] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.394469 20402 raft_consensus.cc:3060] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.401553 20254 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.402868 20402 raft_consensus.cc:2468] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.403399 20294 leader_election.cc:304] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:21.404439 20487 raft_consensus.cc:2804] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.404947 20487 raft_consensus.cc:697] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:21.405771 20487 consensus_queue.cc:237] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.411579 20487 consensus_queue.cc:1048] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:21.415096 20152 catalog_manager.cc:5697] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.421661 20557 consensus_queue.cc:1048] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:21.470544 20559 raft_consensus.cc:993] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.471145 20487 raft_consensus.cc:1081] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.473443 20254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50074
I20260709 12:31:21.473946 20254 raft_consensus.cc:493] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.474215 20254 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.479982 20254 raft_consensus.cc:515] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.481935 20254 leader_election.cc:290] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.483453 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:21.483453 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.484757 20403 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.484757 20327 raft_consensus.cc:3055] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.485464 20327 raft_consensus.cc:740] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:21.486739 20327 consensus_queue.cc:260] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.488927 20327 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.495070 20403 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.497110 20217 leader_election.cc:304] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:21.498205 20600 raft_consensus.cc:2804] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.499770 20600 raft_consensus.cc:697] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:21.500367 20566 raft_consensus.cc:993] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.500615 20327 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.501159 20566 raft_consensus.cc:1081] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer c8380535ade4453897c237ed66a87884 to start an election
I20260709 12:31:21.500834 20600 consensus_queue.cc:237] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.505053 20253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66"
dest_uuid: "c8380535ade4453897c237ed66a87884"
 from {username='slave'} at 127.0.0.1:50084
I20260709 12:31:21.505784 20253 raft_consensus.cc:493] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.506253 20253 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.510221 20148 catalog_manager.cc:5697] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 2372546982404aca8dd9280c3cef67d7 (127.6.76.194) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.515337 20253 raft_consensus.cc:515] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.518371 20253 leader_election.cc:290] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Requested vote from peers 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.518770 20327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:21.519586 20327 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.519698 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d05a7545c5c46b5b96177a5c1e42b66" candidate_uuid: "c8380535ade4453897c237ed66a87884" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.520486 20403 raft_consensus.cc:3055] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.520787 20403 raft_consensus.cc:740] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:21.521690 20403 consensus_queue.cc:260] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.522739 20403 raft_consensus.cc:3060] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.528162 20327 raft_consensus.cc:2468] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.528770 20403 raft_consensus.cc:2468] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8380535ade4453897c237ed66a87884 in term 2.
I20260709 12:31:21.529793 20220 leader_election.cc:304] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:21.530594 20600 raft_consensus.cc:2804] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.531213 20600 raft_consensus.cc:697] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Leader. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:21.532110 20600 consensus_queue.cc:237] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.541504 20148 catalog_manager.cc:5697] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to c8380535ade4453897c237ed66a87884 (127.6.76.193). New cstate: current_term: 2 leader_uuid: "c8380535ade4453897c237ed66a87884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.544039 20542 raft_consensus.cc:993] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: : Instructing follower 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.544703 20600 raft_consensus.cc:1081] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Signalling peer 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.546459 20327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57512
I20260709 12:31:21.547140 20327 raft_consensus.cc:493] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.547590 20327 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.555079 20327 raft_consensus.cc:515] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.556859 20327 leader_election.cc:290] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.558476 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:21.559473 20253 raft_consensus.cc:3055] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.560175 20253 raft_consensus.cc:740] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:21.558476 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20f2b8675e7044f2b8901f87d7900977" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.561383 20403 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.561117 20253 consensus_queue.cc:260] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.562412 20253 raft_consensus.cc:3060] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.567106 20403 raft_consensus.cc:2468] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.568557 20292 leader_election.cc:304] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:21.569042 20253 raft_consensus.cc:2468] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.569463 20487 raft_consensus.cc:2804] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.570076 20487 raft_consensus.cc:697] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:21.570966 20487 consensus_queue.cc:237] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.579912 20148 catalog_manager.cc:5697] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 1 to 2, leader changed from c8380535ade4453897c237ed66a87884 (127.6.76.193) to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.586968 20568 raft_consensus.cc:993] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: : Instructing follower 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.587633 20568 raft_consensus.cc:1081] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Signalling peer 2372546982404aca8dd9280c3cef67d7 to start an election
I20260709 12:31:21.589646 20327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
 from {username='slave'} at 127.0.0.1:57502
I20260709 12:31:21.590214 20327 raft_consensus.cc:493] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:21.590642 20327 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.599524 20327 raft_consensus.cc:515] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.602672 20253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:21.603367 20327 leader_election.cc:290] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195:34907)
I20260709 12:31:21.603933 20253 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.604736 20403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "2372546982404aca8dd9280c3cef67d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
I20260709 12:31:21.605623 20403 raft_consensus.cc:3055] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:21.606024 20403 raft_consensus.cc:740] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:21.606918 20403 consensus_queue.cc:260] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.608067 20403 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:21.614153 20403 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.614250 20253 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2372546982404aca8dd9280c3cef67d7 in term 2.
I20260709 12:31:21.615471 20292 leader_election.cc:304] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:21.616418 20487 raft_consensus.cc:2804] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:21.617010 20487 raft_consensus.cc:697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:21.617897 20487 consensus_queue.cc:237] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:21.626711 20148 catalog_manager.cc:5697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 reported cstate change: term changed from 1 to 2, leader changed from 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195) to 2372546982404aca8dd9280c3cef67d7 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "2372546982404aca8dd9280c3cef67d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:21.720772 20403 raft_consensus.cc:1275] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.722378 20600 consensus_queue.cc:1048] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:21.733860 20327 raft_consensus.cc:1275] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.735827 20617 consensus_queue.cc:1048] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:21.924985 20403 raft_consensus.cc:1275] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.926681 20513 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:21.939729 20327 raft_consensus.cc:1275] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.941712 20513 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:21.957645 20403 raft_consensus.cc:1275] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.959573 20487 consensus_queue.cc:1048] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:21.968945 20253 raft_consensus.cc:1275] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:21.970824 20559 consensus_queue.cc:1048] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.009512 20403 raft_consensus.cc:1275] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.011000 20513 consensus_queue.cc:1048] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.021850 20327 raft_consensus.cc:1275] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer c8380535ade4453897c237ed66a87884: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.023802 20600 consensus_queue.cc:1048] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.094738 20403 raft_consensus.cc:1275] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.096568 20559 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.109730 20254 raft_consensus.cc:1275] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.111212 20487 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.129684 20403 raft_consensus.cc:1275] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.131448 20557 consensus_queue.cc:1048] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:22.145972 20253 raft_consensus.cc:1275] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 2372546982404aca8dd9280c3cef67d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:22.148110 20559 consensus_queue.cc:1048] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:24.182688 20148 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58578:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:31:24.206581 20148 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58578:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:31:24.328027  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:24.342856 20648 tablet.cc:1724] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:24.348429 20220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:39449, user_credentials={real_user=slave}} blocked reactor thread for 47345us
W20260709 12:31:24.386919 20633 log.cc:927] Time spent T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.006s
W20260709 12:31:24.411331 20645 log.cc:927] Time spent T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:31:24.441715 20294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:40555, user_credentials={real_user=slave}} blocked reactor thread for 67084.9us
W20260709 12:31:24.478941 20280 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:629a31e0e0ad48ef9df8ec525def7cb8)
W20260709 12:31:24.480140 20642 log.cc:927] Time spent T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:31:24.478499 20220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.194:39449, user_credentials={real_user=slave}} blocked reactor thread for 87402.1us
W20260709 12:31:24.494122 20292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:34907, user_credentials={real_user=slave}} blocked reactor thread for 106012us
W20260709 12:31:24.534323 20217 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:34907, user_credentials={real_user=slave}} blocked reactor thread for 138985us
I20260709 12:31:24.536415 20666 tablet.cc:1724] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:24.530565 20652 log.cc:927] Time spent T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Append to log took a long time: real 0.156s	user 0.002s	sys 0.000s
W20260709 12:31:24.553474 20659 log.cc:927] Time spent T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
W20260709 12:31:24.601140 20280 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:629a31e0e0ad48ef9df8ec525def7cb8)
W20260709 12:31:24.604874 20294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:40555, user_credentials={real_user=slave}} blocked reactor thread for 51930.9us
W20260709 12:31:24.613507 20280 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:629a31e0e0ad48ef9df8ec525def7cb8)
W20260709 12:31:24.625187 20656 log.cc:927] Time spent T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 12:31:24.635684 20660 log.cc:927] Time spent T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:31:24.644032 20217 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:34907, user_credentials={real_user=slave}} blocked reactor thread for 96132.3us
I20260709 12:31:24.742152 20648 tablet.cc:1724] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.754384 20687 tablet.cc:1724] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.791996 20667 tablet.cc:1724] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.825361 20684 tablet.cc:1724] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:24.828289 20681 log.cc:927] Time spent T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:31:24.816169 20680 tablet.cc:1724] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.836488 20326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
mode: GRACEFUL
new_leader_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:41876
I20260709 12:31:24.837301 20326 raft_consensus.cc:606] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Received request to transfer leadership to TS 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:24.840685 20690 tablet.cc:1724] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:24.848522 20685 log.cc:927] Time spent T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Append to log took a long time: real 0.082s	user 0.003s	sys 0.000s
I20260709 12:31:24.818706 20683 tablet.cc:1724] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.848402 20689 tablet.cc:1724] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.870065 20692 tablet.cc:1724] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.876117 20669 raft_consensus.cc:993] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: : Instructing follower 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:24.876822 20669 raft_consensus.cc:1081] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Signalling peer 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:24.898926 20692 tablet.cc:1724] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.899721 20399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:57982
I20260709 12:31:24.902774 20399 raft_consensus.cc:493] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:24.903381 20399 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:24.836062 20691 tablet.cc:1724] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.935038 20648 tablet.cc:1724] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.941870 20148 catalog_manager.cc:5789] TS c8380535ade4453897c237ed66a87884 (127.6.76.193:40555) does not have the latest schema for tablet 9d05a7545c5c46b5b96177a5c1e42b66 (table soft_deleted_table [id=4d5e01143baf4411937065ac14661254]). Expected version 1 got 0
I20260709 12:31:24.943921 20666 tablet.cc:1724] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.949519 20684 tablet.cc:1724] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.950521 20151 catalog_manager.cc:5789] TS 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449) does not have the latest schema for tablet b59a66e5956f41998bee5e4d84f0713b (table soft_deleted_table [id=4d5e01143baf4411937065ac14661254]). Expected version 1 got 0
I20260709 12:31:24.946760 20667 tablet.cc:1724] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.953831 20151 catalog_manager.cc:5789] TS 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449) does not have the latest schema for tablet 20f2b8675e7044f2b8901f87d7900977 (table soft_deleted_table [id=4d5e01143baf4411937065ac14661254]). Expected version 1 got 0
I20260709 12:31:24.965642 20693 tablet.cc:1724] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.979599 20399 raft_consensus.cc:515] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:24.984099 20690 tablet.cc:1724] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.982551 20680 tablet.cc:1724] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.987932 20683 tablet.cc:1724] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:24.995894 20687 tablet.cc:1724] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.001765 20252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:25.002663 20252 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:25.011909 20399 leader_election.cc:290] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 3 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:25.014222 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a2bad011874fb6afc9ddc4baaa6c05" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:25.014837 20254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914"
dest_uuid: "c8380535ade4453897c237ed66a87884"
mode: GRACEFUL
new_leader_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:52602
I20260709 12:31:25.015143 20329 raft_consensus.cc:3055] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:25.015533 20254 raft_consensus.cc:606] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Received request to transfer leadership to TS 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:25.015682 20329 raft_consensus.cc:740] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:25.017156  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:31:25.017966  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:25.026140 20691 tablet.cc:1724] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.016552 20329 consensus_queue.cc:260] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.029217 20329 raft_consensus.cc:3060] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:25.029954 20252 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 3.
I20260709 12:31:25.029610 20680 tablet.cc:1724] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.034468 20689 tablet.cc:1724] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.039184 20368 leader_election.cc:304] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:25.040545 20566 raft_consensus.cc:2804] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:25.041146 20566 raft_consensus.cc:697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:25.042116 20566 consensus_queue.cc:237] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.054016 20252 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047"
dest_uuid: "c8380535ade4453897c237ed66a87884"
mode: GRACEFUL
new_leader_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:52602
I20260709 12:31:25.054898 20252 raft_consensus.cc:606] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Received request to transfer leadership to TS 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:25.060118 20329 raft_consensus.cc:2468] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 3.
I20260709 12:31:25.064507 20326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d"
dest_uuid: "2372546982404aca8dd9280c3cef67d7"
mode: GRACEFUL
new_leader_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:41876
I20260709 12:31:25.065235 20326 raft_consensus.cc:606] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Received request to transfer leadership to TS 0d452c6b77c24d3383c18bfecf46b553
I20260709 12:31:25.067329  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:31:25.068058  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:25.070999 20151 catalog_manager.cc:5697] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 2 to 3, leader changed from 2372546982404aca8dd9280c3cef67d7 (127.6.76.194) to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:25.080627 20691 tablet.cc:1724] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.083786 20689 tablet.cc:1724] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:25.086153 20665 raft_consensus.cc:993] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: : Instructing follower 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.086994 20665 raft_consensus.cc:1081] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Signalling peer 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.091573 20399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:31:25.092562 20399 raft_consensus.cc:493] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.093127 20399 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:25.110899 20399 raft_consensus.cc:515] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.111750 20152 catalog_manager.cc:6242] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: soft_deleted_table [id=4d5e01143baf4411937065ac14661254] alter complete (version 1)
I20260709 12:31:25.114427 20252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:25.115177 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f9396dffa84a6697298e9e444d9914" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:25.115996 20329 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:25.115316 20252 raft_consensus.cc:3055] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:25.117341 20252 raft_consensus.cc:740] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:25.118400 20252 consensus_queue.cc:260] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.119813 20252 raft_consensus.cc:3060] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:25.119997 20399 leader_election.cc:290] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 3 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:25.125015 20329 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 3.
I20260709 12:31:25.126556 20369 leader_election.cc:304] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7; no voters: 
I20260709 12:31:25.127635 20566 raft_consensus.cc:2804] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:25.128170 20566 raft_consensus.cc:697] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:25.129102 20252 raft_consensus.cc:2468] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 3.
I20260709 12:31:25.129055 20566 consensus_queue.cc:237] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.152498 20152 catalog_manager.cc:5697] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 2 to 3, leader changed from c8380535ade4453897c237ed66a87884 (127.6.76.193) to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:25.271309 20676 raft_consensus.cc:993] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: : Instructing follower 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.272147 20670 raft_consensus.cc:1081] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Signalling peer 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.274201 20399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:57982
I20260709 12:31:25.274974 20399 raft_consensus.cc:493] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.275429 20399 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.282112 20399 raft_consensus.cc:515] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.284282 20399 leader_election.cc:290] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:25.285686 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:25.285480 20252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b4aa4959e742839c0bfc9bc1473d1d" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:25.286465 20329 raft_consensus.cc:3055] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.286481 20252 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.287030 20329 raft_consensus.cc:740] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2372546982404aca8dd9280c3cef67d7, State: Running, Role: LEADER
I20260709 12:31:25.288048 20329 consensus_queue.cc:260] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.289619 20329 raft_consensus.cc:3060] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.297008 20252 raft_consensus.cc:2468] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 2.
I20260709 12:31:25.298688 20368 leader_election.cc:304] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:25.298960 20329 raft_consensus.cc:2468] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 2.
I20260709 12:31:25.300763 20566 raft_consensus.cc:2804] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.301436 20566 raft_consensus.cc:697] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 2 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:25.302313 20566 consensus_queue.cc:237] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.304226 20647 raft_consensus.cc:993] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: : Instructing follower 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.304996 20665 raft_consensus.cc:1081] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Signalling peer 0d452c6b77c24d3383c18bfecf46b553 to start an election
I20260709 12:31:25.307298 20399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047"
dest_uuid: "0d452c6b77c24d3383c18bfecf46b553"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:31:25.308054 20399 raft_consensus.cc:493] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.308530 20399 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.314306 20152 catalog_manager.cc:5697] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 1 to 2, leader changed from 2372546982404aca8dd9280c3cef67d7 (127.6.76.194) to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:25.318694 20399 raft_consensus.cc:515] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.321709 20399 leader_election.cc:290] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 2 election: Requested vote from peers c8380535ade4453897c237ed66a87884 (127.6.76.193:40555), 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449)
I20260709 12:31:25.321939 20252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8380535ade4453897c237ed66a87884"
I20260709 12:31:25.322410 20329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ffab388944f0d9a9a06066cf9c047" candidate_uuid: "0d452c6b77c24d3383c18bfecf46b553" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2372546982404aca8dd9280c3cef67d7"
I20260709 12:31:25.322995 20252 raft_consensus.cc:3055] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.323444 20252 raft_consensus.cc:740] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8380535ade4453897c237ed66a87884, State: Running, Role: LEADER
I20260709 12:31:25.324317 20252 consensus_queue.cc:260] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.325934 20252 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.322995 20329 raft_consensus.cc:3060] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.333823 20252 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 2.
I20260709 12:31:25.334136 20329 raft_consensus.cc:2468] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d452c6b77c24d3383c18bfecf46b553 in term 2.
I20260709 12:31:25.335623 20368 leader_election.cc:304] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d452c6b77c24d3383c18bfecf46b553, 2372546982404aca8dd9280c3cef67d7, c8380535ade4453897c237ed66a87884; no voters: 
I20260709 12:31:25.336833 20566 raft_consensus.cc:2804] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.337441 20566 raft_consensus.cc:697] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 LEADER]: Becoming Leader. State: Replica: 0d452c6b77c24d3383c18bfecf46b553, State: Running, Role: LEADER
I20260709 12:31:25.338438 20566 consensus_queue.cc:237] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } }
I20260709 12:31:25.347167 20152 catalog_manager.cc:5697] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 reported cstate change: term changed from 1 to 2, leader changed from c8380535ade4453897c237ed66a87884 (127.6.76.193) to 0d452c6b77c24d3383c18bfecf46b553 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "0d452c6b77c24d3383c18bfecf46b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d452c6b77c24d3383c18bfecf46b553" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:25.480463 20252 raft_consensus.cc:1275] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:25.482280 20566 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:25.492017 20329 raft_consensus.cc:1275] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:25.493788 20654 consensus_queue.cc:1048] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:25.567956 20329 raft_consensus.cc:1275] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:25.569588 20654 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:25.579833 20252 raft_consensus.cc:1275] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:25.581475 20654 consensus_queue.cc:1048] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:25.873363 20252 raft_consensus.cc:1275] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.875036 20654 consensus_queue.cc:1048] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.891285 20329 raft_consensus.cc:1275] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.893671 20654 consensus_queue.cc:1048] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.905977 20329 raft_consensus.cc:1275] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.907588 20654 consensus_queue.cc:1048] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2372546982404aca8dd9280c3cef67d7" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.920914 20252 raft_consensus.cc:1275] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Refusing update from remote peer 0d452c6b77c24d3383c18bfecf46b553: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.922578 20706 consensus_queue.cc:1048] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8380535ade4453897c237ed66a87884" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:26.069260  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:26.075376  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:26.075991  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:26.084098  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:26.084686  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:27.086166  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:27.095455  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:27.096086  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:27.101830  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:27.102558  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:28.104206  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:28.110515  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:28.111428  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:28.117575  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:28.118146  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:29.119575  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:29.125650  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:29.126371  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:29.131791  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:29.132408  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:30.133913  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:30.140218  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:30.140887  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:30.148600  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:30.149149  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:31.150370  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:31.155670  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:31.156533  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:31.162189  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:31.162684  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:32.163873  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:32.170768  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:32.171298  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:32.177691  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:32.178284  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:33.179608  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:33.185580  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:33.186185  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:33.191372  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:33.191897  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:34.193409  6451 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:34.200018  6451 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:34.200802  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:34.207507  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:34.208031  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:35.293037  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
W20260709 12:31:35.305970 20368 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 12:31:35.319443 20368 consensus_peers.cc:597] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:35.354740 20294 consensus_peers.cc:597] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.363538  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:35.364735  6451 tablet_replica.cc:333] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.365799  6451 raft_consensus.cc:2243] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.366746  6451 raft_consensus.cc:2272] T b59a66e5956f41998bee5e4d84f0713b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.370018  6451 tablet_replica.cc:333] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.371263  6451 raft_consensus.cc:2243] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.372179  6451 raft_consensus.cc:2272] T 20f2b8675e7044f2b8901f87d7900977 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.376490 20294 consensus_peers.cc:597] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.376523  6451 tablet_replica.cc:333] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.377924  6451 raft_consensus.cc:2243] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.378664  6451 raft_consensus.cc:2272] T c1e6593a015f4abb8917944bf6431cde P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.382097  6451 tablet_replica.cc:333] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.382891  6451 raft_consensus.cc:2243] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.383697  6451 raft_consensus.cc:2272] T 165f2afbd9dc4b5ebc36110a8692c340 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.386940  6451 tablet_replica.cc:333] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.387825  6451 raft_consensus.cc:2243] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.388512  6451 raft_consensus.cc:2272] T 61a2bad011874fb6afc9ddc4baaa6c05 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.391482  6451 tablet_replica.cc:333] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.392273  6451 raft_consensus.cc:2243] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.393488  6451 raft_consensus.cc:2272] T f8162708ac5a4022aa155d36674a3cd0 P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.397010  6451 tablet_replica.cc:333] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.397914  6451 raft_consensus.cc:2243] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.399114  6451 raft_consensus.cc:2272] T 629a31e0e0ad48ef9df8ec525def7cb8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.401906  6451 tablet_replica.cc:333] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.402637  6451 raft_consensus.cc:2243] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.403393  6451 raft_consensus.cc:2272] T a6036cf4951d448cac3b1e812244d02b P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.407596  6451 tablet_replica.cc:333] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.408538  6451 raft_consensus.cc:2243] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.409320  6451 raft_consensus.cc:2272] T b4f9396dffa84a6697298e9e444d9914 P c8380535ade4453897c237ed66a87884 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.409842 20294 consensus_peers.cc:597] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.412484  6451 tablet_replica.cc:333] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.413473  6451 raft_consensus.cc:2243] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.414011  6451 raft_consensus.cc:2272] T eb6fa1bf79364874984b9d4160e6c4ef P c8380535ade4453897c237ed66a87884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.418608  6451 tablet_replica.cc:333] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.419628  6451 raft_consensus.cc:2243] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.420337  6451 raft_consensus.cc:2272] T b56bd5b1a67a4cb4985b3ac616570040 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.423596  6451 tablet_replica.cc:333] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.424348  6451 raft_consensus.cc:2243] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.425817  6451 raft_consensus.cc:2272] T d7dd179b8b364dc6a77fba8d32af59da P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.430588  6451 tablet_replica.cc:333] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.431708  6451 raft_consensus.cc:2243] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.433023  6451 raft_consensus.cc:2272] T ea30683a652f4c079ea1c7f9c35f6ead P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.433087 20368 consensus_peers.cc:597] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:35.434926 20368 consensus_peers.cc:597] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.438301  6451 tablet_replica.cc:333] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884: stopping tablet replica
W20260709 12:31:35.438566 20368 consensus_peers.cc:597] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.439548  6451 raft_consensus.cc:2243] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.440776  6451 raft_consensus.cc:2272] T 06577a91b394473e9e25580cb5321af8 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.444090  6451 tablet_replica.cc:333] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.444849  6451 raft_consensus.cc:2243] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.446028  6451 raft_consensus.cc:2272] T 469c697d0b89410a8601a811d219a886 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.450022  6451 tablet_replica.cc:333] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.450779  6451 raft_consensus.cc:2243] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.452041  6451 raft_consensus.cc:2272] T 9d05a7545c5c46b5b96177a5c1e42b66 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.452210 20294 consensus_peers.cc:597] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.456173  6451 tablet_replica.cc:333] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.457230  6451 raft_consensus.cc:2243] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.458091  6451 raft_consensus.cc:2272] T 73b4aa4959e742839c0bfc9bc1473d1d P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.461493  6451 tablet_replica.cc:333] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884: stopping tablet replica
I20260709 12:31:35.462214  6451 raft_consensus.cc:2243] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.462837  6451 raft_consensus.cc:2272] T 025ffab388944f0d9a9a06066cf9c047 P c8380535ade4453897c237ed66a87884 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.468681 20294 consensus_peers.cc:597] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:35.481468 20294 consensus_peers.cc:597] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.522588  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
I20260709 12:31:35.578318  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
I20260709 12:31:35.625504  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:35.626473  6451 tablet_replica.cc:333] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.627624  6451 raft_consensus.cc:2243] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.628710  6451 raft_consensus.cc:2272] T b59a66e5956f41998bee5e4d84f0713b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.631501  6451 tablet_replica.cc:333] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.632315  6451 raft_consensus.cc:2243] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.633375  6451 raft_consensus.cc:2272] T 20f2b8675e7044f2b8901f87d7900977 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.636932  6451 tablet_replica.cc:333] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.637773  6451 raft_consensus.cc:2243] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.638499  6451 raft_consensus.cc:2272] T c1e6593a015f4abb8917944bf6431cde P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.641960  6451 tablet_replica.cc:333] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.643349  6451 raft_consensus.cc:2243] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.644827  6451 raft_consensus.cc:2272] T 165f2afbd9dc4b5ebc36110a8692c340 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.647249 20369 consensus_peers.cc:597] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 -> Peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Couldn't send request to peer 2372546982404aca8dd9280c3cef67d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.648540  6451 tablet_replica.cc:333] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.649303  6451 raft_consensus.cc:2243] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.649829  6451 raft_consensus.cc:2272] T 61a2bad011874fb6afc9ddc4baaa6c05 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.652654  6451 tablet_replica.cc:333] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.653326  6451 raft_consensus.cc:2243] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.653914  6451 raft_consensus.cc:2272] T f8162708ac5a4022aa155d36674a3cd0 P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.657648  6451 tablet_replica.cc:333] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.658535  6451 raft_consensus.cc:2243] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.659343  6451 raft_consensus.cc:2272] T 629a31e0e0ad48ef9df8ec525def7cb8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.662518  6451 tablet_replica.cc:333] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.663499  6451 raft_consensus.cc:2243] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.664253  6451 raft_consensus.cc:2272] T a6036cf4951d448cac3b1e812244d02b P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.668241  6451 tablet_replica.cc:333] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.669014  6451 raft_consensus.cc:2243] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.669618  6451 raft_consensus.cc:2272] T 025ffab388944f0d9a9a06066cf9c047 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.672367  6451 tablet_replica.cc:333] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.673120  6451 raft_consensus.cc:2243] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.673789  6451 raft_consensus.cc:2272] T 9d05a7545c5c46b5b96177a5c1e42b66 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.676345  6451 tablet_replica.cc:333] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.676965  6451 raft_consensus.cc:2243] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.677440  6451 raft_consensus.cc:2272] T 73b4aa4959e742839c0bfc9bc1473d1d P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.680497  6451 tablet_replica.cc:333] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.681340  6451 raft_consensus.cc:2243] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.681950  6451 raft_consensus.cc:2272] T 469c697d0b89410a8601a811d219a886 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.685199  6451 tablet_replica.cc:333] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.686097  6451 raft_consensus.cc:2243] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.687426  6451 raft_consensus.cc:2272] T eb6fa1bf79364874984b9d4160e6c4ef P 2372546982404aca8dd9280c3cef67d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.691016  6451 tablet_replica.cc:333] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.691944  6451 raft_consensus.cc:2243] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:31:35.692821 20368 consensus_peers.cc:597] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:40555: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.692906  6451 raft_consensus.cc:2272] T b4f9396dffa84a6697298e9e444d9914 P 2372546982404aca8dd9280c3cef67d7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.696786 20369 consensus_peers.cc:597] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 -> Peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Couldn't send request to peer 2372546982404aca8dd9280c3cef67d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.697247  6451 tablet_replica.cc:333] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.698033  6451 raft_consensus.cc:2243] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.698616  6451 raft_consensus.cc:2272] T 06577a91b394473e9e25580cb5321af8 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.701365  6451 tablet_replica.cc:333] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.702010  6451 raft_consensus.cc:2243] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.702550  6451 raft_consensus.cc:2272] T d7dd179b8b364dc6a77fba8d32af59da P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.705255  6451 tablet_replica.cc:333] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.705863  6451 raft_consensus.cc:2243] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.707310  6451 raft_consensus.cc:2272] T b56bd5b1a67a4cb4985b3ac616570040 P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.710605  6451 tablet_replica.cc:333] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7: stopping tablet replica
I20260709 12:31:35.711746  6451 raft_consensus.cc:2243] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.712901  6451 raft_consensus.cc:2272] T ea30683a652f4c079ea1c7f9c35f6ead P 2372546982404aca8dd9280c3cef67d7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:35.763900 20369 consensus_peers.cc:597] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 -> Peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Couldn't send request to peer 2372546982404aca8dd9280c3cef67d7. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:39449: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.765455  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
W20260709 12:31:35.776633 20368 consensus_peers.cc:597] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 -> Peer c8380535ade4453897c237ed66a87884 (127.6.76.193:40555): Couldn't send request to peer c8380535ade4453897c237ed66a87884. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:40555: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.816823  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
I20260709 12:31:35.862736  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:35.863873  6451 tablet_replica.cc:333] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
W20260709 12:31:35.864879 20369 consensus_peers.cc:597] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 -> Peer 2372546982404aca8dd9280c3cef67d7 (127.6.76.194:39449): Couldn't send request to peer 2372546982404aca8dd9280c3cef67d7. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:39449: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:35.865041  6451 raft_consensus.cc:2243] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.865890  6451 raft_consensus.cc:2272] T b59a66e5956f41998bee5e4d84f0713b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.868979  6451 tablet_replica.cc:333] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.869753  6451 raft_consensus.cc:2243] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.870388  6451 raft_consensus.cc:2272] T 20f2b8675e7044f2b8901f87d7900977 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.873174  6451 tablet_replica.cc:333] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.873886  6451 raft_consensus.cc:2243] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.874899  6451 raft_consensus.cc:2272] T c1e6593a015f4abb8917944bf6431cde P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.877528  6451 tablet_replica.cc:333] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.878198  6451 raft_consensus.cc:2243] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.878858  6451 raft_consensus.cc:2272] T 165f2afbd9dc4b5ebc36110a8692c340 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.881282  6451 tablet_replica.cc:333] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.881969  6451 raft_consensus.cc:2243] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.882946  6451 raft_consensus.cc:2272] T 61a2bad011874fb6afc9ddc4baaa6c05 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.885697  6451 tablet_replica.cc:333] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.886409  6451 raft_consensus.cc:2243] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.887050  6451 raft_consensus.cc:2272] T 629a31e0e0ad48ef9df8ec525def7cb8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.890005  6451 tablet_replica.cc:333] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.890952  6451 raft_consensus.cc:2243] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.892138  6451 raft_consensus.cc:2272] T a6036cf4951d448cac3b1e812244d02b P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.895434  6451 tablet_replica.cc:333] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.896176  6451 raft_consensus.cc:2243] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.897222  6451 raft_consensus.cc:2272] T b4f9396dffa84a6697298e9e444d9914 P 0d452c6b77c24d3383c18bfecf46b553 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.899799  6451 tablet_replica.cc:333] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.900520  6451 raft_consensus.cc:2243] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.901087  6451 raft_consensus.cc:2272] T eb6fa1bf79364874984b9d4160e6c4ef P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.903853  6451 tablet_replica.cc:333] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.904543  6451 raft_consensus.cc:2243] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.905157  6451 raft_consensus.cc:2272] T b56bd5b1a67a4cb4985b3ac616570040 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.907810  6451 tablet_replica.cc:333] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.908439  6451 raft_consensus.cc:2243] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.909000  6451 raft_consensus.cc:2272] T ea30683a652f4c079ea1c7f9c35f6ead P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.911602  6451 tablet_replica.cc:333] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.912242  6451 raft_consensus.cc:2243] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.912776  6451 raft_consensus.cc:2272] T d7dd179b8b364dc6a77fba8d32af59da P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.915148  6451 tablet_replica.cc:333] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.915853  6451 raft_consensus.cc:2243] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.916498  6451 raft_consensus.cc:2272] T 9d05a7545c5c46b5b96177a5c1e42b66 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.918916  6451 tablet_replica.cc:333] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.919970  6451 raft_consensus.cc:2243] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.921202  6451 raft_consensus.cc:2272] T 73b4aa4959e742839c0bfc9bc1473d1d P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.924011  6451 tablet_replica.cc:333] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.924718  6451 raft_consensus.cc:2243] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.925366  6451 raft_consensus.cc:2272] T 469c697d0b89410a8601a811d219a886 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.928094  6451 tablet_replica.cc:333] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.928748  6451 raft_consensus.cc:2243] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.929356  6451 raft_consensus.cc:2272] T f8162708ac5a4022aa155d36674a3cd0 P 0d452c6b77c24d3383c18bfecf46b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.931888  6451 tablet_replica.cc:333] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.932530  6451 raft_consensus.cc:2243] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:35.933511  6451 raft_consensus.cc:2272] T 025ffab388944f0d9a9a06066cf9c047 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.935928  6451 tablet_replica.cc:333] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553: stopping tablet replica
I20260709 12:31:35.936511  6451 raft_consensus.cc:2243] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:35.937044  6451 raft_consensus.cc:2272] T 06577a91b394473e9e25580cb5321af8 P 0d452c6b77c24d3383c18bfecf46b553 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:35.988863  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:31:36.041429  6451 master.cc:562] Master@127.6.76.254:39479 shutting down...
I20260709 12:31:36.074154  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:36.075044  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:36.075511  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P bf513f8ef2274dc89f5767178052c758: stopping tablet replica
I20260709 12:31:36.099414  6451 master.cc:584] Master@127.6.76.254:39479 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21831 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:31:36.163666  6451 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.76.254:37141
I20260709 12:31:36.165043  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:36.172369 20738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.172295 20739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.173003 20741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.174782  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:36.176530  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:36.176826  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:36.177038  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600296177011 us; error 0 us; skew 500 ppm
I20260709 12:31:36.177700  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:36.181033  6451 webserver.cc:533] Webserver started at http://127.6.76.254:44149/ using document root <none> and password file <none>
I20260709 12:31:36.181665  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:36.181921  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:36.182233  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:36.183897  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/master-0-root/instance:
uuid: "3de6dfcb5254489fbcc080689c2427a5"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.189373  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:31:36.193977 20746 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.195149  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:31:36.195544  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/master-0-root
uuid: "3de6dfcb5254489fbcc080689c2427a5"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.195843  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:36.210718  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:36.212164  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:36.262306  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.254:37141
I20260709 12:31:36.262441 20797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.254:37141 every 8 connection(s)
I20260709 12:31:36.267447 20798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:36.287019 20798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: Bootstrap starting.
I20260709 12:31:36.292870 20798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:36.299007 20798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: No bootstrap required, opened a new log
I20260709 12:31:36.302270 20798 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER }
I20260709 12:31:36.302791 20798 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:36.303095 20798 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de6dfcb5254489fbcc080689c2427a5, State: Initialized, Role: FOLLOWER
I20260709 12:31:36.303907 20798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER }
I20260709 12:31:36.304440 20798 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:31:36.304697 20798 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:31:36.305049 20798 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:36.310649 20798 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER }
I20260709 12:31:36.311363 20798 leader_election.cc:304] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3de6dfcb5254489fbcc080689c2427a5; no voters: 
I20260709 12:31:36.312798 20798 leader_election.cc:290] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:31:36.313346 20801 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:36.315114 20801 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 1 LEADER]: Becoming Leader. State: Replica: 3de6dfcb5254489fbcc080689c2427a5, State: Running, Role: LEADER
I20260709 12:31:36.316045 20801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER }
I20260709 12:31:36.316493 20798 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:31:36.322453 20802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3de6dfcb5254489fbcc080689c2427a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER } }
I20260709 12:31:36.323415 20802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:36.323706 20803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3de6dfcb5254489fbcc080689c2427a5. Latest consensus state: current_term: 1 leader_uuid: "3de6dfcb5254489fbcc080689c2427a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de6dfcb5254489fbcc080689c2427a5" member_type: VOTER } }
I20260709 12:31:36.324790 20803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:36.341701 20809 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:31:36.342794  6451 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:31:36.347949 20809 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:31:36.359336 20809 catalog_manager.cc:1383] Generated new cluster ID: 5d64df34dbeb4fd98883f7d73a4a469c
I20260709 12:31:36.359863 20809 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:31:36.375831 20809 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:31:36.377501 20809 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:31:36.394064 20809 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: Generated new TSK 0
I20260709 12:31:36.394857 20809 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:31:36.410943  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:36.418757 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.419308 20820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.423748 20822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.423875  6451 server_base.cc:1061] running on GCE node
I20260709 12:31:36.425251  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:36.425529  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:36.425724  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600296425702 us; error 0 us; skew 500 ppm
I20260709 12:31:36.426345  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:36.429173  6451 webserver.cc:533] Webserver started at http://127.6.76.193:45921/ using document root <none> and password file <none>
I20260709 12:31:36.429764  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:36.429996  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:36.430590  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:36.432245  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-0-root/instance:
uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.437772  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 12:31:36.442435 20827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.443760  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:36.444161  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-0-root
uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.444514  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:36.462949  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:36.464387  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:36.466418  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:36.469453  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:36.469748  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.470067  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:36.470510  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.522346  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.193:41987
I20260709 12:31:36.522492 20889 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.193:41987 every 8 connection(s)
I20260709 12:31:36.528613  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:36.538622 20894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.540776 20895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.545696 20890 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37141
I20260709 12:31:36.546482 20890 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:36.546833  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:36.546880 20897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.547667 20890 heartbeater.cc:507] Master 127.6.76.254:37141 requested a full tablet report, sending...
I20260709 12:31:36.548477  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:36.548805  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:36.549013  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600296548995 us; error 0 us; skew 500 ppm
I20260709 12:31:36.549955  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:36.550992 20763 ts_manager.cc:194] Registered new tserver with Master: f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:36.553431  6451 webserver.cc:533] Webserver started at http://127.6.76.194:46851/ using document root <none> and password file <none>
I20260709 12:31:36.554116  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:36.554286  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:36.554152 20763 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45316
I20260709 12:31:36.554898  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:36.556775  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-1-root/instance:
uuid: "fa4da9dd3d364a828c6b7a31784f7244"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.562516  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:31:36.567029 20902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.568306  6451 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:36.568684  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-1-root
uuid: "fa4da9dd3d364a828c6b7a31784f7244"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.569044  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:36.590943  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:36.592453  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:36.594197  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:36.597113  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:36.597429  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.597715  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:36.597922  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.650282  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.194:33421
I20260709 12:31:36.650409 20964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.194:33421 every 8 connection(s)
I20260709 12:31:36.656428  6451 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:36.664722 20968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:36.668587 20969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.668753  6451 server_base.cc:1061] running on GCE node
W20260709 12:31:36.670289 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:36.671942  6451 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:36.672325  6451 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:36.672593  6451 hybrid_clock.cc:648] HybridClock initialized: now 1783600296672561 us; error 0 us; skew 500 ppm
I20260709 12:31:36.673350  6451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:36.673516 20965 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37141
I20260709 12:31:36.674008 20965 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:36.675195 20965 heartbeater.cc:507] Master 127.6.76.254:37141 requested a full tablet report, sending...
I20260709 12:31:36.676787  6451 webserver.cc:533] Webserver started at http://127.6.76.195:36359/ using document root <none> and password file <none>
I20260709 12:31:36.677639  6451 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:36.677930  6451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:36.677839 20763 ts_manager.cc:194] Registered new tserver with Master: fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:36.678485  6451 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:36.680326 20763 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45320
I20260709 12:31:36.680505  6451 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-2-root/instance:
uuid: "fafc10eb3165402cb0d70a21c20cd235"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.686667  6451 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:31:36.691903 20976 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.693221  6451 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:31:36.693616  6451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-2-root
uuid: "fafc10eb3165402cb0d70a21c20cd235"
format_stamp: "Formatted at 2026-07-09 12:31:36 on dist-test-slave-n447"
I20260709 12:31:36.693938  6451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OvpZt/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599819042277-6451-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:36.714212  6451 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:36.715806  6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:36.717685  6451 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:36.720847  6451 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:36.721103  6451 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.721453  6451 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:36.721633  6451 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:36.772508  6451 rpc_server.cc:307] RPC server started. Bound to: 127.6.76.195:43415
I20260709 12:31:36.772828 21038 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.76.195:43415 every 8 connection(s)
I20260709 12:31:36.790984 21039 heartbeater.cc:344] Connected to a master server at 127.6.76.254:37141
I20260709 12:31:36.791688 21039 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:36.792685 21039 heartbeater.cc:507] Master 127.6.76.254:37141 requested a full tablet report, sending...
I20260709 12:31:36.795600 20763 ts_manager.cc:194] Registered new tserver with Master: fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:36.796294  6451 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019748106s
I20260709 12:31:36.798159 20763 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45322
I20260709 12:31:36.871635 20763 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45336:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:36.876716 20763 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:36.992071 21004 tablet_service.cc:1511] Processing CreateTablet for tablet 684198e026d847e8a2aa98460be581d7 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:36.993858 20853 tablet_service.cc:1511] Processing CreateTablet for tablet bc1ef7292abf405aac75bc7fdfc17c95 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:36.996341 20853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1ef7292abf405aac75bc7fdfc17c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:36.996487 21003 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd3283bf33d4b7d8f9c27dc654736f2 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:36.997519 20855 tablet_service.cc:1511] Processing CreateTablet for tablet 684198e026d847e8a2aa98460be581d7 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:36.999178 20855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684198e026d847e8a2aa98460be581d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.007925 20851 tablet_service.cc:1511] Processing CreateTablet for tablet 1384335a40084b168d9b7f8e34082e77 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:37.009932 20851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1384335a40084b168d9b7f8e34082e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.010730 20850 tablet_service.cc:1511] Processing CreateTablet for tablet 8d27db4c416c467c8d03ef16089e1ece (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:36.999818 21002 tablet_service.cc:1511] Processing CreateTablet for tablet bc1ef7292abf405aac75bc7fdfc17c95 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:37.000798 20854 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd3283bf33d4b7d8f9c27dc654736f2 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:37.004071 20930 tablet_service.cc:1511] Processing CreateTablet for tablet 684198e026d847e8a2aa98460be581d7 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:37.004652 20929 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd3283bf33d4b7d8f9c27dc654736f2 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:37.005810 20928 tablet_service.cc:1511] Processing CreateTablet for tablet bc1ef7292abf405aac75bc7fdfc17c95 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:37.006352 20926 tablet_service.cc:1511] Processing CreateTablet for tablet 1384335a40084b168d9b7f8e34082e77 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:37.006196 20927 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3dfde2374a4f13b32100863dac37e1 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:36.994481 21004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684198e026d847e8a2aa98460be581d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:36.994036 20852 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3dfde2374a4f13b32100863dac37e1 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:37.041517 20925 tablet_service.cc:1511] Processing CreateTablet for tablet 8d27db4c416c467c8d03ef16089e1ece (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:37.045018 20850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d27db4c416c467c8d03ef16089e1ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.047430 21001 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3dfde2374a4f13b32100863dac37e1 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:37.048833 21003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd3283bf33d4b7d8f9c27dc654736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.048902 21000 tablet_service.cc:1511] Processing CreateTablet for tablet 1384335a40084b168d9b7f8e34082e77 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:37.048177 20930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684198e026d847e8a2aa98460be581d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.051167 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1384335a40084b168d9b7f8e34082e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.052925 20852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3dfde2374a4f13b32100863dac37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.053944 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3dfde2374a4f13b32100863dac37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.058357 20854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd3283bf33d4b7d8f9c27dc654736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.058670 21002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1ef7292abf405aac75bc7fdfc17c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.067308 20925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d27db4c416c467c8d03ef16089e1ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.069167 20926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1384335a40084b168d9b7f8e34082e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.070158 20928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1ef7292abf405aac75bc7fdfc17c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.074443 20929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd3283bf33d4b7d8f9c27dc654736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.084299 20927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3dfde2374a4f13b32100863dac37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.097813 20999 tablet_service.cc:1511] Processing CreateTablet for tablet 8d27db4c416c467c8d03ef16089e1ece (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:37.100970 20999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d27db4c416c467c8d03ef16089e1ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.127009 20849 tablet_service.cc:1511] Processing CreateTablet for tablet 74c2f38bda214598879c9ed964510d3c (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:37.129333 20849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c2f38bda214598879c9ed964510d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.152202 20924 tablet_service.cc:1511] Processing CreateTablet for tablet 74c2f38bda214598879c9ed964510d3c (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:37.153903 20924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c2f38bda214598879c9ed964510d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.164553 21057 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.186625 21001 tablet_service.cc:1511] Processing CreateTablet for tablet 74c2f38bda214598879c9ed964510d3c (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:37.188534 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c2f38bda214598879c9ed964510d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.190260 20855 tablet_service.cc:1511] Processing CreateTablet for tablet 59a1a25f3bab4360937643f99c1ba522 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:37.193822 20855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a1a25f3bab4360937643f99c1ba522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.202518 20924 tablet_service.cc:1511] Processing CreateTablet for tablet 59a1a25f3bab4360937643f99c1ba522 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:37.203629 21058 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.204212 20924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a1a25f3bab4360937643f99c1ba522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.216030 21004 tablet_service.cc:1511] Processing CreateTablet for tablet 59a1a25f3bab4360937643f99c1ba522 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:37.218060 21004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a1a25f3bab4360937643f99c1ba522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.217661 20849 tablet_service.cc:1511] Processing CreateTablet for tablet e093ef11e3de47f4811c86f49d3c6d26 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:37.219617 20849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e093ef11e3de47f4811c86f49d3c6d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.236409 20923 tablet_service.cc:1511] Processing CreateTablet for tablet e093ef11e3de47f4811c86f49d3c6d26 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:37.238854 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e093ef11e3de47f4811c86f49d3c6d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.243145 21000 tablet_service.cc:1511] Processing CreateTablet for tablet e093ef11e3de47f4811c86f49d3c6d26 (DEFAULT_TABLE table=test-workload [id=226ee8e6e8694d94b41009083abc8215]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:37.245604 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e093ef11e3de47f4811c86f49d3c6d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:37.256261 21059 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.265406 21059 tablet_bootstrap.cc:654] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.267324 21057 tablet_bootstrap.cc:654] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.271165 21058 tablet_bootstrap.cc:654] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.291778 21058 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.292464 21058 ts_tablet_manager.cc:1403] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.089s	user 0.010s	sys 0.008s
I20260709 12:31:37.298853 21057 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.299746 21057 ts_tablet_manager.cc:1403] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.006s
I20260709 12:31:37.298807 21058 raft_consensus.cc:359] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.300203 21058 raft_consensus.cc:385] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.300701 21058 raft_consensus.cc:740] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.300977 21059 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.301612 21059 ts_tablet_manager.cc:1403] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.012s
I20260709 12:31:37.301829 21058 consensus_queue.cc:260] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.305419 21057 raft_consensus.cc:359] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.305737 20890 heartbeater.cc:499] Master 127.6.76.254:37141 was elected leader, sending a full tablet report...
I20260709 12:31:37.305701 21059 raft_consensus.cc:359] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.306746 21058 ts_tablet_manager.cc:1434] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:31:37.306805 21059 raft_consensus.cc:385] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.307272 21057 raft_consensus.cc:385] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.307500 21059 raft_consensus.cc:740] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.307811 21057 raft_consensus.cc:740] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.308480 21058 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.308570 21059 consensus_queue.cc:260] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.309075 21057 consensus_queue.cc:260] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.315644 21039 heartbeater.cc:499] Master 127.6.76.254:37141 was elected leader, sending a full tablet report...
I20260709 12:31:37.316749 21057 ts_tablet_manager.cc:1434] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 12:31:37.319897 21057 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.320298 21059 ts_tablet_manager.cc:1434] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.009s
I20260709 12:31:37.321929 21058 tablet_bootstrap.cc:654] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.324024 20965 heartbeater.cc:499] Master 127.6.76.254:37141 was elected leader, sending a full tablet report...
W20260709 12:31:37.332046 20811 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:37.332211 21059 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.339579 21057 tablet_bootstrap.cc:654] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.342199 21058 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.343134 21058 ts_tablet_manager.cc:1403] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.005s
I20260709 12:31:37.343858 21059 tablet_bootstrap.cc:654] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.347867 21058 raft_consensus.cc:359] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.348979 21058 raft_consensus.cc:385] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.350421 21058 raft_consensus.cc:740] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.350469 21064 raft_consensus.cc:493] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.351325 21064 raft_consensus.cc:515] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.351532 21058 consensus_queue.cc:260] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.354130 21058 ts_tablet_manager.cc:1434] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:31:37.356769 21058 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.365025 21058 tablet_bootstrap.cc:654] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.373219 21057 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.374269 21064 leader_election.cc:290] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.377405 21057 ts_tablet_manager.cc:1403] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.059s	user 0.006s	sys 0.029s
I20260709 12:31:37.384423 21057 raft_consensus.cc:359] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.391186 21057 raft_consensus.cc:385] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.391731 21057 raft_consensus.cc:740] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.394801 21058 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.397512 21058 ts_tablet_manager.cc:1403] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.013s
I20260709 12:31:37.398093 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1384335a40084b168d9b7f8e34082e77" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.399108 21014 raft_consensus.cc:2468] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.399901 21059 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.400571 21059 ts_tablet_manager.cc:1403] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.016s
I20260709 12:31:37.401626 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1384335a40084b168d9b7f8e34082e77" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.401978 20903 leader_election.cc:304] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:37.401588 21058 raft_consensus.cc:359] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.402918 21058 raft_consensus.cc:385] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.403386 21058 raft_consensus.cc:740] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.403439 21064 raft_consensus.cc:2804] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.403995 21064 raft_consensus.cc:493] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.404374 21064 raft_consensus.cc:3060] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.404340 21058 consensus_queue.cc:260] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.404366 21059 raft_consensus.cc:359] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.405742 21059 raft_consensus.cc:385] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:31:37.405999 20966 tablet.cc:2406] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:31:37.406121 21059 raft_consensus.cc:740] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.407637 21059 consensus_queue.cc:260] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.408013 21063 raft_consensus.cc:493] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.408880 21063 raft_consensus.cc:515] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.397449 21057 consensus_queue.cc:260] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
W20260709 12:31:37.411944 20904 leader_election.cc:343] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.415583 21057 ts_tablet_manager.cc:1434] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.002s
I20260709 12:31:37.417059 21057 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.419286 21064 raft_consensus.cc:515] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.423156 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1384335a40084b168d9b7f8e34082e77" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:37.424487 21063 leader_election.cc:290] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.425107 21057 tablet_bootstrap.cc:654] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
W20260709 12:31:37.425160 20904 leader_election.cc:343] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.425480 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1384335a40084b168d9b7f8e34082e77" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:37.429111 21014 raft_consensus.cc:3060] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.430043 21058 ts_tablet_manager.cc:1434] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 12:31:37.431512 21058 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.426124 21064 leader_election.cc:290] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.428063 21059 ts_tablet_manager.cc:1434] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.004s
I20260709 12:31:37.451963 21059 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.461870 21014 raft_consensus.cc:2468] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.462303 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.463812 21058 tablet_bootstrap.cc:654] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.466487 20903 leader_election.cc:304] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:37.468060 21064 raft_consensus.cc:2804] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.468719 21064 raft_consensus.cc:697] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:37.469885 21064 consensus_queue.cc:237] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
W20260709 12:31:37.470773 20828 leader_election.cc:343] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.482110 21058 tablet_bootstrap.cc:492] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.482901 21058 ts_tablet_manager.cc:1403] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.003s
I20260709 12:31:37.486577 21058 raft_consensus.cc:359] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.490358 21058 raft_consensus.cc:385] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.490805 21058 raft_consensus.cc:740] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.491930 21058 consensus_queue.cc:260] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.492769 21059 tablet_bootstrap.cc:654] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.493592 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.497565 21058 ts_tablet_manager.cc:1434] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:31:37.498899 21058 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
W20260709 12:31:37.500278 20828 leader_election.cc:343] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.497361 20763 catalog_manager.cc:5697] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:37.500885 20828 leader_election.cc:304] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae; no voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:37.501986 21057 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.502359 21063 raft_consensus.cc:2749] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:37.503193 21057 ts_tablet_manager.cc:1403] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.086s	user 0.029s	sys 0.048s
I20260709 12:31:37.509037 21057 raft_consensus.cc:359] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.510717 21057 raft_consensus.cc:385] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.511364 21057 raft_consensus.cc:740] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.511624 21058 tablet_bootstrap.cc:654] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.513384 21057 consensus_queue.cc:260] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.515666 21059 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.516482 21059 ts_tablet_manager.cc:1403] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.010s
I20260709 12:31:37.516535 21057 ts_tablet_manager.cc:1434] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 12:31:37.518465 21057 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.520617 21059 raft_consensus.cc:359] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.521972 21059 raft_consensus.cc:385] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.522452 21059 raft_consensus.cc:740] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.523645 21059 consensus_queue.cc:260] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.526754 21059 ts_tablet_manager.cc:1434] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:31:37.529939 21057 tablet_bootstrap.cc:654] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.528112 21059 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.541153 21059 tablet_bootstrap.cc:654] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.548542 21065 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.549382 21065 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.550206 21057 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.550866 21057 ts_tablet_manager.cc:1403] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.008s
I20260709 12:31:37.559444 21059 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.560444 21059 ts_tablet_manager.cc:1403] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.005s
I20260709 12:31:37.565310 21059 raft_consensus.cc:359] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.554733 21057 raft_consensus.cc:359] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.567111 21057 raft_consensus.cc:385] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.566116 21065 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:37.568068 21059 raft_consensus.cc:385] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.568514 21059 raft_consensus.cc:740] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.569984 21059 consensus_queue.cc:260] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.567582 21057 raft_consensus.cc:740] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.572625 21057 consensus_queue.cc:260] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.575758 21057 ts_tablet_manager.cc:1434] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.003s
I20260709 12:31:37.579847 21057 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.586108 21063 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.578088 21059 ts_tablet_manager.cc:1434] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 12:31:37.586901 21063 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.588354 21059 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.592603 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.594470 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.595409 20865 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
W20260709 12:31:37.595640 20828 leader_election.cc:343] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.596774 21059 tablet_bootstrap.cc:654] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.597540 20978 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:37.598790 21065 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.599196 21065 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.599660 21065 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.600055 21057 tablet_bootstrap.cc:654] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.599756 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.601414 21063 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.612906 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
W20260709 12:31:37.614913 20977 leader_election.cc:343] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.615379 21059 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.616504 21059 ts_tablet_manager.cc:1403] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 12:31:37.621515 21059 raft_consensus.cc:359] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.622888 21059 raft_consensus.cc:385] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.623435 21059 raft_consensus.cc:740] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.624766 21059 consensus_queue.cc:260] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.628991 21059 ts_tablet_manager.cc:1434] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:31:37.629979 21058 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.630779 21058 ts_tablet_manager.cc:1403] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.132s	user 0.049s	sys 0.055s
I20260709 12:31:37.631068 21059 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.634331 21064 raft_consensus.cc:493] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.634552 21058 raft_consensus.cc:359] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.635730 21058 raft_consensus.cc:385] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.635282 21064 raft_consensus.cc:515] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.637997 21065 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.639569 21014 raft_consensus.cc:2393] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46fed7fa12a4023b3c1302ae536a6ae in current term 1: Already voted for candidate fafc10eb3165402cb0d70a21c20cd235 in this term.
I20260709 12:31:37.640731 21065 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:37.636132 21058 raft_consensus.cc:740] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.641077 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.642547 21058 consensus_queue.cc:260] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.644275 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:37.645285 20865 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.646524 21058 ts_tablet_manager.cc:1434] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:31:37.649566 20828 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae; no voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:37.642732 21013 raft_consensus.cc:2468] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.652755 21057 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.652942 20903 leader_election.cc:304] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:37.653579 21058 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.653885 21057 ts_tablet_manager.cc:1403] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.040s
I20260709 12:31:37.654305 21073 raft_consensus.cc:2804] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.655059 21073 raft_consensus.cc:493] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.655539 21073 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.657074 21064 leader_election.cc:290] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.658913 21057 raft_consensus.cc:359] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.659572 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.660171 21057 raft_consensus.cc:385] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.660624 21057 raft_consensus.cc:740] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.661968 21057 consensus_queue.cc:260] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.663761 21058 tablet_bootstrap.cc:654] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
W20260709 12:31:37.671810 20904 leader_election.cc:343] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.681020 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:37.681208 21065 raft_consensus.cc:493] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.681823 20865 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:37.681957 21065 raft_consensus.cc:515] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.682658 21063 raft_consensus.cc:2749] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:37.683012 21073 raft_consensus.cc:515] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.684927 21058 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.685994 21058 ts_tablet_manager.cc:1403] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.011s
I20260709 12:31:37.686308 21073 leader_election.cc:290] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.686321 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
W20260709 12:31:37.687712 20977 leader_election.cc:343] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.688035 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:37.688866 21013 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:37.689566 20978 leader_election.cc:343] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.690543 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:37.690491 21058 raft_consensus.cc:359] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.691804 21058 raft_consensus.cc:385] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.692142 21058 raft_consensus.cc:740] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
W20260709 12:31:37.693034 20904 leader_election.cc:343] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.692981 21058 consensus_queue.cc:260] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.695664 21057 ts_tablet_manager.cc:1434] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.006s
I20260709 12:31:37.695823 21058 ts_tablet_manager.cc:1434] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 12:31:37.697610 21058 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.698740 21065 leader_election.cc:290] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:37.697676 21057 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.700169 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
W20260709 12:31:37.702286 20977 leader_election.cc:343] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.702942 20977 leader_election.cc:304] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:37.679003 21059 tablet_bootstrap.cc:654] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.707042 21058 tablet_bootstrap.cc:654] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.708243 21065 raft_consensus.cc:2749] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:37.709899 21057 tablet_bootstrap.cc:654] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.715124 20978 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:37.717051 21065 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.716931 21063 raft_consensus.cc:493] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.718842 21065 raft_consensus.cc:697] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:37.718314 21063 raft_consensus.cc:515] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.719950 21013 raft_consensus.cc:2468] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.720170 21065 consensus_queue.cc:237] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.724021 21063 leader_election.cc:290] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.724341 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.725514 21057 tablet_bootstrap.cc:492] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.721779 20903 leader_election.cc:304] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:37.726315 21057 ts_tablet_manager.cc:1403] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.004s
W20260709 12:31:37.726402 20828 leader_election.cc:343] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.726923 21073 raft_consensus.cc:2804] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.727604 21073 raft_consensus.cc:697] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:37.728591 21073 consensus_queue.cc:237] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.730611 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.731495 20940 raft_consensus.cc:2468] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:37.731397 21057 raft_consensus.cc:359] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.732534 21057 raft_consensus.cc:385] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.732872 21057 raft_consensus.cc:740] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.733800 21057 consensus_queue.cc:260] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.737211 20760 catalog_manager.cc:5697] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 0 to 1, leader changed from <none> to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:37.738358 21057 ts_tablet_manager.cc:1434] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:31:37.739598 21058 tablet_bootstrap.cc:492] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.740223 21057 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.740368 21058 ts_tablet_manager.cc:1403] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.002s
I20260709 12:31:37.741657 20828 leader_election.cc:304] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:37.742260 21059 tablet_bootstrap.cc:492] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.742933 21059 ts_tablet_manager.cc:1403] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.007s
I20260709 12:31:37.743654 21063 raft_consensus.cc:2804] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.744179 21063 raft_consensus.cc:493] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.744585 21063 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.744823 21058 raft_consensus.cc:359] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.745769 21058 raft_consensus.cc:385] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.746280 21058 raft_consensus.cc:740] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.747385 21059 raft_consensus.cc:359] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.749007 21057 tablet_bootstrap.cc:654] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.749289 21059 raft_consensus.cc:385] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.749661 21059 raft_consensus.cc:740] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.750618 21059 consensus_queue.cc:260] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.753729 21059 ts_tablet_manager.cc:1434] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:31:37.747375 21058 consensus_queue.cc:260] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.758080 21058 ts_tablet_manager.cc:1434] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:31:37.759881 21058 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.762463 21063 raft_consensus.cc:515] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.767509 21059 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.776379 20763 catalog_manager.cc:5697] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:37.777868 21057 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.771116 21058 tablet_bootstrap.cc:654] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.779623 21057 ts_tablet_manager.cc:1403] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.020s
I20260709 12:31:37.782158 21059 tablet_bootstrap.cc:654] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.784794 21063 leader_election.cc:290] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.786705 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:37.787186 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
W20260709 12:31:37.789131 20828 leader_election.cc:343] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:37.787618 20940 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.791113 21057 raft_consensus.cc:359] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.792393 21057 raft_consensus.cc:385] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.792863 21057 raft_consensus.cc:740] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.794243 21057 consensus_queue.cc:260] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.798578 21057 ts_tablet_manager.cc:1434] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:31:37.799686 20940 raft_consensus.cc:2468] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:37.800369 21057 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.801637 20828 leader_election.cc:304] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:37.803575 21073 raft_consensus.cc:493] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.804222 21073 raft_consensus.cc:515] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.809348 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.809423 21073 leader_election.cc:290] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.809656 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.803036 21063 raft_consensus.cc:2804] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.810186 21013 raft_consensus.cc:2468] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.810638 21063 raft_consensus.cc:697] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:37.810708 20865 raft_consensus.cc:2468] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.812009 21063 consensus_queue.cc:237] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.812858 20904 leader_election.cc:304] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:37.815457 21073 raft_consensus.cc:2804] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.818573 21073 raft_consensus.cc:493] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.819125 21073 raft_consensus.cc:3060] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.816139 21057 tablet_bootstrap.cc:654] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.824411 20763 catalog_manager.cc:5697] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:37.825791 21058 tablet_bootstrap.cc:492] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.826532 21058 ts_tablet_manager.cc:1403] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.015s
I20260709 12:31:37.826843 21057 tablet_bootstrap.cc:492] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.827528 21057 ts_tablet_manager.cc:1403] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.011s
I20260709 12:31:37.830660 21057 raft_consensus.cc:359] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.830153 21058 raft_consensus.cc:359] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.831560 21057 raft_consensus.cc:385] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.831827 21058 raft_consensus.cc:385] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.831985 21057 raft_consensus.cc:740] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.832392 21058 raft_consensus.cc:740] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.830214 21073 raft_consensus.cc:515] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.833019 21057 consensus_queue.cc:260] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.834379 21058 consensus_queue.cc:260] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.835685 21057 ts_tablet_manager.cc:1434] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:31:37.835834 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:37.836548 20865 raft_consensus.cc:3060] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.836817 21058 ts_tablet_manager.cc:1434] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:31:37.837956 21058 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:37.832593 21059 tablet_bootstrap.cc:492] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.839491 21059 ts_tablet_manager.cc:1403] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.005s
I20260709 12:31:37.847002 21058 tablet_bootstrap.cc:654] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.847515 20865 raft_consensus.cc:2468] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.841333 21057 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:37.849413 20904 leader_election.cc:304] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:37.850656 21064 raft_consensus.cc:2804] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.855974 21059 raft_consensus.cc:359] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.858136 21073 leader_election.cc:290] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.858661 21059 raft_consensus.cc:385] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.859115 21059 raft_consensus.cc:740] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.859196 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d27db4c416c467c8d03ef16089e1ece" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:37.860267 21013 raft_consensus.cc:3060] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.857990 21057 tablet_bootstrap.cc:654] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.861268 21064 raft_consensus.cc:697] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:37.864990 21059 consensus_queue.cc:260] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:37.865875 21064 consensus_queue.cc:237] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.872170 21059 ts_tablet_manager.cc:1434] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:31:37.872588 21073 consensus_queue.cc:1048] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:37.873723 21059 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.874838 21057 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:37.875555 21057 ts_tablet_manager.cc:1403] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.000s
I20260709 12:31:37.878199 21013 raft_consensus.cc:2468] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.883116 21058 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:37.884032 21058 ts_tablet_manager.cc:1403] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.012s
I20260709 12:31:37.884744 21059 tablet_bootstrap.cc:654] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.878847 21057 raft_consensus.cc:359] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.885640 21057 raft_consensus.cc:385] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.886147 21057 raft_consensus.cc:740] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.887152 21057 consensus_queue.cc:260] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.887743 21058 raft_consensus.cc:359] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.888710 21058 raft_consensus.cc:385] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.889088 21058 raft_consensus.cc:740] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.890385 21057 ts_tablet_manager.cc:1434] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:31:37.890168 21058 consensus_queue.cc:260] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.893163 21058 ts_tablet_manager.cc:1434] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:31:37.891479 20763 catalog_manager.cc:5697] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:37.898039 21059 tablet_bootstrap.cc:492] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.898989 21059 ts_tablet_manager.cc:1403] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.007s
I20260709 12:31:37.898658 20865 raft_consensus.cc:3060] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.903014 21059 raft_consensus.cc:359] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.903932 21059 raft_consensus.cc:385] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.904289 21059 raft_consensus.cc:740] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.905131 21059 consensus_queue.cc:260] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.907552 21059 ts_tablet_manager.cc:1434] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:31:37.908824 21059 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:37.915594 21059 tablet_bootstrap.cc:654] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:37.920321 21064 raft_consensus.cc:493] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.921362 21064 raft_consensus.cc:515] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.924983 21073 consensus_queue.cc:1048] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:37.931654 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.931993 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.932524 20865 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.932888 21013 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:37.932538 21059 tablet_bootstrap.cc:492] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:37.934154 21059 ts_tablet_manager.cc:1403] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.012s
I20260709 12:31:37.934767 20903 leader_election.cc:304] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:37.926191 21064 leader_election.cc:290] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.937496 21064 raft_consensus.cc:2804] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.937938 21064 raft_consensus.cc:493] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.938334 21064 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.939532 21065 raft_consensus.cc:493] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.941025 21065 raft_consensus.cc:515] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.943671 21065 leader_election.cc:290] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:37.939457 21059 raft_consensus.cc:359] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.944748 21059 raft_consensus.cc:385] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:37.945549 21059 raft_consensus.cc:740] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:37.945876 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.947831 21059 consensus_queue.cc:260] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.952735 21059 ts_tablet_manager.cc:1434] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:31:37.954077 21063 raft_consensus.cc:493] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.954842 21063 raft_consensus.cc:515] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.958520 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.959715 20940 raft_consensus.cc:2393] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fafc10eb3165402cb0d70a21c20cd235 in current term 1: Already voted for candidate f46fed7fa12a4023b3c1302ae536a6ae in this term.
I20260709 12:31:37.959349 21064 raft_consensus.cc:515] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.961330 20977 leader_election.cc:304] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:37.962357 21064 leader_election.cc:290] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.963496 21065 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.964054 21063 leader_election.cc:290] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:37.964622 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:37.965335 20865 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.965812 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:37.966871 21014 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.968298 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.969156 20940 raft_consensus.cc:2468] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:37.971635 20828 leader_election.cc:304] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:37.973359 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:37.974202 21013 raft_consensus.cc:2468] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:37.975514 21063 raft_consensus.cc:2804] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.976490 21063 raft_consensus.cc:493] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:37.976871 21063 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.977593 21065 raft_consensus.cc:2749] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:37.982640 20865 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.983764 21065 raft_consensus.cc:493] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.984463 21065 raft_consensus.cc:515] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.987576 21065 leader_election.cc:290] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:37.987576 20904 leader_election.cc:304] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:37.989845 21064 raft_consensus.cc:2804] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:37.991496 21064 raft_consensus.cc:697] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:37.992571 21064 consensus_queue.cc:237] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.994942 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:37.995172 21014 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:37.992972 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:37.995864 20940 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:37.995733 21065 raft_consensus.cc:493] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:37.997212 20865 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:37.997752 20977 leader_election.cc:304] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:37.997534 21065 raft_consensus.cc:515] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:37.999003 21089 raft_consensus.cc:2804] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:37.999506 21089 raft_consensus.cc:493] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:38.000006 21089 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:37.999603 21063 raft_consensus.cc:515] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.001369 21065 leader_election.cc:290] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:38.002493 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:38.003130 21063 leader_election.cc:290] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.003422 20865 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:38.004485 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.005251 20940 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.005262 20978 leader_election.cc:304] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.011420 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:38.012595 21065 raft_consensus.cc:2804] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:38.012753 20939 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:38.013110 21065 raft_consensus.cc:493] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:38.013239 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.013578 21065 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.013985 21014 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.011305 21089 raft_consensus.cc:515] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.021441 20940 raft_consensus.cc:2468] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:38.021019 20761 catalog_manager.cc:5697] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.023144 21089 leader_election.cc:290] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:38.023206 20828 leader_election.cc:304] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:38.024813 21063 raft_consensus.cc:2804] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:38.024917 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.025453 21014 raft_consensus.cc:2468] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:38.025539 21063 raft_consensus.cc:697] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.025985 20865 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.027464 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.027199 21063 consensus_queue.cc:237] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.028172 20940 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.036088 20865 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:38.037894 20978 leader_election.cc:304] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.039026 21089 raft_consensus.cc:2804] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:38.027375 21065 raft_consensus.cc:515] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.043304 21065 leader_election.cc:290] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:38.045004 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.045801 20865 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.049124 20940 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:38.058844 20865 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:38.059420 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.060560 20940 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.060442 20761 catalog_manager.cc:5697] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.061156 21089 raft_consensus.cc:697] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.061057 20978 leader_election.cc:304] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.064456 21063 raft_consensus.cc:493] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:38.064193 21089 consensus_queue.cc:237] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.065768 21063 raft_consensus.cc:515] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.065210 21115 raft_consensus.cc:2804] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:38.068533 21115 raft_consensus.cc:697] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.071514 20940 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:38.071261 21115 consensus_queue.cc:237] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.074616 21063 leader_election.cc:290] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.076900 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:38.077970 21014 raft_consensus.cc:2393] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46fed7fa12a4023b3c1302ae536a6ae in current term 1: Already voted for candidate fa4da9dd3d364a828c6b7a31784f7244 in this term.
I20260709 12:31:38.080001 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:38.082683 20828 leader_election.cc:304] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae; no voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:38.083745 21063 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.083156 20761 catalog_manager.cc:5697] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 0 to 1, leader changed from <none> to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:38.091629 21063 raft_consensus.cc:2749] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:38.100080 20761 catalog_manager.cc:5697] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 0 to 1, leader changed from <none> to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:38.140285 20940 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:38.149504 21115 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.151949 21064 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:38.152611 21064 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:38.154743 21064 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:38.156812 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:38.155700 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:38.157696 20865 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:38.159456 20904 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:38.160436 21064 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:38.160912 21064 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fafc10eb3165402cb0d70a21c20cd235)
I20260709 12:31:38.161314 21064 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.166162 21089 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.178418 21064 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:38.180540 20940 raft_consensus.cc:1240] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Rejecting Update request from peer fafc10eb3165402cb0d70a21c20cd235 for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:38.182443 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.181806 21064 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Requested vote from peers fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:38.186877 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.186908 21089 consensus_queue.cc:1059] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.188485 21089 raft_consensus.cc:3055] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.188905 21089 raft_consensus.cc:740] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.189508 20904 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244; no voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:38.189989 21089 consensus_queue.cc:260] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:38.191535 21089 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.191756 21064 raft_consensus.cc:2749] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:31:38.205276 21014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684198e026d847e8a2aa98460be581d7"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:31:38.206140 21014 raft_consensus.cc:606] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:38.206513 21014 raft_consensus.cc:614] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:31:38.222620 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:36378
I20260709 12:31:38.223465 20940 raft_consensus.cc:606] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:38.231415 21014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:31:38.232040 21014 raft_consensus.cc:606] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:38.238286 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:36378
I20260709 12:31:38.239048 20940 raft_consensus.cc:606] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:38.239358 21064 consensus_queue.cc:1048] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:38.248739 21073 raft_consensus.cc:993] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.249511 21102 raft_consensus.cc:1081] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.252205 20865 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41716
I20260709 12:31:38.253605 20865 raft_consensus.cc:493] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.254055 20865 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.257246 21064 consensus_queue.cc:1048] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.263720 20865 raft_consensus.cc:515] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.265090 20864 raft_consensus.cc:1240] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Rejecting Update request from peer fa4da9dd3d364a828c6b7a31784f7244 for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:38.269189 21064 consensus_queue.cc:1059] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.272353 20865 leader_election.cc:290] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.272989 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.275048 21073 consensus_queue.cc:1048] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.277813 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3dfde2374a4f13b32100863dac37e1" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.278802 21014 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.278671 20940 raft_consensus.cc:3055] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.280934 20940 raft_consensus.cc:740] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:38.283169 20940 consensus_queue.cc:260] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.284863 20940 raft_consensus.cc:3060] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.290812 21014 raft_consensus.cc:2468] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:38.291540 21107 consensus_queue.cc:1048] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.292503 20828 leader_election.cc:304] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.294077 21063 raft_consensus.cc:2804] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.294757 21063 raft_consensus.cc:697] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.295713 21063 consensus_queue.cc:237] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.305135 20865 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
mode: GRACEFUL
new_leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:41746
I20260709 12:31:38.305949 20865 raft_consensus.cc:606] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Received request to transfer leadership to TS fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:38.312494 20940 raft_consensus.cc:2468] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
W20260709 12:31:38.313333 21102 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:31:38.314793 21014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
mode: GRACEFUL
new_leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:31:38.315644 21014 raft_consensus.cc:606] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Received request to transfer leadership to TS fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:38.320261 20763 catalog_manager.cc:5697] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.329128 20864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
mode: GRACEFUL
new_leader_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:41746
I20260709 12:31:38.329916 20864 raft_consensus.cc:606] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Received request to transfer leadership to TS fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:38.369942 21063 consensus_queue.cc:1048] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:38.383888 21063 consensus_queue.cc:1048] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:38.394658 21072 raft_consensus.cc:993] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: : Instructing follower fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:38.395673 21063 raft_consensus.cc:1081] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Signalling peer fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:38.398181 20940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:36356
I20260709 12:31:38.398913 20940 raft_consensus.cc:493] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.399904 20940 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.403087 21102 consensus_queue.cc:1048] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.407505 20940 raft_consensus.cc:515] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.411263 21064 raft_consensus.cc:993] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.411976 21073 raft_consensus.cc:1081] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.412101 20940 leader_election.cc:290] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.414713 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.415604 21013 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.415730 20865 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41716
I20260709 12:31:38.414816 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74c2f38bda214598879c9ed964510d3c" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.417398 20864 raft_consensus.cc:3055] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.416718 20865 raft_consensus.cc:493] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.417985 20865 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.421622 21073 consensus_queue.cc:1048] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.425850 20864 raft_consensus.cc:740] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.426122 21063 consensus_queue.cc:1048] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.427194 20864 consensus_queue.cc:260] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.429461 20864 raft_consensus.cc:3060] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.436451 21013 raft_consensus.cc:2468] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:38.442508 20903 leader_election.cc:304] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.443723 21073 raft_consensus.cc:2804] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.444336 21073 raft_consensus.cc:697] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:38.445402 21073 consensus_queue.cc:237] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.457020 21129 raft_consensus.cc:993] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: : Instructing follower fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:38.457899 21129 raft_consensus.cc:1081] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Signalling peer fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:38.460413 21014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:57780
I20260709 12:31:38.461197 21014 raft_consensus.cc:493] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.461634 21014 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.466915 21063 consensus_queue.cc:1048] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.463440 20864 raft_consensus.cc:2468] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:38.470180 21089 consensus_queue.cc:1048] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.437968 20865 raft_consensus.cc:515] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.472146 20863 raft_consensus.cc:1240] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Rejecting Update request from peer fa4da9dd3d364a828c6b7a31784f7244 for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:38.475255 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.476055 20940 raft_consensus.cc:3055] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.476467 20940 raft_consensus.cc:740] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:38.477238 20865 leader_election.cc:290] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.477541 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd3283bf33d4b7d8f9c27dc654736f2" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.478255 21013 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.484372 20761 catalog_manager.cc:5697] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 1 to 2, leader changed from f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193) to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.477749 20940 consensus_queue.cc:260] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.489480 21064 consensus_queue.cc:1059] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.492015 21013 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:38.493338 21014 raft_consensus.cc:515] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.494637 21012 raft_consensus.cc:1240] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Rejecting Update request from peer f46fed7fa12a4023b3c1302ae536a6ae for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:38.494859 20828 leader_election.cc:304] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:38.496991 21063 raft_consensus.cc:2804] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.497735 21063 raft_consensus.cc:697] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.499023 21063 consensus_queue.cc:237] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.500463 21014 leader_election.cc:290] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:38.502038 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.502803 20865 raft_consensus.cc:3055] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.503790 20865 raft_consensus.cc:740] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.504833 20865 consensus_queue.cc:260] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.505950 20865 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.499815 21089 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:31:38.515533 20760 catalog_manager.cc:5697] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.516784 20937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e093ef11e3de47f4811c86f49d3c6d26" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.520673 20865 raft_consensus.cc:2468] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 2.
I20260709 12:31:38.522437 21065 consensus_queue.cc:1048] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:38.523494 20940 raft_consensus.cc:3060] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.524199 20978 leader_election.cc:304] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:38.534209 20940 raft_consensus.cc:2468] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:38.539891 21065 raft_consensus.cc:2804] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.540664 21065 raft_consensus.cc:697] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.541919 21065 consensus_queue.cc:237] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.569620 20760 catalog_manager.cc:5697] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 1 to 2, leader changed from f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193) to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:38.572841 21065 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:38.583562 21137 raft_consensus.cc:993] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.584188 21089 raft_consensus.cc:1081] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:38.586643 20865 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41732
I20260709 12:31:38.587412 20865 raft_consensus.cc:493] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.587846 20865 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.594175 21089 raft_consensus.cc:993] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: : Instructing follower fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:38.594965 21089 raft_consensus.cc:1081] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Signalling peer fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:38.597517 20939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:36370
I20260709 12:31:38.598435 20939 raft_consensus.cc:493] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:38.598968 20939 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.598690 20865 raft_consensus.cc:515] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.602581 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:38.603413 20940 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.608434 20865 leader_election.cc:290] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.610002 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1ef7292abf405aac75bc7fdfc17c95" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.610778 21013 raft_consensus.cc:3055] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.611112 21013 raft_consensus.cc:740] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.612043 21013 consensus_queue.cc:260] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.612591 20939 raft_consensus.cc:515] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.613407 21013 raft_consensus.cc:3060] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.614934 20940 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:38.616791 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:38.617236 20939 leader_election.cc:290] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:38.617110 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:38.617787 20865 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.616806 20828 leader_election.cc:304] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:38.618351 21014 raft_consensus.cc:3055] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:38.619257 21063 raft_consensus.cc:2804] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.619447 21014 raft_consensus.cc:740] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:38.619841 21063 raft_consensus.cc:697] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:38.620548 21014 consensus_queue.cc:260] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.620819 21063 consensus_queue.cc:237] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.622503 21014 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:38.622601 21013 raft_consensus.cc:2468] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:38.627817 20865 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:38.629336 20904 leader_election.cc:304] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:38.630421 21064 raft_consensus.cc:2804] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:38.630985 21064 raft_consensus.cc:697] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:38.631425 21014 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:38.632036 21064 consensus_queue.cc:237] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:38.631866 20763 catalog_manager.cc:5697] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.641667 20760 catalog_manager.cc:5697] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 1 to 2, leader changed from fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195) to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:38.753069 20939 raft_consensus.cc:1275] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:38.754886 21063 consensus_queue.cc:1048] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:38.762936 21014 raft_consensus.cc:1275] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:38.765764 21141 consensus_queue.cc:1048] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:38.929627 21013 raft_consensus.cc:1275] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:38.931545 21064 consensus_queue.cc:1048] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:38.943720 20865 raft_consensus.cc:1275] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:38.945778 21102 consensus_queue.cc:1048] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:39.003003 20939 raft_consensus.cc:1275] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.004575 21141 consensus_queue.cc:1048] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:39.014603 21013 raft_consensus.cc:1275] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.017417 21129 consensus_queue.cc:1048] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:39.044407 20939 raft_consensus.cc:3060] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:39.052395 20939 raft_consensus.cc:1275] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.054318 21065 consensus_queue.cc:1048] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:39.065920 20863 raft_consensus.cc:1275] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.068336 21138 consensus_queue.cc:1048] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:39.070329 20939 raft_consensus.cc:1275] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.076115 21129 consensus_queue.cc:1048] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:39.087572 21013 raft_consensus.cc:1275] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.089478 21129 consensus_queue.cc:1048] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:39.097787 20863 raft_consensus.cc:1275] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.099434 21102 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:39.110472 21013 raft_consensus.cc:1275] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:39.112722 21102 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:39.377470 21138 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:39.378093 21138 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:39.380558 21138 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:39.381686 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:39.381781 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:39.383632 20978 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:39.384557 21138 raft_consensus.cc:2749] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:31:39.672415 21129 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fafc10eb3165402cb0d70a21c20cd235)
I20260709 12:31:39.672932 21129 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:39.675402 21129 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:39.676419 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:39.676777 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:39.677510 20939 raft_consensus.cc:2393] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46fed7fa12a4023b3c1302ae536a6ae in current term 2: Already voted for candidate fa4da9dd3d364a828c6b7a31784f7244 in this term.
I20260709 12:31:39.677701 21013 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:39.679185 20828 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:39.680135 21129 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:31:39.680572 21129 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election (detected failure of leader fafc10eb3165402cb0d70a21c20cd235)
I20260709 12:31:39.680841 21129 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:39.687170 21129 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:39.689206 21129 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:39.690303 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:39.690420 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:39.691359 20939 raft_consensus.cc:2393] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f46fed7fa12a4023b3c1302ae536a6ae in current term 2: Already voted for candidate fa4da9dd3d364a828c6b7a31784f7244 in this term.
I20260709 12:31:39.697603 21013 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:39.699079 20828 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:39.700160 21129 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:39.701004 21129 raft_consensus.cc:697] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:39.702093 21129 consensus_queue.cc:237] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:39.713239 20760 catalog_manager.cc:5697] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:40.172441 20939 raft_consensus.cc:1275] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:40.174340 21129 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:40.188838 21013 raft_consensus.cc:1275] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:40.190781 21129 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:41.367887 20761 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47966:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:41.370457 20761 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:41.435585 20849 tablet_service.cc:1511] Processing CreateTablet for tablet ae1a8e410df043388bac1b5053cfb6a3 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:41.437147 20849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1a8e410df043388bac1b5053cfb6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.436748 20923 tablet_service.cc:1511] Processing CreateTablet for tablet ae1a8e410df043388bac1b5053cfb6a3 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:41.438386 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1a8e410df043388bac1b5053cfb6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.438563 21000 tablet_service.cc:1511] Processing CreateTablet for tablet ae1a8e410df043388bac1b5053cfb6a3 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:41.440275 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1a8e410df043388bac1b5053cfb6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.444293 21001 tablet_service.cc:1511] Processing CreateTablet for tablet 09d4c6ef82fe428d9e548d64e102f04f (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:41.446056 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d4c6ef82fe428d9e548d64e102f04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.447654 20854 tablet_service.cc:1511] Processing CreateTablet for tablet 09d4c6ef82fe428d9e548d64e102f04f (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:41.448311 20925 tablet_service.cc:1511] Processing CreateTablet for tablet 09d4c6ef82fe428d9e548d64e102f04f (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:41.449357 20854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d4c6ef82fe428d9e548d64e102f04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.450064 20925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d4c6ef82fe428d9e548d64e102f04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.469681 21166 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:41.471905 20852 tablet_service.cc:1511] Processing CreateTablet for tablet 964a87c057f5451cac346b58374b2c03 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:41.474110 20852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964a87c057f5451cac346b58374b2c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.480682 20924 tablet_service.cc:1511] Processing CreateTablet for tablet 964a87c057f5451cac346b58374b2c03 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:41.482400 20924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964a87c057f5451cac346b58374b2c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.495270 20850 tablet_service.cc:1511] Processing CreateTablet for tablet 0de23adb26e845518c999234e98db506 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:41.496974 20850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de23adb26e845518c999234e98db506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.500742 21166 tablet_bootstrap.cc:654] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.490060 21000 tablet_service.cc:1511] Processing CreateTablet for tablet 964a87c057f5451cac346b58374b2c03 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:41.508531 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964a87c057f5451cac346b58374b2c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.513703 21168 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.519295 21166 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:41.520000 21166 ts_tablet_manager.cc:1403] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.004s
I20260709 12:31:41.520488 21004 tablet_service.cc:1511] Processing CreateTablet for tablet 0de23adb26e845518c999234e98db506 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:41.522235 21004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de23adb26e845518c999234e98db506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.518889 20930 tablet_service.cc:1511] Processing CreateTablet for tablet 0de23adb26e845518c999234e98db506 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:41.524911 20930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de23adb26e845518c999234e98db506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.531919 20854 tablet_service.cc:1511] Processing CreateTablet for tablet c7a6d9c76dc34f70a705c1a3a8c30165 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:41.532052 21168 tablet_bootstrap.cc:654] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.533751 20854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a6d9c76dc34f70a705c1a3a8c30165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.536453 21167 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.538998 21166 raft_consensus.cc:359] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.540179 21166 raft_consensus.cc:385] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.540530 21166 raft_consensus.cc:740] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.547161 21166 consensus_queue.cc:260] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.548035 20923 tablet_service.cc:1511] Processing CreateTablet for tablet c7a6d9c76dc34f70a705c1a3a8c30165 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:41.555037 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a6d9c76dc34f70a705c1a3a8c30165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.558562 21167 tablet_bootstrap.cc:654] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.565521 21001 tablet_service.cc:1511] Processing CreateTablet for tablet c7a6d9c76dc34f70a705c1a3a8c30165 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:41.567404 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a6d9c76dc34f70a705c1a3a8c30165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.568559 21166 ts_tablet_manager.cc:1434] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.003s
I20260709 12:31:41.573678 21166 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:41.592052 21168 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.592826 21168 ts_tablet_manager.cc:1403] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.080s	user 0.017s	sys 0.000s
I20260709 12:31:41.593019 20850 tablet_service.cc:1511] Processing CreateTablet for tablet f4cf6deaf7dd4671bba915f364d3c307 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:41.594530 20850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cf6deaf7dd4671bba915f364d3c307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.596884 21168 raft_consensus.cc:359] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.598021 21168 raft_consensus.cc:385] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.598430 21168 raft_consensus.cc:740] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.600518 21166 tablet_bootstrap.cc:654] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.601501 21168 consensus_queue.cc:260] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.604525 21168 ts_tablet_manager.cc:1434] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:31:41.605880 21168 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.614434 21168 tablet_bootstrap.cc:654] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.619758 21167 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.620539 21167 ts_tablet_manager.cc:1403] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.084s	user 0.028s	sys 0.012s
I20260709 12:31:41.624384 21167 raft_consensus.cc:359] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.625401 21167 raft_consensus.cc:385] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.625835 21167 raft_consensus.cc:740] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.626747 21167 consensus_queue.cc:260] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.629932 21167 ts_tablet_manager.cc:1434] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:31:41.636612 21167 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.637259 20923 tablet_service.cc:1511] Processing CreateTablet for tablet f4cf6deaf7dd4671bba915f364d3c307 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:41.639401 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cf6deaf7dd4671bba915f364d3c307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.641566 21000 tablet_service.cc:1511] Processing CreateTablet for tablet f4cf6deaf7dd4671bba915f364d3c307 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:41.643680 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cf6deaf7dd4671bba915f364d3c307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.651825 21167 tablet_bootstrap.cc:654] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.654104 20924 tablet_service.cc:1511] Processing CreateTablet for tablet 8e46ffe3e22c4982865a3cf870217f52 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:41.656664 20924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e46ffe3e22c4982865a3cf870217f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.641569 20854 tablet_service.cc:1511] Processing CreateTablet for tablet 8e46ffe3e22c4982865a3cf870217f52 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:41.660941 20854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e46ffe3e22c4982865a3cf870217f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.668727 21167 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.669332 21168 tablet_bootstrap.cc:492] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.669497 21167 ts_tablet_manager.cc:1403] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.006s
I20260709 12:31:41.669956 21168 ts_tablet_manager.cc:1403] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.004s
I20260709 12:31:41.674993 21168 raft_consensus.cc:359] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.675987 21168 raft_consensus.cc:385] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.676421 21168 raft_consensus.cc:740] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.677448 21168 consensus_queue.cc:260] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.686812 20852 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3aba9102064e48bf859475ec69a37d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:41.686173 21004 tablet_service.cc:1511] Processing CreateTablet for tablet 8e46ffe3e22c4982865a3cf870217f52 (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:41.688475 20852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3aba9102064e48bf859475ec69a37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.688699 21004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e46ffe3e22c4982865a3cf870217f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.698964 20930 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3aba9102064e48bf859475ec69a37d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:41.700836 20930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3aba9102064e48bf859475ec69a37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.706002 21001 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3aba9102064e48bf859475ec69a37d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:41.707343 20854 tablet_service.cc:1511] Processing CreateTablet for tablet 8c49d46931024b6bbd42e5829a95f74d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:41.711757 20925 tablet_service.cc:1511] Processing CreateTablet for tablet 8c49d46931024b6bbd42e5829a95f74d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:41.713135 20854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c49d46931024b6bbd42e5829a95f74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.713383 20925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c49d46931024b6bbd42e5829a95f74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.713716 21002 tablet_service.cc:1511] Processing CreateTablet for tablet 8c49d46931024b6bbd42e5829a95f74d (DEFAULT_TABLE table=soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:41.709745 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3aba9102064e48bf859475ec69a37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.679320 21167 raft_consensus.cc:359] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.717262 21167 raft_consensus.cc:385] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.717633 21167 raft_consensus.cc:740] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.719004 21002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c49d46931024b6bbd42e5829a95f74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:41.719133 21168 ts_tablet_manager.cc:1434] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.004s
I20260709 12:31:41.720611 21168 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.699551 21166 tablet_bootstrap.cc:492] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:41.724798 21166 ts_tablet_manager.cc:1403] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.151s	user 0.020s	sys 0.012s
I20260709 12:31:41.724591 21167 consensus_queue.cc:260] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.727543 21167 ts_tablet_manager.cc:1434] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.000s
I20260709 12:31:41.728816 21167 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.739063 21168 tablet_bootstrap.cc:654] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.747826 21166 raft_consensus.cc:359] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.755494 21166 raft_consensus.cc:385] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.755630 21167 tablet_bootstrap.cc:654] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.766992 21166 raft_consensus.cc:740] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.771927 21166 consensus_queue.cc:260] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.775172 21166 ts_tablet_manager.cc:1434] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.005s
I20260709 12:31:41.787979 21166 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:41.800331 21168 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.801062 21168 ts_tablet_manager.cc:1403] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.039s
I20260709 12:31:41.802874 21166 tablet_bootstrap.cc:654] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.804984 21168 raft_consensus.cc:359] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.805971 21168 raft_consensus.cc:385] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.806327 21168 raft_consensus.cc:740] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.808205 21168 consensus_queue.cc:260] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.810428 21167 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.811518 21167 ts_tablet_manager.cc:1403] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.004s
I20260709 12:31:41.811697 21155 raft_consensus.cc:493] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:41.812369 21155 raft_consensus.cc:515] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.814806 21168 ts_tablet_manager.cc:1434] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:31:41.816838 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:41.816222 21167 raft_consensus.cc:359] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.817664 21167 raft_consensus.cc:385] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.817647 20863 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:41.818753 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:41.819623 21014 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:41.820194 20904 leader_election.cc:304] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:41.816264 21168 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.818789 21167 raft_consensus.cc:740] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.821363 21154 raft_consensus.cc:2804] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:41.821918 21154 raft_consensus.cc:493] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:41.822197 21167 consensus_queue.cc:260] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.825157 21167 ts_tablet_manager.cc:1434] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:31:41.822274 21154 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.826874 21167 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.817808 21155 leader_election.cc:290] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:41.834566 21168 tablet_bootstrap.cc:654] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.837445 21166 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:41.838249 21063 raft_consensus.cc:493] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:41.838308 21167 tablet_bootstrap.cc:654] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.839046 21063 raft_consensus.cc:515] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.847625 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
W20260709 12:31:41.849931 20828 leader_election.cc:343] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:41.850899 21154 raft_consensus.cc:515] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.860111 21168 tablet_bootstrap.cc:492] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.860725 21154 leader_election.cc:290] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:41.861267 21168 ts_tablet_manager.cc:1403] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.022s
I20260709 12:31:41.853410 21063 leader_election.cc:290] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:41.854475 21167 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.863413 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:41.863896 21167 ts_tablet_manager.cc:1403] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.004s
I20260709 12:31:41.864269 21013 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.864373 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:41.865711 20863 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.838307 21166 ts_tablet_manager.cc:1403] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.005s
I20260709 12:31:41.855167 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:41.868126 21168 raft_consensus.cc:359] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
W20260709 12:31:41.869016 20828 leader_election.cc:343] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:41.869791 20828 leader_election.cc:304] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae; no voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:41.871945 21167 raft_consensus.cc:359] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.873576 21167 raft_consensus.cc:385] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.874006 21167 raft_consensus.cc:740] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.869140 21168 raft_consensus.cc:385] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.874918 21167 consensus_queue.cc:260] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.875015 21168 raft_consensus.cc:740] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.877780 21167 ts_tablet_manager.cc:1434] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:31:41.878078 21013 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:41.871044 21166 raft_consensus.cc:359] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.878767 21168 consensus_queue.cc:260] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.880119 21063 raft_consensus.cc:2749] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:41.879531 21166 raft_consensus.cc:385] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.881127 21166 raft_consensus.cc:740] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.882604 21168 ts_tablet_manager.cc:1434] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.003s
I20260709 12:31:41.884088 21168 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.883296 20903 leader_election.cc:304] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:41.885128 21153 raft_consensus.cc:2804] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:41.882210 21166 consensus_queue.cc:260] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.889115 21166 ts_tablet_manager.cc:1434] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:31:41.890462 21166 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:41.892040 21168 tablet_bootstrap.cc:654] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.879031 21167 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.897794 21153 raft_consensus.cc:697] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:41.902300 21063 raft_consensus.cc:493] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:41.903056 21063 raft_consensus.cc:515] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.904356 21153 consensus_queue.cc:237] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.905864 21063 leader_election.cc:290] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:41.907619 21167 tablet_bootstrap.cc:654] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.909526 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964a87c057f5451cac346b58374b2c03" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:41.910794 20939 raft_consensus.cc:2468] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:41.905479 21166 tablet_bootstrap.cc:654] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.914165 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964a87c057f5451cac346b58374b2c03" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:41.915091 21014 raft_consensus.cc:2468] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:41.902204 20863 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:41.918936 20828 leader_election.cc:304] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:41.922444 21063 raft_consensus.cc:2804] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:41.922979 21063 raft_consensus.cc:493] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:41.923477 21063 raft_consensus.cc:3060] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.923825 21167 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.925451 21167 ts_tablet_manager.cc:1403] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.000s
I20260709 12:31:41.932305 21166 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:41.933475 21166 ts_tablet_manager.cc:1403] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.000s
I20260709 12:31:41.936547 21168 tablet_bootstrap.cc:492] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.937288 21168 ts_tablet_manager.cc:1403] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.054s	user 0.031s	sys 0.019s
I20260709 12:31:41.937608 21166 raft_consensus.cc:359] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.938637 21166 raft_consensus.cc:385] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.939646 21167 raft_consensus.cc:359] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.940618 21167 raft_consensus.cc:385] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.940974 21167 raft_consensus.cc:740] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.940232 20763 catalog_manager.cc:5697] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:41.941341 21168 raft_consensus.cc:359] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.938970 21166 raft_consensus.cc:740] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.942659 21168 raft_consensus.cc:385] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.941905 21167 consensus_queue.cc:260] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.942019 21063 raft_consensus.cc:515] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.943328 21168 raft_consensus.cc:740] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.944442 21168 consensus_queue.cc:260] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.946069 21167 ts_tablet_manager.cc:1434] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:31:41.946830 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964a87c057f5451cac346b58374b2c03" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:41.947384 21167 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.947459 21168 ts_tablet_manager.cc:1434] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:31:41.947513 20939 raft_consensus.cc:3060] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.948980 21168 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.957631 21168 tablet_bootstrap.cc:654] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.957631 21167 tablet_bootstrap.cc:654] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.959137 21166 consensus_queue.cc:260] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.964264 20939 raft_consensus.cc:2468] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:41.960772 21063 leader_election.cc:290] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:41.966252 21166 ts_tablet_manager.cc:1434] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.009s
I20260709 12:31:41.967963 21166 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:41.968878 20828 leader_election.cc:304] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:41.970420 21063 raft_consensus.cc:2804] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:41.971434 21063 raft_consensus.cc:697] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:41.972584 21063 consensus_queue.cc:237] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.973353 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964a87c057f5451cac346b58374b2c03" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:41.974123 21013 raft_consensus.cc:3060] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:41.974663 21168 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:41.974799 21167 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:41.975438 21168 ts_tablet_manager.cc:1403] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.011s
I20260709 12:31:41.975518 21167 ts_tablet_manager.cc:1403] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.008s
I20260709 12:31:41.979550 21168 raft_consensus.cc:359] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.980654 21168 raft_consensus.cc:385] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.979867 21167 raft_consensus.cc:359] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.981031 21168 raft_consensus.cc:740] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.981287 21167 raft_consensus.cc:385] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:41.981771 21167 raft_consensus.cc:740] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:41.981942 21168 consensus_queue.cc:260] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.982918 21167 consensus_queue.cc:260] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.985054 21168 ts_tablet_manager.cc:1434] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.007s
I20260709 12:31:41.986544 21168 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:41.986753 21167 ts_tablet_manager.cc:1434] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 12:31:41.987080 21166 tablet_bootstrap.cc:654] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.988933 21167 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:41.993170 21065 raft_consensus.cc:493] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:41.994005 21065 raft_consensus.cc:515] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:41.995136 21168 tablet_bootstrap.cc:654] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:41.996955 21065 leader_election.cc:290] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:41.999285 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.000252 20863 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:42.001578 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.002429 20939 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:42.004925 20978 leader_election.cc:304] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.007266 21065 raft_consensus.cc:2804] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.008024 21065 raft_consensus.cc:493] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.008553 21065 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.011475 21166 tablet_bootstrap.cc:492] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:42.012312 21166 ts_tablet_manager.cc:1403] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.004s
I20260709 12:31:42.014055 21167 tablet_bootstrap.cc:654] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.016508 21166 raft_consensus.cc:359] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.018203 21166 raft_consensus.cc:385] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.018625 21166 raft_consensus.cc:740] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.020335 21166 consensus_queue.cc:260] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.021152 21168 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:42.022549 21168 ts_tablet_manager.cc:1403] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.036s	user 0.026s	sys 0.008s
I20260709 12:31:42.024179 21166 ts_tablet_manager.cc:1434] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:31:42.025794 21167 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:42.026672 21167 ts_tablet_manager.cc:1403] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.005s
I20260709 12:31:42.026245 21168 raft_consensus.cc:359] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.027362 21168 raft_consensus.cc:385] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.027732 21168 raft_consensus.cc:740] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.028749 21168 consensus_queue.cc:260] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.025753 21166 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:42.031692 21168 ts_tablet_manager.cc:1434] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:31:42.032905 21168 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:42.035820 21167 raft_consensus.cc:359] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.036978 21167 raft_consensus.cc:385] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.037518 21167 raft_consensus.cc:740] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.039191 21166 tablet_bootstrap.cc:654] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.035053 20760 catalog_manager.cc:5697] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.038870 21013 raft_consensus.cc:2468] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:42.042802 21167 consensus_queue.cc:260] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.052654 21168 tablet_bootstrap.cc:654] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.057245 21065 raft_consensus.cc:515] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.058188 21166 tablet_bootstrap.cc:492] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:42.060232 21153 raft_consensus.cc:493] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.061335 21166 ts_tablet_manager.cc:1403] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.002s
I20260709 12:31:42.061244 21153 raft_consensus.cc:515] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.063995 21153 leader_election.cc:290] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.060360 21167 ts_tablet_manager.cc:1434] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.003s
I20260709 12:31:42.066345 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.067649 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.066787 21181 raft_consensus.cc:493] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.070102 20863 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.070973 21167 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:42.071909 20904 leader_election.cc:304] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.073020 21153 raft_consensus.cc:2804] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.073611 21153 raft_consensus.cc:493] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.074061 21153 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.070470 21013 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.075038 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:42.075740 20939 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.070478 21181 raft_consensus.cc:515] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.066449 21166 raft_consensus.cc:359] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.077148 21166 raft_consensus.cc:385] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.077472 21166 raft_consensus.cc:740] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.078372 21166 consensus_queue.cc:260] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.080370 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.081185 20940 raft_consensus.cc:2468] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:42.078910 21181 leader_election.cc:290] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.084062 20828 leader_election.cc:304] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.084475 21166 ts_tablet_manager.cc:1434] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:31:42.073021 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:42.085834 21166 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:42.086099 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.086891 20863 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.092017 21063 raft_consensus.cc:2804] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.092715 21063 raft_consensus.cc:493] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.093124 21063 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:42.092060 20828 leader_election.cc:343] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:42.093092 21168 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:42.100678 21168 ts_tablet_manager.cc:1403] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.003s
I20260709 12:31:42.103809 21063 raft_consensus.cc:515] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.107558 21167 tablet_bootstrap.cc:654] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.107666 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:42.108525 20940 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.113036 20939 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:42.113791 21065 leader_election.cc:290] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:42.107579 21166 tablet_bootstrap.cc:654] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.118594 21154 raft_consensus.cc:493] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.119385 21154 raft_consensus.cc:515] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.121896 21154 leader_election.cc:290] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.122028 21168 raft_consensus.cc:359] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.122876 21168 raft_consensus.cc:385] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.123198 21168 raft_consensus.cc:740] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.123613 20863 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 1.
I20260709 12:31:42.123683 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a6d9c76dc34f70a705c1a3a8c30165" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.113979 21153 raft_consensus.cc:515] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.127676 20865 raft_consensus.cc:2468] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.129520 20978 leader_election.cc:304] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.130793 21065 raft_consensus.cc:2804] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.131520 21065 raft_consensus.cc:697] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:42.132157 21168 consensus_queue.cc:260] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.133008 21065 consensus_queue.cc:237] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.134207 21168 ts_tablet_manager.cc:1434] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.003s
I20260709 12:31:42.136269 20904 leader_election.cc:304] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.136802 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a6d9c76dc34f70a705c1a3a8c30165" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.138983 21168 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: Bootstrap starting.
I20260709 12:31:42.140949 21063 leader_election.cc:290] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.142323 21154 raft_consensus.cc:2804] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.142890 21154 raft_consensus.cc:493] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.143294 21154 raft_consensus.cc:3060] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.148430 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:42.142263 21013 raft_consensus.cc:2468] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.149334 20865 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.149580 21153 leader_election.cc:290] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.152990 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
W20260709 12:31:42.156129 20828 leader_election.cc:343] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:42.172798 21167 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:42.173494 21167 ts_tablet_manager.cc:1403] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.103s	user 0.010s	sys 0.011s
I20260709 12:31:42.178845 21167 raft_consensus.cc:359] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.179855 21167 raft_consensus.cc:385] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.180311 21167 raft_consensus.cc:740] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.182972 20940 raft_consensus.cc:2468] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:42.184731 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.185981 21013 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.186815 20828 leader_election.cc:304] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:42.188103 21181 raft_consensus.cc:2804] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.188735 21167 consensus_queue.cc:260] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.188957 21181 raft_consensus.cc:697] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:42.192809 21181 consensus_queue.cc:237] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.191996 21154 raft_consensus.cc:515] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.205127 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a6d9c76dc34f70a705c1a3a8c30165" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:42.205307 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a6d9c76dc34f70a705c1a3a8c30165" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.205925 20863 raft_consensus.cc:3060] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.194123 21167 ts_tablet_manager.cc:1434] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:31:42.205981 21014 raft_consensus.cc:3060] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.210104 21013 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.210444 20762 catalog_manager.cc:5697] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 0 to 1, leader changed from <none> to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 1 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:42.212121 20903 leader_election.cc:304] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.213152 21153 raft_consensus.cc:2804] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.213698 21153 raft_consensus.cc:697] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:42.214568 21153 consensus_queue.cc:237] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.218650 20863 raft_consensus.cc:2468] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.220319 20904 leader_election.cc:304] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.221549 21155 raft_consensus.cc:2804] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.193238 21166 tablet_bootstrap.cc:492] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:42.231458 21166 ts_tablet_manager.cc:1403] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.146s	user 0.043s	sys 0.014s
I20260709 12:31:42.233284 20865 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.201493 21168 tablet_bootstrap.cc:654] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.239686 20760 catalog_manager.cc:5697] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.246273 21166 raft_consensus.cc:359] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.247522 21166 raft_consensus.cc:385] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.247856 21166 raft_consensus.cc:740] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.248697 21166 consensus_queue.cc:260] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.250072 21167 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: Bootstrap starting.
I20260709 12:31:42.256083 21155 raft_consensus.cc:697] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:42.257321 21155 consensus_queue.cc:237] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.264231 21177 raft_consensus.cc:493] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.265225 21177 raft_consensus.cc:515] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.270188 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.294304 21014 raft_consensus.cc:2468] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.252722 21154 leader_election.cc:290] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.303248 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.304402 20940 raft_consensus.cc:2393] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fafc10eb3165402cb0d70a21c20cd235 in current term 1: Already voted for candidate f46fed7fa12a4023b3c1302ae536a6ae in this term.
I20260709 12:31:42.301157 21177 leader_election.cc:290] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:42.310600 20977 leader_election.cc:304] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fafc10eb3165402cb0d70a21c20cd235; no voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:42.310284 20763 catalog_manager.cc:5697] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.318215 21166 ts_tablet_manager.cc:1434] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.086s	user 0.012s	sys 0.004s
I20260709 12:31:42.314873 21177 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.328198 21167 tablet_bootstrap.cc:654] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: Neither blocks nor log segments found. Creating new log.
W20260709 12:31:42.315287 20903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.195:43415, user_credentials={real_user=slave}} blocked reactor thread for 91017.3us
I20260709 12:31:42.331790 21166 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:42.339680 21168 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: No bootstrap required, opened a new log
I20260709 12:31:42.340453 21168 ts_tablet_manager.cc:1403] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: Time spent bootstrapping tablet: real 0.205s	user 0.020s	sys 0.005s
I20260709 12:31:42.343868 21168 raft_consensus.cc:359] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.344000 21204 consensus_queue.cc:1048] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.344662 21168 raft_consensus.cc:385] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.345052 21168 raft_consensus.cc:740] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.345793 21168 consensus_queue.cc:260] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.347539 21166 tablet_bootstrap.cc:654] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.349237 21168 ts_tablet_manager.cc:1434] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:31:42.358503 20763 catalog_manager.cc:5697] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.364924 21177 raft_consensus.cc:2749] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:42.372097 21167 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: No bootstrap required, opened a new log
I20260709 12:31:42.374583 21167 ts_tablet_manager.cc:1403] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: Time spent bootstrapping tablet: real 0.135s	user 0.027s	sys 0.008s
I20260709 12:31:42.375526 21166 tablet_bootstrap.cc:492] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:42.376565 21166 ts_tablet_manager.cc:1403] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.009s
I20260709 12:31:42.379781 21166 raft_consensus.cc:359] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.380662 21166 raft_consensus.cc:385] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.383083 21166 raft_consensus.cc:740] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.384119 21166 consensus_queue.cc:260] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.387094 21166 ts_tablet_manager.cc:1434] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:31:42.388669 21166 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: Bootstrap starting.
I20260709 12:31:42.393204 21167 raft_consensus.cc:359] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.394778 21204 raft_consensus.cc:493] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.395576 21204 raft_consensus.cc:515] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.398504 21166 tablet_bootstrap.cc:654] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:42.399669 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.400503 21014 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.401590 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.402153 20903 leader_election.cc:304] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.402504 20863 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.403355 21154 raft_consensus.cc:2804] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.403802 21154 raft_consensus.cc:493] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.404237 21154 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.405316 21153 consensus_queue.cc:1048] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.411386 21204 leader_election.cc:290] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
W20260709 12:31:42.421890 20811 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:42.431885 21166 tablet_bootstrap.cc:492] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: No bootstrap required, opened a new log
I20260709 12:31:42.434166 21181 raft_consensus.cc:493] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.436432 21181 raft_consensus.cc:515] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.434849 21154 raft_consensus.cc:515] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.435319 21063 raft_consensus.cc:493] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.439723 21063 raft_consensus.cc:515] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.440388 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.440968 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:42.394390 21167 raft_consensus.cc:385] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.441717 20865 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.441846 21167 raft_consensus.cc:740] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.442835 21167 consensus_queue.cc:260] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.444204 20940 raft_consensus.cc:2468] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:42.445575 21167 ts_tablet_manager.cc:1434] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 12:31:42.447320 21181 leader_election.cc:290] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.447703 21177 raft_consensus.cc:493] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.447772 21204 raft_consensus.cc:493] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:42.448649 21177 raft_consensus.cc:515] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.448664 21204 raft_consensus.cc:515] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.451429 21204 leader_election.cc:290] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.453116 21063 leader_election.cc:290] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.453490 21177 leader_election.cc:290] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:42.453589 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.455307 21166 ts_tablet_manager.cc:1403] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: Time spent bootstrapping tablet: real 0.067s	user 0.027s	sys 0.015s
I20260709 12:31:42.455511 21013 raft_consensus.cc:2468] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.456139 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.456909 21014 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.457101 20903 leader_election.cc:304] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.458201 21204 raft_consensus.cc:2804] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.458628 21204 raft_consensus.cc:493] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.459008 21204 raft_consensus.cc:3060] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.459164 21166 raft_consensus.cc:359] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.460132 21166 raft_consensus.cc:385] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:42.460464 21166 raft_consensus.cc:740] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Initialized, Role: FOLLOWER
I20260709 12:31:42.461390 21166 consensus_queue.cc:260] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.461615 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.464218 21166 ts_tablet_manager.cc:1434] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:31:42.467054 21014 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.472323 20865 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.484741 21204 raft_consensus.cc:515] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.468573 21154 leader_election.cc:290] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.495648 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.496642 20865 raft_consensus.cc:2468] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 0.
I20260709 12:31:42.499570 21204 leader_election.cc:290] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.494241 20903 leader_election.cc:304] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.502068 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.502171 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" is_pre_election: true
I20260709 12:31:42.502985 21014 raft_consensus.cc:2468] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 0.
I20260709 12:31:42.503153 20863 raft_consensus.cc:2468] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 0.
I20260709 12:31:42.504559 21013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.505312 21204 raft_consensus.cc:2804] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.506304 21204 raft_consensus.cc:697] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:42.498347 20940 raft_consensus.cc:2393] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fafc10eb3165402cb0d70a21c20cd235 in current term 1: Already voted for candidate fa4da9dd3d364a828c6b7a31784f7244 in this term.
I20260709 12:31:42.507288 21204 consensus_queue.cc:237] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.512980 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244" is_pre_election: true
I20260709 12:31:42.517011 20940 raft_consensus.cc:2393] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46fed7fa12a4023b3c1302ae536a6ae in current term 1: Already voted for candidate fa4da9dd3d364a828c6b7a31784f7244 in this term.
I20260709 12:31:42.518417 20978 leader_election.cc:304] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.505319 21013 raft_consensus.cc:3060] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.514540 20828 leader_election.cc:304] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.514899 21011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235" is_pre_election: true
I20260709 12:31:42.524304 20863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cf6deaf7dd4671bba915f364d3c307" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:42.524296 21217 raft_consensus.cc:2804] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:42.525204 20863 raft_consensus.cc:3060] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.525233 21217 raft_consensus.cc:493] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:42.527004 21217 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.527666 21216 consensus_queue.cc:1048] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.526115 20763 catalog_manager.cc:5697] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.534044 20863 raft_consensus.cc:2468] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.536469 20904 leader_election.cc:304] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.537918 21203 raft_consensus.cc:2804] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.538030 20828 leader_election.cc:304] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae; no voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:42.538628 21203 raft_consensus.cc:697] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:42.539688 21203 consensus_queue.cc:237] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.543046 21013 raft_consensus.cc:2468] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 1.
I20260709 12:31:42.544045 21177 raft_consensus.cc:2764] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:31:42.545313 21181 raft_consensus.cc:2749] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:42.546278 21181 consensus_queue.cc:1048] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:42.555262 21217 raft_consensus.cc:515] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.563534 21177 consensus_queue.cc:1048] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.565517 21217 leader_election.cc:290] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.568933 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:42.569846 20939 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.577203 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.578158 21014 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:42.578902 20763 catalog_manager.cc:5697] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 1 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.583688 20939 raft_consensus.cc:2468] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:42.591075 21177 consensus_queue.cc:1048] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.600023 21014 raft_consensus.cc:2468] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 1.
I20260709 12:31:42.585807 20828 leader_election.cc:304] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:42.602543 21217 raft_consensus.cc:2804] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:42.603338 21217 raft_consensus.cc:697] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:42.604396 21217 consensus_queue.cc:237] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.630812 20762 catalog_manager.cc:5697] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 1 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.706620 21204 consensus_queue.cc:1048] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.722880 21203 consensus_queue.cc:1048] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.742302 21217 consensus_queue.cc:1048] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:42.752676 21063 consensus_queue.cc:1048] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:42.775286 21204 consensus_queue.cc:1048] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.776749 21203 consensus_queue.cc:1048] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:42.890586 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:36378
I20260709 12:31:42.891459 20940 raft_consensus.cc:606] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:42.895675 21153 raft_consensus.cc:993] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:42.896266 21153 raft_consensus.cc:1081] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:42.898589 20863 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41716
I20260709 12:31:42.898798 21014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:31:42.899626 21014 raft_consensus.cc:606] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:42.899649 20863 raft_consensus.cc:493] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:42.900486 20863 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:42.910099 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0de23adb26e845518c999234e98db506"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:36378
I20260709 12:31:42.911140 20940 raft_consensus.cc:606] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Received request to transfer leadership to TS f46fed7fa12a4023b3c1302ae536a6ae
I20260709 12:31:42.910842 20863 raft_consensus.cc:515] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.913698 20863 leader_election.cc:290] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:42.915169 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:42.915642 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae1a8e410df043388bac1b5053cfb6a3" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:42.916074 20940 raft_consensus.cc:3055] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:42.916232 21014 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:42.916493 20940 raft_consensus.cc:740] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:42.917523 20940 consensus_queue.cc:260] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.919090 20940 raft_consensus.cc:3060] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:42.926162 21014 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:42.927843 20828 leader_election.cc:304] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:42.928239 21155 consensus_queue.cc:1048] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.928598 20940 raft_consensus.cc:2468] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:42.928913 21217 raft_consensus.cc:2804] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:42.929683 21217 raft_consensus.cc:697] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:42.930778 21217 consensus_queue.cc:237] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:42.931510 20865 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
mode: GRACEFUL
new_leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:41746
I20260709 12:31:42.932248 20865 raft_consensus.cc:606] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Received request to transfer leadership to TS fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:42.943094 20763 catalog_manager.cc:5697] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:42.946106 21203 consensus_queue.cc:1048] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:42.950732 20864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
mode: GRACEFUL
new_leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:41746
I20260709 12:31:42.961040 20864 raft_consensus.cc:606] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Received request to transfer leadership to TS fa4da9dd3d364a828c6b7a31784f7244
I20260709 12:31:42.973541 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:36378
I20260709 12:31:42.974249 20940 raft_consensus.cc:606] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Received request to transfer leadership to TS fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:43.039474 21226 raft_consensus.cc:993] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:43.040014 21177 raft_consensus.cc:1081] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:43.041831 20864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41732
I20260709 12:31:43.042353 20864 raft_consensus.cc:493] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:43.042685 20864 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.049346 20864 raft_consensus.cc:515] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.051149 20864 leader_election.cc:290] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:43.052695 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:43.053252 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d4c6ef82fe428d9e548d64e102f04f" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:43.053665 20940 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.054037 21014 raft_consensus.cc:3055] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:43.054410 21014 raft_consensus.cc:740] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:43.055294 21014 consensus_queue.cc:260] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.056643 21014 raft_consensus.cc:3060] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.061233 20940 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:43.062768 20828 leader_election.cc:304] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: 
I20260709 12:31:43.063424 21014 raft_consensus.cc:2468] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:43.063886 21217 raft_consensus.cc:2804] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:43.064677 21217 raft_consensus.cc:697] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:43.065518 21217 consensus_queue.cc:237] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.073848 20760 catalog_manager.cc:5697] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:43.093396 21217 consensus_queue.cc:1048] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:43.098029 21153 consensus_queue.cc:1048] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:43.103901 21153 consensus_queue.cc:1048] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:43.110625 21153 raft_consensus.cc:993] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:43.111482 21204 raft_consensus.cc:1081] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Signalling peer fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:43.113770 21011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:57772
I20260709 12:31:43.117115 21011 raft_consensus.cc:493] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:43.117681 21011 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.126981 21063 raft_consensus.cc:993] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: : Instructing follower fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:43.127656 21181 raft_consensus.cc:1081] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Signalling peer fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:43.130311 21011 raft_consensus.cc:515] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.133244 20939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:36356
I20260709 12:31:43.134028 20939 raft_consensus.cc:493] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:43.134464 20939 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.140846 20940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:43.141556 20940 raft_consensus.cc:3055] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:43.141832 20940 raft_consensus.cc:740] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:43.142598 20940 consensus_queue.cc:260] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.143548 20940 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.145340 20939 raft_consensus.cc:515] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.146248 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c49d46931024b6bbd42e5829a95f74d" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:43.149448 20865 raft_consensus.cc:3060] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.140122 21011 leader_election.cc:290] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:43.153478 21217 consensus_queue.cc:1048] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:43.151194 20940 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 2.
I20260709 12:31:43.151942 20938 raft_consensus.cc:1240] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Rejecting Update request from peer f46fed7fa12a4023b3c1302ae536a6ae for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:43.157564 20977 leader_election.cc:304] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:43.157795 20865 raft_consensus.cc:2468] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 2.
I20260709 12:31:43.159204 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:43.160022 20864 raft_consensus.cc:3055] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:43.160377 20864 raft_consensus.cc:740] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:43.160624 21177 raft_consensus.cc:2804] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:43.161007 21217 consensus_queue.cc:1059] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:43.161301 21177 raft_consensus.cc:697] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 2 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:43.162274 21177 consensus_queue.cc:237] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.163966 20864 consensus_queue.cc:260] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.165148 20864 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.166246 20939 leader_election.cc:290] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:43.167413 21014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3aba9102064e48bf859475ec69a37d" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:43.174501 20864 raft_consensus.cc:2468] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
W20260709 12:31:43.175376 21216 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:31:43.176344 20904 leader_election.cc:304] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fa4da9dd3d364a828c6b7a31784f7244; no voters: fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:43.178284 21155 raft_consensus.cc:2804] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:43.178905 21155 raft_consensus.cc:697] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:43.179842 21155 consensus_queue.cc:237] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.186911 20762 catalog_manager.cc:5697] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 1 to 2, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 2 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:43.198045 20760 catalog_manager.cc:5697] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 1 to 2, leader changed from f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193) to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:43.225142 21203 raft_consensus.cc:993] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:43.225700 21203 raft_consensus.cc:1081] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Signalling peer f46fed7fa12a4023b3c1302ae536a6ae to start an election
I20260709 12:31:43.227843 20864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0de23adb26e845518c999234e98db506"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
 from {username='slave'} at 127.0.0.1:41716
I20260709 12:31:43.228456 20864 raft_consensus.cc:493] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:43.228809 20864 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.237031 20864 raft_consensus.cc:515] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.239482 20864 leader_election.cc:290] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:43.240469 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:43.241042 21011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de23adb26e845518c999234e98db506" candidate_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:43.241286 20939 raft_consensus.cc:3055] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:43.241670 20939 raft_consensus.cc:740] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:43.241724 21011 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.242786 20939 consensus_queue.cc:260] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.243741 21216 raft_consensus.cc:993] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: : Instructing follower fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:43.244357 21217 raft_consensus.cc:1081] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Signalling peer fa4da9dd3d364a828c6b7a31784f7244 to start an election
I20260709 12:31:43.244468 20939 raft_consensus.cc:3060] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.246610 20938 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
 from {username='slave'} at 127.0.0.1:36356
I20260709 12:31:43.247394 20938 raft_consensus.cc:493] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:43.247835 20938 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.250749 21011 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:43.252194 20828 leader_election.cc:304] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:43.253762 21217 raft_consensus.cc:2804] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:43.253950 20939 raft_consensus.cc:2468] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46fed7fa12a4023b3c1302ae536a6ae in term 2.
I20260709 12:31:43.254390 21217 raft_consensus.cc:697] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Leader. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:43.255483 21217 consensus_queue.cc:237] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.257198 20938 raft_consensus.cc:515] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.260500 20938 leader_election.cc:290] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195:43415)
I20260709 12:31:43.260656 20864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:43.261646 20864 raft_consensus.cc:3055] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:43.262042 20864 raft_consensus.cc:740] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:43.262105 21011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e46ffe3e22c4982865a3cf870217f52" candidate_uuid: "fa4da9dd3d364a828c6b7a31784f7244" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
I20260709 12:31:43.262863 21011 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.263139 20864 consensus_queue.cc:260] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.263899 20760 catalog_manager.cc:5697] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae reported cstate change: term changed from 1 to 2, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193). New cstate: current_term: 2 leader_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:43.264890 20864 raft_consensus.cc:3060] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.272154 21011 raft_consensus.cc:2468] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:43.274330 20903 leader_election.cc:304] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:43.275138 20864 raft_consensus.cc:2468] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa4da9dd3d364a828c6b7a31784f7244 in term 2.
I20260709 12:31:43.275527 21203 raft_consensus.cc:2804] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:43.276171 21203 raft_consensus.cc:697] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Becoming Leader. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:43.277186 21203 consensus_queue.cc:237] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:43.285810 20760 catalog_manager.cc:5697] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 reported cstate change: term changed from 1 to 2, leader changed from f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193) to fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194). New cstate: current_term: 2 leader_uuid: "fa4da9dd3d364a828c6b7a31784f7244" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:43.328689 20938 raft_consensus.cc:1275] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.330279 21217 consensus_queue.cc:1048] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.340899 21011 raft_consensus.cc:1275] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.343621 21063 consensus_queue.cc:1048] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.559391 20864 raft_consensus.cc:1275] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.561518 21177 consensus_queue.cc:1048] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:43.573472 20938 raft_consensus.cc:1275] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.577178 21226 consensus_queue.cc:1048] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:43.641508 20938 raft_consensus.cc:1275] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.643515 21063 consensus_queue.cc:1048] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.656513 21011 raft_consensus.cc:1275] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.660202 21217 consensus_queue.cc:1048] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:43.667804 20864 raft_consensus.cc:1275] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.669701 21203 consensus_queue.cc:1048] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.686889 21011 raft_consensus.cc:3060] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:43.699731 21011 raft_consensus.cc:1275] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.701606 21155 consensus_queue.cc:1048] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.734602 21014 raft_consensus.cc:1275] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.736320 21155 consensus_queue.cc:1048] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:43.746796 20864 raft_consensus.cc:1275] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Refusing update from remote peer fa4da9dd3d364a828c6b7a31784f7244: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.749313 21155 consensus_queue.cc:1048] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.795912 20938 raft_consensus.cc:1275] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.799278 21063 consensus_queue.cc:1048] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:43.810546 21014 raft_consensus.cc:1275] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Refusing update from remote peer f46fed7fa12a4023b3c1302ae536a6ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:43.812460 21063 consensus_queue.cc:1048] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:45.978348 20760 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47966:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:31:46.004954 20760 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47966:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:31:46.063463  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:31:46.269640 21270 log.cc:927] Time spent T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:31:46.335850 20978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.76.193:41987, user_credentials={real_user=slave}} blocked reactor thread for 103099us
I20260709 12:31:46.368822 21297 tablet.cc:1724] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.371600 21299 tablet.cc:1724] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:46.424494 21284 log.cc:927] Time spent T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
W20260709 12:31:46.457437 21285 log.cc:927] Time spent T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:31:46.460150 21291 log.cc:927] Time spent T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:31:46.344306 21294 tablet.cc:1724] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:46.461598 21272 log.cc:927] Time spent T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Append to log took a long time: real 0.225s	user 0.005s	sys 0.000s
I20260709 12:31:46.501791 21310 tablet.cc:1724] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.516920 20940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522"
dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
mode: GRACEFUL
new_leader_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:40760
I20260709 12:31:46.517822 20940 raft_consensus.cc:606] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Received request to transfer leadership to TS fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:46.466253 21302 tablet.cc:1724] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.544279 21315 tablet.cc:1724] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.478811 21311 tablet.cc:1724] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.585744 21305 tablet.cc:1724] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.596792 21304 tablet.cc:1724] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.641980 21297 tablet.cc:1724] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.622038 21320 tablet.cc:1724] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.677747 21321 tablet.cc:1724] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:46.679625 21312 log.cc:927] Time spent T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Append to log took a long time: real 0.096s	user 0.000s	sys 0.003s
I20260709 12:31:46.689374 21318 tablet.cc:1724] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.696285 21274 raft_consensus.cc:993] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: : Instructing follower fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:46.697036 21274 raft_consensus.cc:1081] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Signalling peer fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:46.701628 21010 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:57772
I20260709 12:31:46.702363 21010 raft_consensus.cc:493] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:46.702744 21010 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:46.705446 21305 tablet.cc:1724] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.708262 21294 tablet.cc:1724] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.742134 21305 tablet.cc:1724] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.749287 20865 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684198e026d847e8a2aa98460be581d7"
dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
mode: GRACEFUL
new_leader_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:46350
I20260709 12:31:46.750121 20865 raft_consensus.cc:606] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Received request to transfer leadership to TS fafc10eb3165402cb0d70a21c20cd235
I20260709 12:31:46.752193  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:31:46.752965  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:46.756618 21318 tablet.cc:1724] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.758674 21305 tablet.cc:1724] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.760459 21294 tablet.cc:1724] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.775370 21315 tablet.cc:1724] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.785326 21318 tablet.cc:1724] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.792721 21299 tablet.cc:1724] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.795670 21010 raft_consensus.cc:515] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:46.800076 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:46.800765 20865 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:46.801530 21010 leader_election.cc:290] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 election: Requested vote from peers f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987), fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421)
I20260709 12:31:46.804224 21297 tablet.cc:1724] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.805107 20936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a1a25f3bab4360937643f99c1ba522" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:46.805866 20936 raft_consensus.cc:3055] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:46.806205 20936 raft_consensus.cc:740] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fa4da9dd3d364a828c6b7a31784f7244, State: Running, Role: LEADER
I20260709 12:31:46.807104 20936 consensus_queue.cc:260] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:46.807201 21302 tablet.cc:1724] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.808688 20936 raft_consensus.cc:3060] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:46.813973 21304 tablet.cc:1724] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.819589 21321 tablet.cc:1724] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.829761 20763 catalog_manager.cc:5789] TS f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987) does not have the latest schema for tablet 0de23adb26e845518c999234e98db506 (table soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8]). Expected version 1 got 0
I20260709 12:31:46.838483 20865 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 3.
I20260709 12:31:46.841315 21310 tablet.cc:1724] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:46.847739 20978 leader_election.cc:304] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f46fed7fa12a4023b3c1302ae536a6ae, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:46.849689 21226 raft_consensus.cc:2804] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:46.850287 21226 raft_consensus.cc:697] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 3 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:46.851300 21226 consensus_queue.cc:237] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } }
I20260709 12:31:46.874907 20936 raft_consensus.cc:2468] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 3.
I20260709 12:31:46.897753 20760 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: soft_deleted_table [id=187cde7f99fa41e08976546e9b6c99f8] alter complete (version 1)
I20260709 12:31:47.011554 20760 catalog_manager.cc:5697] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 2 to 3, leader changed from fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194) to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:47.254673 20939 raft_consensus.cc:1275] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:47.256934 21226 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:47.273195 20865 raft_consensus.cc:1275] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:47.281405 21283 consensus_queue.cc:1048] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:47.294344 21261 raft_consensus.cc:993] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: : Instructing follower fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:47.295478 21216 raft_consensus.cc:1081] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Signalling peer fafc10eb3165402cb0d70a21c20cd235 to start an election
I20260709 12:31:47.298808 21014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "684198e026d847e8a2aa98460be581d7"
dest_uuid: "fafc10eb3165402cb0d70a21c20cd235"
 from {username='slave'} at 127.0.0.1:57780
I20260709 12:31:47.299757 21014 raft_consensus.cc:493] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:47.300191 21014 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:47.312847 21014 raft_consensus.cc:515] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:47.316301 21014 leader_election.cc:290] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 election: Requested vote from peers fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421), f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987)
I20260709 12:31:47.316672 20939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa4da9dd3d364a828c6b7a31784f7244"
I20260709 12:31:47.317235 20865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684198e026d847e8a2aa98460be581d7" candidate_uuid: "fafc10eb3165402cb0d70a21c20cd235" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f46fed7fa12a4023b3c1302ae536a6ae"
I20260709 12:31:47.317775 20939 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:47.318163 20865 raft_consensus.cc:3055] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:47.318627 20865 raft_consensus.cc:740] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f46fed7fa12a4023b3c1302ae536a6ae, State: Running, Role: LEADER
I20260709 12:31:47.320041 20865 consensus_queue.cc:260] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:47.322088 20865 raft_consensus.cc:3060] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:47.330368 20939 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 3.
I20260709 12:31:47.332288 20977 leader_election.cc:304] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa4da9dd3d364a828c6b7a31784f7244, fafc10eb3165402cb0d70a21c20cd235; no voters: 
I20260709 12:31:47.333000 20865 raft_consensus.cc:2468] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafc10eb3165402cb0d70a21c20cd235 in term 3.
I20260709 12:31:47.333578 21226 raft_consensus.cc:2804] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:47.334242 21226 raft_consensus.cc:697] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 3 LEADER]: Becoming Leader. State: Replica: fafc10eb3165402cb0d70a21c20cd235, State: Running, Role: LEADER
I20260709 12:31:47.335722 21226 consensus_queue.cc:237] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } }
I20260709 12:31:47.346773 20763 catalog_manager.cc:5697] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 reported cstate change: term changed from 2 to 3, leader changed from f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193) to fafc10eb3165402cb0d70a21c20cd235 (127.6.76.195). New cstate: current_term: 3 leader_uuid: "fafc10eb3165402cb0d70a21c20cd235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fafc10eb3165402cb0d70a21c20cd235" member_type: VOTER last_known_addr { host: "127.6.76.195" port: 43415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:47.754462  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:47.762612  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:47.763295  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:47.814880 20939 raft_consensus.cc:1275] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:47.816963 21330 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4da9dd3d364a828c6b7a31784f7244" member_type: VOTER last_known_addr { host: "127.6.76.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:47.832773 20865 raft_consensus.cc:1275] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Refusing update from remote peer fafc10eb3165402cb0d70a21c20cd235: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:47.835988 21330 consensus_queue.cc:1048] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46fed7fa12a4023b3c1302ae536a6ae" member_type: VOTER last_known_addr { host: "127.6.76.193" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:48.764642  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:48.771160  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:48.771791  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:49.773371  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:49.781939  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:49.782557  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:50.783798  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:50.789330  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:50.789832  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:51.791049  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:51.799677  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:51.800493  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:52.801748  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:52.807328  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:52.807883  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:53.809221  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:53.815764  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:53.816480  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:54.818068  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:54.824867  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:54.825690  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:55.827623  6451 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:55.833559  6451 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:55.834229  6451 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:56.920173  6451 tablet_server.cc:179] TabletServer@127.6.76.193:0 shutting down...
I20260709 12:31:56.978602  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:56.979807  6451 tablet_replica.cc:333] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
W20260709 12:31:56.980309 20978 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 17 similar messages]
I20260709 12:31:56.981427  6451 raft_consensus.cc:2243] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:56.982524  6451 raft_consensus.cc:2272] T 8e46ffe3e22c4982865a3cf870217f52 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:56.985070 20978 consensus_peers.cc:597] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:56.986263  6451 tablet_replica.cc:333] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:56.987304  6451 raft_consensus.cc:2243] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:56.988018  6451 raft_consensus.cc:2272] T 8c49d46931024b6bbd42e5829a95f74d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.024204  6451 tablet_replica.cc:333] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.029241  6451 raft_consensus.cc:2243] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.031736  6451 raft_consensus.cc:2272] T c7a6d9c76dc34f70a705c1a3a8c30165 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.036486  6451 tablet_replica.cc:333] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.038458  6451 raft_consensus.cc:2243] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.042017  6451 raft_consensus.cc:2272] T 0de23adb26e845518c999234e98db506 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:57.047821 20904 consensus_peers.cc:597] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.048079  6451 tablet_replica.cc:333] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.049528  6451 raft_consensus.cc:2243] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.050689  6451 raft_consensus.cc:2272] T f4cf6deaf7dd4671bba915f364d3c307 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:57.052327 20904 consensus_peers.cc:597] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.056375  6451 tablet_replica.cc:333] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.058233  6451 raft_consensus.cc:2243] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.060889  6451 raft_consensus.cc:2272] T ae1a8e410df043388bac1b5053cfb6a3 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.064821  6451 tablet_replica.cc:333] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.065744  6451 raft_consensus.cc:2243] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.067498  6451 raft_consensus.cc:2272] T 09d4c6ef82fe428d9e548d64e102f04f P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.071575  6451 tablet_replica.cc:333] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.073143  6451 raft_consensus.cc:2243] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.074685  6451 raft_consensus.cc:2272] T e093ef11e3de47f4811c86f49d3c6d26 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.082422  6451 tablet_replica.cc:333] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.084510  6451 raft_consensus.cc:2243] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:31:57.087404 20978 consensus_peers.cc:597] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.088537  6451 raft_consensus.cc:2272] T 8cd3283bf33d4b7d8f9c27dc654736f2 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.093948  6451 tablet_replica.cc:333] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.095058  6451 raft_consensus.cc:2243] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.096141  6451 raft_consensus.cc:2272] T 2b3aba9102064e48bf859475ec69a37d P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.101172  6451 tablet_replica.cc:333] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.102262  6451 raft_consensus.cc:2243] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.104496  6451 raft_consensus.cc:2272] T 964a87c057f5451cac346b58374b2c03 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.108126  6451 tablet_replica.cc:333] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.108924  6451 raft_consensus.cc:2243] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.110651  6451 raft_consensus.cc:2272] T 1f3dfde2374a4f13b32100863dac37e1 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.115083  6451 tablet_replica.cc:333] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.116007  6451 raft_consensus.cc:2243] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.117017  6451 raft_consensus.cc:2272] T 74c2f38bda214598879c9ed964510d3c P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.120671  6451 tablet_replica.cc:333] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.121590  6451 raft_consensus.cc:2243] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.122362  6451 raft_consensus.cc:2272] T 59a1a25f3bab4360937643f99c1ba522 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.125578  6451 tablet_replica.cc:333] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.126762  6451 raft_consensus.cc:2243] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.127629  6451 raft_consensus.cc:2272] T 1384335a40084b168d9b7f8e34082e77 P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.131273  6451 tablet_replica.cc:333] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.132095  6451 raft_consensus.cc:2243] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.132838  6451 raft_consensus.cc:2272] T 8d27db4c416c467c8d03ef16089e1ece P f46fed7fa12a4023b3c1302ae536a6ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.137934  6451 tablet_replica.cc:333] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.139143  6451 raft_consensus.cc:2243] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.141011  6451 raft_consensus.cc:2272] T bc1ef7292abf405aac75bc7fdfc17c95 P f46fed7fa12a4023b3c1302ae536a6ae [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:57.142777 20978 consensus_peers.cc:597] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.145306  6451 tablet_replica.cc:333] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae: stopping tablet replica
I20260709 12:31:57.146145  6451 raft_consensus.cc:2243] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.147058  6451 raft_consensus.cc:2272] T 684198e026d847e8a2aa98460be581d7 P f46fed7fa12a4023b3c1302ae536a6ae [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:57.167377 20904 consensus_peers.cc:597] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.223954  6451 tablet_server.cc:196] TabletServer@127.6.76.193:0 shutdown complete.
W20260709 12:31:57.265266 20904 consensus_peers.cc:597] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.290938  6451 tablet_server.cc:179] TabletServer@127.6.76.194:0 shutting down...
W20260709 12:31:57.302107 20904 consensus_peers.cc:597] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:57.312757 20977 consensus_peers.cc:597] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 -> Peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Couldn't send request to peer fa4da9dd3d364a828c6b7a31784f7244. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:57.317400 20904 consensus_peers.cc:597] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:57.336561 20904 consensus_peers.cc:597] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.337108  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:57.338281  6451 tablet_replica.cc:333] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.339540  6451 raft_consensus.cc:2243] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.340731  6451 raft_consensus.cc:2272] T 8e46ffe3e22c4982865a3cf870217f52 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.343868  6451 tablet_replica.cc:333] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.344743  6451 raft_consensus.cc:2243] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.345510  6451 raft_consensus.cc:2272] T 0de23adb26e845518c999234e98db506 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.348456  6451 tablet_replica.cc:333] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.349267  6451 raft_consensus.cc:2243] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.350325  6451 raft_consensus.cc:2272] T 8c49d46931024b6bbd42e5829a95f74d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.354065  6451 tablet_replica.cc:333] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.354938  6451 raft_consensus.cc:2243] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.356319  6451 raft_consensus.cc:2272] T c7a6d9c76dc34f70a705c1a3a8c30165 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.359486  6451 tablet_replica.cc:333] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.360270  6451 raft_consensus.cc:2243] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.361306  6451 raft_consensus.cc:2272] T 09d4c6ef82fe428d9e548d64e102f04f P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.367514  6451 tablet_replica.cc:333] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.369293  6451 raft_consensus.cc:2243] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.371908  6451 raft_consensus.cc:2272] T f4cf6deaf7dd4671bba915f364d3c307 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.377241  6451 tablet_replica.cc:333] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.378350  6451 raft_consensus.cc:2243] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.379415  6451 raft_consensus.cc:2272] T ae1a8e410df043388bac1b5053cfb6a3 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.383126  6451 tablet_replica.cc:333] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.384223  6451 raft_consensus.cc:2243] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.385290  6451 raft_consensus.cc:2272] T e093ef11e3de47f4811c86f49d3c6d26 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.388801  6451 tablet_replica.cc:333] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.389614  6451 raft_consensus.cc:2243] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.390259  6451 raft_consensus.cc:2272] T 59a1a25f3bab4360937643f99c1ba522 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.393153  6451 tablet_replica.cc:333] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.393843  6451 raft_consensus.cc:2243] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.395757  6451 raft_consensus.cc:2272] T 2b3aba9102064e48bf859475ec69a37d P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.399386  6451 tablet_replica.cc:333] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.400323  6451 raft_consensus.cc:2243] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.401122  6451 raft_consensus.cc:2272] T 964a87c057f5451cac346b58374b2c03 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.404235  6451 tablet_replica.cc:333] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.405014  6451 raft_consensus.cc:2243] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.405804  6451 raft_consensus.cc:2272] T 1f3dfde2374a4f13b32100863dac37e1 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.408821  6451 tablet_replica.cc:333] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.409528  6451 raft_consensus.cc:2243] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.410161  6451 raft_consensus.cc:2272] T 684198e026d847e8a2aa98460be581d7 P fa4da9dd3d364a828c6b7a31784f7244 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.413079  6451 tablet_replica.cc:333] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.413827  6451 raft_consensus.cc:2243] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.414542  6451 raft_consensus.cc:2272] T bc1ef7292abf405aac75bc7fdfc17c95 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.417523  6451 tablet_replica.cc:333] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.418280  6451 raft_consensus.cc:2243] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.419634  6451 raft_consensus.cc:2272] T 8d27db4c416c467c8d03ef16089e1ece P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.422901  6451 tablet_replica.cc:333] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.423647  6451 raft_consensus.cc:2243] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.425101  6451 raft_consensus.cc:2272] T 1384335a40084b168d9b7f8e34082e77 P fa4da9dd3d364a828c6b7a31784f7244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.428522  6451 tablet_replica.cc:333] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.429370  6451 raft_consensus.cc:2243] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.430231  6451 raft_consensus.cc:2272] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.433943  6451 tablet_replica.cc:333] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244: stopping tablet replica
I20260709 12:31:57.434706  6451 raft_consensus.cc:2243] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.435947  6451 raft_consensus.cc:2272] T 74c2f38bda214598879c9ed964510d3c P fa4da9dd3d364a828c6b7a31784f7244 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:57.449591 20978 consensus_peers.cc:597] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 -> Peer f46fed7fa12a4023b3c1302ae536a6ae (127.6.76.193:41987): Couldn't send request to peer f46fed7fa12a4023b3c1302ae536a6ae. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.193:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.493009  6451 tablet_server.cc:196] TabletServer@127.6.76.194:0 shutdown complete.
I20260709 12:31:57.545115  6451 tablet_server.cc:179] TabletServer@127.6.76.195:0 shutting down...
W20260709 12:31:57.555922 20977 consensus_peers.cc:597] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 -> Peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Couldn't send request to peer fa4da9dd3d364a828c6b7a31784f7244. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:33421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:57.582753 20977 consensus_peers.cc:597] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 -> Peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Couldn't send request to peer fa4da9dd3d364a828c6b7a31784f7244. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:33421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.598131  6451 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:57.599735  6451 tablet_replica.cc:333] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.600759  6451 raft_consensus.cc:2243] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.601821  6451 raft_consensus.cc:2272] T 8e46ffe3e22c4982865a3cf870217f52 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.604817  6451 tablet_replica.cc:333] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.605576  6451 raft_consensus.cc:2243] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.606238  6451 raft_consensus.cc:2272] T 0de23adb26e845518c999234e98db506 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.609059  6451 tablet_replica.cc:333] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.609807  6451 raft_consensus.cc:2243] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.610903  6451 raft_consensus.cc:2272] T 8c49d46931024b6bbd42e5829a95f74d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.613622  6451 tablet_replica.cc:333] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.614711  6451 raft_consensus.cc:2243] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.615700  6451 raft_consensus.cc:2272] T c7a6d9c76dc34f70a705c1a3a8c30165 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.618983  6451 tablet_replica.cc:333] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.619834  6451 raft_consensus.cc:2243] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.620615  6451 raft_consensus.cc:2272] T 09d4c6ef82fe428d9e548d64e102f04f P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.623713  6451 tablet_replica.cc:333] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.624511  6451 raft_consensus.cc:2243] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.625319  6451 raft_consensus.cc:2272] T f4cf6deaf7dd4671bba915f364d3c307 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.629367  6451 tablet_replica.cc:333] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.630270  6451 raft_consensus.cc:2243] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:31:57.630990 20977 consensus_peers.cc:597] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 -> Peer fa4da9dd3d364a828c6b7a31784f7244 (127.6.76.194:33421): Couldn't send request to peer fa4da9dd3d364a828c6b7a31784f7244. Status: Network error: Client connection negotiation failed: client connection to 127.6.76.194:33421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:57.631184  6451 raft_consensus.cc:2272] T ae1a8e410df043388bac1b5053cfb6a3 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.634752  6451 tablet_replica.cc:333] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.635763  6451 raft_consensus.cc:2243] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.637305  6451 raft_consensus.cc:2272] T e093ef11e3de47f4811c86f49d3c6d26 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.640347  6451 tablet_replica.cc:333] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.641093  6451 raft_consensus.cc:2243] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.641727  6451 raft_consensus.cc:2272] T 74c2f38bda214598879c9ed964510d3c P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.644491  6451 tablet_replica.cc:333] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.645165  6451 raft_consensus.cc:2243] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.646175  6451 raft_consensus.cc:2272] T 59a1a25f3bab4360937643f99c1ba522 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.648729  6451 tablet_replica.cc:333] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.649389  6451 raft_consensus.cc:2243] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.650012  6451 raft_consensus.cc:2272] T 8cd3283bf33d4b7d8f9c27dc654736f2 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.652710  6451 tablet_replica.cc:333] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.653654  6451 raft_consensus.cc:2243] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.654353  6451 raft_consensus.cc:2272] T bc1ef7292abf405aac75bc7fdfc17c95 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.657418  6451 tablet_replica.cc:333] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.658142  6451 raft_consensus.cc:2243] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.658689  6451 raft_consensus.cc:2272] T 8d27db4c416c467c8d03ef16089e1ece P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.661743  6451 tablet_replica.cc:333] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.662352  6451 raft_consensus.cc:2243] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.663575  6451 raft_consensus.cc:2272] T 684198e026d847e8a2aa98460be581d7 P fafc10eb3165402cb0d70a21c20cd235 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.667091  6451 tablet_replica.cc:333] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.667831  6451 raft_consensus.cc:2243] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.668433  6451 raft_consensus.cc:2272] T 2b3aba9102064e48bf859475ec69a37d P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.671006  6451 tablet_replica.cc:333] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.671675  6451 raft_consensus.cc:2243] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.672298  6451 raft_consensus.cc:2272] T 964a87c057f5451cac346b58374b2c03 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.675437  6451 tablet_replica.cc:333] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.676024  6451 raft_consensus.cc:2243] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.676652  6451 raft_consensus.cc:2272] T 1f3dfde2374a4f13b32100863dac37e1 P fafc10eb3165402cb0d70a21c20cd235 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.679407  6451 tablet_replica.cc:333] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235: stopping tablet replica
I20260709 12:31:57.679924  6451 raft_consensus.cc:2243] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:57.680424  6451 raft_consensus.cc:2272] T 1384335a40084b168d9b7f8e34082e77 P fafc10eb3165402cb0d70a21c20cd235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.731809  6451 tablet_server.cc:196] TabletServer@127.6.76.195:0 shutdown complete.
I20260709 12:31:57.786944  6451 master.cc:562] Master@127.6.76.254:37141 shutting down...
I20260709 12:31:57.823998  6451 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:57.824759  6451 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:57.825196  6451 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3de6dfcb5254489fbcc080689c2427a5: stopping tablet replica
I20260709 12:31:57.848984  6451 master.cc:584] Master@127.6.76.254:37141 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (21753 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (43585 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (498703 ms total)
[  PASSED  ] 8 tests.
I20260709 12:31:57.919410  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2231 messages since previous log ~33 seconds ago
I20260709 12:31:57.919792  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2231 messages since previous log ~33 seconds ago
I20260709 12:31:57.920020  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2223 messages since previous log ~33 seconds ago
I20260709 12:31:57.920280  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~52 seconds ago
I20260709 12:31:57.920483  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~52 seconds ago
I20260709 12:31:57.920718  6451 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~152 seconds ago
I20260709 12:31:57.920993  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 1 messages since previous log ~152 seconds ago
I20260709 12:31:57.921208  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~135 seconds ago
I20260709 12:31:57.921481  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 1 messages since previous log ~377 seconds ago
I20260709 12:31:57.921823  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 16 messages since previous log ~0 seconds ago
I20260709 12:31:57.922204  6451 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~20 seconds ago
